- Learn
- Computer Science & IT
- Information Technology
- Linux Basics
- Linux Text Processing
Linux Text Processing: Pipes and Redirection
BeginnerLesson 3 of 7 in Linux Basics Course
What you will learn
- Display short files in full, page through long files, and show only the first or last lines of a file.
- Display and page through files with cat and lessPrint short files with cat, read long files one screen at a time with less, and move and search inside less.
- Show the first or last lines with head and tailUse head and tail to show the start or end of a file, skip a header line, pick out a range of lines and follow a growing log file.
- Display and page through files with cat and less
- Explain the three standard streams of a command and redirect input, output and error messages to and from files.
- Understand standard input, output and errorDescribe the three standard streams, why output and errors are separate, and how a command can read its input from a file instead of the keyboard.
- Redirect output and errors to filesSave output and error messages to files by overwriting or appending, combine or discard streams, and avoid destroying a file by redirecting onto it.
- Understand standard input, output and error
- Connect small commands with pipes to count, sort, de-duplicate and cut text, answering questions about data in a single command line.
- Connect commands with pipes and teeUse the pipe to pass one command's output to the next, build and check a pipeline stage by stage, and save a copy along the way with tee.
- Count, sort and remove duplicate linesCount lines, words and bytes with wc, order lines alphabetically, numerically or by a field with sort, and remove or count duplicates with uniq.
- Extract columns with cutSelect fields from delimited text and characters from fixed positions with cut, and use it inside pipelines to answer questions about data.
- Connect commands with pipes and tee
- Find lines in files with grep, and describe the text to look for precisely with basic and extended regular expressions.
- Search files with grep and its main optionsUse grep to print, count, number or invert matching lines, search whole folders, and choose between pattern and fixed-text searches.
- Write basic regular expressionsBuild patterns from literal characters, the dot, anchors, bracket expressions and the star, and avoid confusing them with file-name wildcards.
- Use extended regular expressions with grep -EUse the operators ?, +, { }, | and ( ) with grep -E to match optional, repeated and alternative text, and print only the matching parts.
- Search files with grep and its main options
- Locate files anywhere in the directory tree with find and locate, and run commands on the files found.
- Search the directory tree with findUse find with name, type, size and time tests, combine tests, and avoid the quoting and permission problems beginners meet.
- Act on found files and use faster lookupsRun commands on every file find returns, delete matches safely, and know when locate or which is the quicker tool.
- Search the directory tree with find
- Open, change, save and close text files in the two terminal editors found on most Linux systems: the simple nano and the modal vi.
- Edit and save a file in nanoOpen a file in nano, type and change text, search and replace, cut and paste lines, and save and exit correctly.
- Work in vi's modesSwitch between vi's normal, insert and command-line modes, move and edit with single-key commands, and save or quit without losing work.
- Edit and save a file in nano
Linux Text Processing study guide
1View files from the terminal
A text file may hold a few notes or thousands of settings. Choose a view that fits what you need: the whole file, a screen you can move through, or a small part near either end. You will learn how to move and search without editing, select an exact line range, and watch text arrive in a growing file. These viewing commands also accept output from other commands, so the same skills help you inspect a long directory listing or a pipeline's results.
1.1 Display and page through files with cat and less
Short files are easy to print. Long files are easier to read when the display waits for you. Neither cat nor the normal reading view in less edits the file.
Print or join text with cat
cat notes.txt sends the entire file to standard output, normally your terminal. The name comes from concatenate: with several file names, it prints their contents in the order you give them. It does not insert a heading between files.
Suppose a.txt contains the lines "a" and "b", and b.txt contains "c d". Run:
cat -n a.txt b.txt
You see three numbered lines. The numbers continue across the two files; -n numbers the output, not the stored files. To save the joined text, use cat a.txt b.txt > ab.txt. The new file contains the three original lines without numbering. Including -n would save the numbers too.
Printing a large file may push its beginning beyond the visible screen. A pager keeps you in control of the view. less /etc/services opens a long text file and shows a screenful. It can show the first screen before reading the entire file, which helps with large files.
Move through less
The shell prompt is hidden while less is running. The program is waiting for a navigation key:
| Key | Action |
|---|---|
Space or f | Move forward one screen |
b | Move back one screen |
Down arrow, Enter, or j | Move down one line |
Up arrow or k | Move up one line |
g / G | Jump to the first / last line |
h / q | Show help / quit |
Uppercase matters: g and G go to opposite ends. To finish reading, press q; waiting for a new shell prompt without quitting can make the pager seem stuck.
Search inside the view
Type /, enter a word, and press Enter to search forward. For example, /ssh finds the next matching text. Type ?ssh and Enter to search backward. Then n repeats the search in its current direction, while N goes in the opposite direction. After a backward search, n continues backward.
The search is a regular expression. /^# finds a line beginning with "#" because ^ marks the start of a line. You can learn the pattern rules separately while using simple words now.
less -N notes.txt adds line numbers to the display. For wide records, less -S notes.txt keeps a long line on one display row by cutting off the visible right end; Left and Right arrows move sideways. The text remains in the file. Without -S, long lines wrap onto extra screen rows. more is an older pager with fewer navigation choices.
Page output from another command
ls -l /usr/bin | less lets you browse the long listing. The pipe delivers the listing to the pager. For an existing file, less notes.txt opens it directly; cat notes.txt | less introduces an extra command for the same view.
Use these commands on text. Printing a binary program such as /bin/ls can show control characters and garble the terminal. If that happens, type reset and press Enter to restore the display.
Key Takeaways
catprints files in the order supplied;-nnumbers the combined output.lesspages text without editing it;qreturns to the shell.- In
less,/searches forward,?searches backward, andnrepeats the current direction. less -Nshows line numbers;less -Slets you scroll across wide lines.
1.2 Show the first or last lines with head and tail
When you need only a file's opening or latest entries, printing everything hides the useful part. head selects the beginning, and tail selects the end. Both print 10 lines by default.
Choose a number of lines
If numbers.txt contains 1 through 100, one number per line, head numbers.txt prints 1 through 10, while tail numbers.txt prints 91 through 100. Use -n to change the count:
head -n 3 numbers.txt
tail -n 3 numbers.txt
The first command prints 1, 2, 3. The second prints 98, 99, 100. If a file has fewer than the requested lines, the command prints what is available.
head -c 5 selects five bytes instead of five lines. For text beginning with "banana", it prints "banan". Bytes are storage units; with UTF-8 text, a character may use more than one byte, so a byte count is not always a character count.
Read a starting position correctly
A plus sign changes the meaning of tail -n. Compare:
| Command | Meaning |
|---|---|
tail -n 2 file | Print the last two lines |
tail -n +2 file | Print from line 2 through the end |
tail -n +2 scores.csv drops the first line, useful when that line holds column headings. It does not pick only the second line. On the numbered file, tail -n +98 numbers.txt prints 98 through 100.
GNU head also accepts a negative count: head -n -97 numbers.txt omits the last 97 lines, leaving 1 through 3. The sign belongs to the count and is part of the instruction.
Select a middle range
To display lines 11 through 20, first keep everything through line 20. Then keep the last 10 lines of that selection:
head -n 20 numbers.txt | tail -n 10
After head, the data is 1 through 20. The last ten lines are 11 through 20. An inclusive range contains the end number minus the start number plus one lines; that final one accounts for including both ends. Using tail -n 11 would also include line 10.
Both commands can read a pipe when no input file is named. ls /etc | head -n 5 limits a listing to its first five names. With ls -lt | head -n 4, remember that the first output line is the "total" line: only three entries follow it.
Watch a file grow
tail -f /var/log/app.log first displays the final lines, then waits and prints text appended to the file. Use it while a program writes new entries. Press Ctrl+C to stop following and return to the shell. The waiting view is intentional; it is different from the completed one-time output of plain tail.
tail -F also retries and follows the file name when a log is removed and created again during rotation. With several input files, head and tail label each file's output with a header such as "==> fruit.txt <==". Add -q when you need the selected text without those headers.
Key Takeaways
headandtaildefault to 10 lines;-nsets a line count.tail -n +Kstarts at line K; it does not mean the last K lines.- For an inclusive range, keep the first ending-line count, then the last range-length count.
tail -ffollows appended text; Ctrl+C stops it.tail -Ffollows a file name through replacement, and-qsuppresses multi-file headers.
2Control where input, output and errors go
A command can produce useful results and error messages during the same run. Linux gives these separate routes, alongside a route for input. The shell can connect each route to a file before the command starts. Understanding those connections lets you save a report, keep errors in a separate file, and supply data without typing it. The order of the redirections matters, and an output file can be emptied before the program reads any input.
2.1 Understand standard input, output and error
A command communicates through standard streams. Thinking of input and output as separate routes explains how the same program can read a keyboard, read a file, or receive another command's output.
Identify the three streams
A file descriptor is the number used to identify an open stream. A normal shell command inherits these three:
| Number | Name | Usual terminal connection |
|---|---|---|
| 0 | Standard input, stdin | Keyboard input |
| 1 | Standard output, stdout | Normal results on the screen |
| 2 | Standard error, stderr | Errors and status messages on the screen |
Although stdout and stderr normally share the screen, they are separate connections. A program can report a result on stdout and explain a problem on stderr during the same run.
For example, ls /etc/hostname /nonexistent > out.txt lists one existing path and encounters one missing path. The listing goes to out.txt. The "No such file or directory" message remains on the terminal because > redirects stdout, not stderr. Seeing an error on the screen does not mean output redirection failed.
Give a command input from a file
Commands such as cat, sort, wc, grep, head, and tail can read stdin when no input file is named. Run sort alone, type a few lines, and press Ctrl+D at the beginning of an empty line. That signals the end of input, letting sort print the sorted lines. Ctrl+C stops the command instead of completing the input.
The input operator < connects a file to stdin:
wc -l < fruit.txt
If fruit.txt has six newline-terminated lines, the output is "6". Compare wc -l fruit.txt, which prints "6 fruit.txt". In the redirected form the shell opens the file, and wc receives a stream without the original file name. In the argument form, wc opens the named file itself.
The shell prepares redirections before it starts the program. sort < fruit.txt and < fruit.txt sort therefore connect the same file to the same input stream. The usual placement after the command is easier to read.
Supply a few lines directly
A here document places input below the command. The word after << marks where that input ends:
cat <<EOF > greeting.txt
Hello
See you soon
EOF
cat receives the two greeting lines on stdin and writes them to the output file. The last line contains only EOF; it is a delimiter, not part of the saved text. Another delimiter word can be used if the closing line matches it exactly.
In Bash, a here string supplies one string plus a newline. wc -w <<< "a b c" reports three words. These two forms supply input directly without first creating an input file.
Read the arrow before running the command
sort < fruit.txt reads the file. sort > fruit.txt opens it as an output destination, empties existing contents, and waits for input from the keyboard. The shell performs that destructive opening before sort runs. Keep input files separate from output destinations unless the command has a documented way to update its input safely.
Key Takeaways
- stdin, stdout, and stderr use file descriptors 0, 1, and 2.
<supplies a file through stdin;>changes stdout's destination.- Ctrl+D on an empty input line signals end of input; Ctrl+C stops the command.
- A here document uses a matching delimiter on a line by itself.
2.2 Redirect output and errors to files
Redirection changes a stream's destination. It is a shell operation, so the same operators work with many different programs. Start by deciding which stream to send and whether an existing file should be replaced or extended.
Save normal results or append them
echo one > log.txt creates the file or empties it before writing the line "one". A later echo two >> log.txt appends "two", leaving two lines. >> also creates a missing file.
| Operator | Effect |
|---|---|
> out.txt | Write stdout, replacing existing contents |
>> out.txt | Append stdout |
2> err.txt | Write stderr, replacing existing contents |
2>> err.txt | Append stderr |
ls /etc/hostname /nonexistent > out.txt 2> err.txt separates the successful listing from the missing-path error. The descriptor must touch the operator. ls 2> err.txt redirects errors; ls 2 > err.txt asks ls to list a file named "2" and redirects its normal output.
Combine streams in the correct order
2>&1 points stderr at stdout's current destination. Read redirections from left to right:
ls /etc/hostname /nonexistent > all.txt 2>&1
First, stdout moves to all.txt. Then stderr copies that connection. Both streams now reach the file. Bash's &> all.txt is a shorter form, and &>> all.txt appends both streams.
Reversing the order changes the result:
ls /etc/hostname /nonexistent 2>&1 > out.txt
First, stderr points where stdout currently points: the terminal. Then stdout moves to the file. Stderr keeps the earlier connection, so the error remains visible. 2>&1 makes a connection at that moment; it does not make stderr follow every later change to stdout.
Discard a stream deliberately
/dev/null discards data written to it. find / -name passwd 2>/dev/null leaves matching paths on stdout while hiding diagnostics such as "Permission denied". Those messages may identify unsearched folders, so discarding them also removes information about gaps in the search.
Protect an input file
Do not write sort fruit.txt > fruit.txt. The shell empties the destination before sort reads it. Use a different output path, check the result, and then rename it. For sort specifically, sort -o fruit.txt fruit.txt uses its supported output-file option to update that file safely.
Bash can also reject accidental overwriting:
set -o noclobber
With this setting, ordinary > fails for an existing regular file. >> still appends. Use >| for an intentional overwrite, or set +o noclobber to turn the setting off. The protection is useful when saving repeated reports, but it does not change a command's own file-writing options.
Key Takeaways
>overwrites, while>>appends; use descriptor 2 for stderr.> all.txt 2>&1captures both streams because redirections run left to right.2>&1 > out.txtleaves stderr at stdout's earlier destination./dev/nulldiscards the selected stream, including useful diagnostics.- Avoid redirecting onto an input file;
sort -oprovides a supported alternative. - Bash's noclobber rejects ordinary
>on existing regular files;>|overrides it.
3Combine commands into pipelines
Small text commands become more useful when their inputs and outputs fit together. A pipeline can remove a header, select a column, group repeated values, and count them without creating a file after every step. Each stage changes what the next one receives. Work through the stages separately so that you can tell whether a wrong total comes from the selection, the ordering, or the counting, and save intermediate results when you need to inspect them.
3.1 Connect commands with pipes and tee
A pipe, written |, joins the normal output of one command to the input of the next. You can connect several small jobs without saving a temporary file between them.
Follow the flow of text
In ls /etc | wc -l, ls writes names and wc -l counts the newline characters it receives. A command that reads stdin, selects or changes text, and writes stdout is called a filter. sort, uniq, wc, cut, grep, head, and tail can all serve as filters.
The general shape is command1 | command2 | command3. Each stage receives the previous stage's output, which may have a different form from the original file. A later stage cannot recover lines or columns that an earlier stage removed.
Build a count in stages
Suppose fruit.txt contains six lines: "banana", "apple", "cherry", "apple", "banana", "apple". To count the different names, first run:
sort fruit.txt
The output has three "apple" lines, two "banana" lines, and one "cherry" line. Equal names are now next to one another. Add a stage:
sort fruit.txt | uniq
uniq removes adjacent repeats, leaving "apple", "banana", and "cherry". Finally, sort fruit.txt | uniq | wc -l reports 3. Counting the original file directly would report six lines, which answers a different question.
Run each growing pipeline and inspect its result before adding the next command. For very long output, temporarily add | head to see the first ten lines. This makes it easier to spot a header that was not removed or a field that was selected incorrectly.
Keep errors separate
An ordinary pipe carries stdout, not stderr. With one existing path and one missing path, ls /etc/hostname /nonexistent | wc -l counts the one normal listing line. The error still appears on the terminal.
To include diagnostics in the data, use ls /etc/hostname /nonexistent 2>&1 | wc -l, which counts two lines in this example. Bash's |& combines stdout and stderr into the pipe as well. Whether including errors is useful depends on what you want the final count to mean.
The stages run concurrently. tail -f /var/log/app.log | grep ERROR displays newly appended matching lines on the terminal. This is a live view, not a process that waits for the file to finish. Programs can buffer output, however; immediate display is not a general promise for every possible pipeline.
Save a copy along the route
tee writes its input both to a file and to stdout:
sort fruit.txt | uniq | tee unique.txt | wc -l
The terminal shows 3, while unique.txt stores the three names. tee -a appends instead of replacing the saved file. Placing tee earlier would save an earlier form of the data.
A pipe connects to a command, while > names a file. ls > less creates a file named "less"; it does not start the pager. The receiving command must also read stdin: piping text into echo or ls does not turn that text into their arguments.
Key Takeaways
|connects stdout to the next command's stdin.- Check a pipeline one stage at a time; each stage receives the previous stage's output.
- stderr stays outside an ordinary pipe unless you combine the streams.
teesaves input and passes it onward;tee -aappends.- Piped text is stdin, not a list of command arguments.
3.2 Count, sort and remove duplicate lines
Counting and ordering are separate jobs. Before using a count, decide whether you need records, words, bytes, or distinct values. Before removing duplicates, decide whether equal lines are already next to one another.
Count the right thing with wc
Plain wc file reports lines, words, and bytes, in that order, then the file name. The six-line fruit list containing "banana", "apple", "cherry", "apple", "banana", "apple" gives "6 6 39 fruit.txt". Each line contributes its letters plus a newline byte.
| Option | What it counts |
|---|---|
-l | Newline characters |
-w | Words separated by whitespace |
-c | Bytes |
-m | Characters |
A final piece of text without a newline is not counted by wc -l. Text containing "one", a newline, and then "two" therefore has a count of 1. In UTF-8, "café" followed by a newline uses six bytes but five characters because "é" uses two bytes. With several input files, wc prints their counts and an additional total.
Choose text or numeric sorting
sort writes ordered lines without changing the input file. Its default order follows the current language setting. sort -f compares without distinguishing uppercase from lowercase.
Text sorting compares characters: "100", "25", "9" appear in that order because their first digits are "1", "2", "9". sort -n compares numeric values, giving 9, 25, 100. Add -r for reverse order, as in sort -rn. sort -h understands size suffixes such as "2K", "300M", and "1G".
Sort on a field
For comma-separated records, -t, sets the separator. -k3,3nr selects only field 3, compares numerically, and reverses the order:
tail -n +2 scores.csv | sort -t, -k3,3nr | head -n 3
Given "name,city,score" followed by "Ana,Lisbon,82", "Ben,Oslo,75", "Chen,Lisbon,91", "Dara,Oslo,68", and "Eli,Lisbon,88", the result is Chen, Eli, then Ana with their complete records. The first stage drops the header. Otherwise its nonnumeric score key counts as zero during numeric sorting.
Without -t, blanks separate fields. -k2 extends the key from field 2 through the line's end; -k2,2 limits it to field 2.
Group duplicates before counting them
uniq compares each line with the adjacent run. On the unsorted fruit list, it leaves all six lines because no equal names are adjacent. sort fruit.txt | uniq prints the three different names; sort -u fruit.txt can also produce a distinct sorted list.
After sorting, uniq -c reports each run's count: "3 apple", "2 banana", "1 cherry", with spaces padding the counts. uniq -d prints one copy of each repeated value, and uniq -u prints only values that occur once. uniq -i ignores case in its comparisons.
For the most frequent values first, use sort fruit.txt | uniq -c | sort -rn. The initial sort groups identical lines; the final sort ranks their counts. Omitting the first sort counts scattered runs separately.
Key Takeaways
wc -lcounts newlines,-wwords,-cbytes, and-mcharacters.sort -ncompares numbers;-rreverses order;-hhandles size suffixes.sort -t, -k3,3nrsorts comma-separated records by field 3 numerically in descending order.- Remove a header before sorting data records.
uniqremoves adjacent repeats; sort first when equal values are scattered.uniq -ccounts runs,-dselects repeated values, and-uselects values occurring once.sort | uniq -c | sort -rngroups values, counts them, and ranks the counts.
3.3 Extract columns with cut
A record often holds more information than a question needs. cut selects fields or fixed positions from each line and writes the selection to stdout. It does not rearrange or edit the original file.
Select fields by their separator
A delimiter separates fields. In /etc/passwd, colons separate the user name, password placeholder, user ID, group ID, full name, home folder, and login shell. For this example record:
alice:x:1001:1001:Alice Smith:/home/alice:/bin/bash
cut -d: -f1,6 prints "alice:/home/alice". Field numbers begin at 1, and the delimiter remains between selected fields. cut -d: -f5 prints "Alice Smith"; the internal space is part of that field because the delimiter is a colon.
| Selection | Fields printed |
|---|---|
-f1,3 | Fields 1 and 3 |
-f2-4 | Fields 2 through 4 |
-f3- | Field 3 through the last field |
The list selects fields, not a new order. cut -d, -f3,1 still prints field 1 followed by field 3. By default the separator is a tab, so cut -f2 file selects the second tab-separated field.
A line with no delimiter is printed whole. For example, cut -f2 prints "no tabs here" unchanged because it has no tab. Add -s to suppress lines with no delimiter. A line containing delimiters but lacking a requested field is a different case; -s is not a general test for complete records.
Select fixed positions
cut -c1-5 selects positions 1 through 5. On the account record above, it prints "alice". Position selection suits fixed-width text, where the desired columns start at the same places on every line. Field selection suits records whose field lengths vary but whose separators stay consistent.
Check how separators behave
With cut -d' ' -f2, every single space is a separator. The text "a b" has an empty second field between its two spaces, so the result is an empty line. If repeated spaces are just padding between columns, tr -s ' ' | cut -d' ' -f2 first squeezes each run of spaces to one.
cut also treats commas inside quoted values as separators. A quoted CSV value such as "Smith, Ann" is not kept together automatically. Use simple delimiter-separated examples only when the field boundaries really match the delimiter rule.
Count values from a selected column
To count cities in scores.csv, start with cut -d, -f2 scores.csv. With the records for Ana, Ben, Chen, Dara, and Eli, this prints "city", "Lisbon", "Oslo", "Lisbon", "Oslo", "Lisbon". Then:
cut -d, -f2 scores.csv | tail -n +2 | sort | uniq -c | sort -rn
Dropping the first line removes "city". Sorting groups the three Lisbon entries and the two Oslo entries; uniq -c counts them, and the last sort places "3 Lisbon" before "2 Oslo". Forgetting the header stage creates an extra value, "1 city", because a heading is still text to these commands.
Key Takeaways
cut -dsets the delimiter and-fselects fields numbered from 1.- Field lists select fields in their original order;
-cselects fixed positions. - The default delimiter is a tab;
-somits lines with no delimiter. - Repeated spaces create empty fields, and
cutdoes not parse CSV quoting. - For frequency counts, remove the header, select the value, group equal lines, and count.
4Search text with grep and regular expressions
Searching text starts with deciding what counts as a match. You may need a literal phrase, a whole word, a complete line, or a pattern such as two letters followed by four digits. grep can display the matching lines or report their numbers, counts, and file names. Regular expressions describe the patterns, while quoting keeps the shell from changing them first. You will build from simple matches to grouped alternatives and repeated parts.
4.1 Search files with grep and its main options
grep searches text and normally prints every line containing a match. A useful search starts by choosing both the pattern and the kind of result you need: lines, counts, line numbers, or file names.
Start with a quoted pattern
The usual form is grep 'pattern' file. With no file argument, grep reads stdin. Searches are case-sensitive unless you add -i.
Suppose server.log has five entries: an INFO startup line, an ERROR disk-full line, a WARN retry line, a lowercase "error timeout" line, and an INFO backup line. grep 'ERROR' server.log prints only the uppercase ERROR line. grep -i 'error' server.log prints both error lines.
Quote patterns even when they look simple. grep 'disk full' server.log searches for the two words together. Without quotes, grep disk full server.log uses "disk" as the pattern and treats "full" as another file name. Quotes also keep the shell from expanding stars and other special characters before grep receives them.
Choose the output you need
| Option | Result |
|---|---|
-i | Match without distinguishing case |
-v | Print lines that do not match |
-c | Print the number of matching lines |
-n | Prefix matches with their line numbers |
-w | Require a whole-word match |
-x | Require a whole-line match |
Options combine: grep -ic 'error' server.log reports 2, while grep -n 'WARN' server.log identifies line 3. -c counts matching lines, not the number of occurrences inside them. A line containing "error error" contributes one to the count.
Consider the lines "error", "terror", "errors", and "Error found". Plain grep 'error' matches the first three. grep -w 'error' matches only the first because "error" is not a whole word in "terror" or "errors". Adding -i also matches "Error found". -x asks for an entire matching line, so "Error found" still fails a whole-line search for "error".
Search across files
When several files are searched, grep normally prefixes matches with file names. grep -r 'TODO' notes searches files throughout the folder tree. Add -l to print only names of files with matches: grep -rl 'TODO' notes. Uppercase -L lists files without a match. These are file-selection results, not counts of matching lines.
Decide whether the pattern is literal
By default, grep interprets a regular expression. A dot matches any single character, so '10.0.0.7' can also match "10a0b0c7". grep -F '10.0.0.7' file searches for the exact text. Escaping each dot, as in '10\.0\.0\.7', also makes the dots literal.
grep -q suppresses normal output and reports through its exit status: 0 for a match, 1 for no match, and normally 2 for an error in GNU grep. A missing input file is an error, not evidence that the pattern was absent.
In a pipeline, grep -i 'error' server.log | wc -l counts matching lines just as grep -ic does. The first form leaves room for other filters between selecting lines and counting them.
Key Takeaways
grepprints matching lines and is case-sensitive unless-iis used.- Quote the pattern so the shell passes it intact.
-ccounts matching lines;-nnumbers them;-vselects nonmatches.-wrequires a whole word and-xa whole line.-rsearches a tree,-llists matching files, and-Llists nonmatching files.-Ftreats a pattern as fixed text;-qreports a search through exit status.
4.2 Write basic regular expressions
A regular expression, or regex, describes text that can match. grep normally looks for a matching part anywhere in a line, then prints the whole line. Most letters match themselves; special characters describe positions, choices, or repetition.
Match a character or a position
The dot . matches one character within a line. With the lines "cat", "cart", "coat", "cut", "act", "scat", and "color", the pattern 'c.t' matches "cat", "cut", and "scat". The last contains "cat" as a matching part. "coat" has two characters between "c" and "t", so one dot cannot cover them.
Use anchors to constrain positions. ^ marks the start of a line and $ the end:
| Pattern | What it requires |
|---|---|
^c | A line starting with "c" |
t$ | A line ending with "t" |
^c.t$ | Exactly "c", one character, then "t" |
^$ | An empty line |
grep '^c.t$' words.txt prints only "cat" and "cut". Anchoring both ends rules out a longer line such as "scat".
Choose one character from a set
A bracket expression matches one character. [ao] means either "a" or "o", so 'c[ao]t' matches the "cat" portion of "cat" and "scat". [0-9] describes a digit. A caret immediately after the opening bracket negates the set: 'c[^a]t' matches "cut" but not "cat". This caret excludes a character; it does not mark a line's beginning there.
Named classes describe categories in the current language setting. [[:digit:]] selects a digit, [[:alpha:]] a letter, [[:alnum:]] a letter or digit, and [[:upper:]] or [[:lower:]] a letter of the stated case. [[:space:]] includes spaces, tabs, and other whitespace characters. Keep both pairs of square brackets: the class name sits inside the outer set expression.
Repeat the preceding item
* means zero or more occurrences of the item immediately before it. 'ca*t' can match "ct", "cat", or "caat". It also matches "act" because that line contains "ct", using zero occurrences of "a". To match any run of characters, combine dot and star as .*.
'^c.*t$' matches "cat", "cart", "coat", and "cut" in the word list. A filename wildcard and a regex use different star rules: the shell pattern *.txt describes names ending in ".txt", but the regex for that ending is '\.txt$'.
Make a special character literal
A backslash can remove a special meaning. \. matches a real dot, \* a star, and \[ an opening square bracket. [.] also selects a literal dot. grep '\.conf$' file selects lines ending in ".conf".
A configuration search can remove simple comments and empty lines with grep -v '^#' app.conf | grep -v '^$'. The first test recognizes comments only when "#" is the first character. Default basic syntax treats unescaped ?, +, braces, |, and parentheses as ordinary characters. GNU grep supports backslashed operator forms; grep -E provides extended syntax without those backslashes.
Key Takeaways
- grep searches for a matching part unless anchors require the whole line.
.matches one character;^and$anchor the start and end.- A bracket expression selects one character;
[^a]excludes "a". - Named classes such as
[[:digit:]]require both pairs of square brackets. *repeats the preceding item zero or more times;.*matches a run of characters.- Use
\.or[.]for a literal dot.
4.3 Use extended regular expressions with grep -E
Extended regular expressions add useful ways to express optional parts, repeated groups, and alternatives. Use grep -E so the pattern is read in this syntax. The dot, anchors, sets, and star keep their earlier meanings.
Count repetitions
An operator follows the item it modifies. That item can be one character, a bracket expression, or a group:
| Operator | Number of occurrences |
|---|---|
? | Zero or one |
+ | One or more |
{n} | Exactly n |
{n,} | At least n |
{n,m} | From n through m |
'logs?' matches "log" and "logs": only the "s" is optional. 'ca+t' requires at least one "a", so it matches "cat" and the matching part in "scat", but not "ct" inside "act". Omitting -E changes an unescaped question mark or plus sign into literal text in basic syntax.
For an entire code of two uppercase letters and four digits, use:
grep -E '^[A-Z]{2}[0-9]{4}$' codes.txt
"AB1234" and "QZ0042" match. "A12345" has only one leading letter, "ab1234" uses lowercase, and "XY98765" has an extra digit. The two anchors prevent a valid-looking section inside a longer code from qualifying. Adding -i would also allow the lowercase code.
Group alternatives
| means an alternative inside an extended pattern. 'cat|cut' searches for either word anywhere in a line. Parentheses set the boundaries of a group: 'c(a|o)t' chooses "a" or "o" between "c" and "t". A following quantifier applies to the whole group, so '(ab)+' matches one or more copies of "ab".
Keep the pattern quoted. In grep -E cat|cut words.txt, the shell interprets the unquoted vertical bar as a pipe between commands. In grep -E 'cat|cut' words.txt, it delivers that character to grep as part of one pattern.
Print matches instead of complete lines
-o prints only matching parts, one per output line. The command below extracts a dotted-number shape from a longer line:
grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}' server.log
The parenthesized part is one to three digits followed by a literal dot; {3} repeats it three times. The final part adds one to three digits. It extracts "10.0.0.7" from a line that contains it, but also accepts "999.999.999.999". Matching a shape does not check whether each number is within an address's valid range.
Remove blank and comment lines together
grep -Ev '^[[:space:]]*(#|$)' app.conf selects active settings. Read the pattern in order: start at the beginning, allow whitespace, then match either "#" or the end of the line. It therefore recognizes indented comments and whitespace-only lines as well as empty lines. -v keeps the lines that do not fit that description, such as "port=8080". The older command name egrep is obsolescent in GNU grep; use grep -E.
Key Takeaways
grep -Eenables unescaped?,+, braces,|, and parentheses as operators.- Quantifiers apply to the preceding character, set, or group.
- Use anchors around a pattern when the entire record must fit it.
|supplies alternatives and parentheses group them; quote the pattern.-ooutputs each matching part, not the whole line.- A shape-matching regex may still accept values that are invalid in the real system.
5Find files by name, type, size and age
A file search asks where an item is; a text search asks what is inside it. Start a file search in the right folder, then narrow it by name, type, size, or age. You can pass the matching paths to another command without breaking names that contain spaces. A current search of the directory tree and a quick database lookup have different strengths. Before applying an action to many matches, inspect the selection and understand when the action runs.
5.1 Search the directory tree with find
find walks a directory tree and tests the items it reaches. Choose the starting folder first: find project prints project itself and the entries below it. Adding tests narrows the selection.
Match names, paths, and types
find project -name '*.txt' selects names ending in ".txt" anywhere below the starting folder. Quote the wildcard so the shell passes it unchanged. -name is case-sensitive, so "Guide.TXT" needs -iname '*.txt' for a case-insensitive match.
The pattern must match the whole item's name, without its parent folders. -name 'guide' does not match "guide.txt"; -name 'guide*' does. Use -path '*/src/*' when the folder portion matters. find prints traversal order, so add | sort when you want ordered paths.
-type f selects regular files, -type d directories, and -type l symbolic links. find project -type f -name '*.txt' requires both the type and name tests to pass. A directory named "notes.txt" fails the type test even though its name fits.
Understand size comparisons
-size compares a size after rounding it up to the selected unit. c means bytes, k means 1,024-byte kibibytes, M means 1,048,576-byte mebibytes, and G means gibibytes. A leading plus means greater than, a minus means less than, and no sign means equal to the specified number of units.
For -size +100M, the rounded size must exceed 100 MiB units. A file one byte larger than 100 MiB rounds to 101 units and qualifies. A nonempty file smaller than 1 MiB rounds to one unit, so -size -1M finds only empty files. Use a byte test for an exact byte threshold, or a smaller unit for a useful small-file range.
Test age and limit depth
-mtime uses completed 24-hour periods since modification. -mtime -7 selects items modified less than seven days ago. -mtime +30 selects ages with more than 30 completed periods: at least 31 days. -mmin -30 uses minutes for recent changes. -user alice selects an owner, and -empty selects empty files or directories.
To examine the starting folder and its immediate entries without entering subfolders, use find project -maxdepth 1 -type f. Put -maxdepth before the tests.
Combine tests and preserve patterns
Tests written side by side mean AND. -o means OR, as in find project -name '*.txt' -o -name '*.md'. ! -name '*.c' negates a name test. For simple selection, keep the relationship between tests clear before adding actions.
An unquoted wildcard is expanded by the shell in your current folder. If only notes.txt matches there, find . -name *.txt becomes a search for that one name, missing other text files. With several local matches, the command may fail because extra names appear inside the test expression.
A search of protected folders may report "Permission denied". 2>/dev/null hides those errors but cannot make unreadable folders searchable. find checks the tree while running; a live file system can still change before a later command uses the returned paths.
Key Takeaways
findwalks from a starting path and prints items passing its tests.- Quote name wildcards;
-nametests the whole name and-pathtests the full path. - Use
-type f,d, orlfor regular files, directories, or symbolic links. - Size tests round up before comparison;
-size -1Mselects only empty files. -mtime +30starts at 31 completed days, while-mminuses minutes.- Adjacent tests mean AND,
-omeans OR, and!negates a test.
5.2 Act on found files and use faster lookups
After selecting files, you can give their paths to another command. Keeping the selection and the action in the same find expression avoids treating spaces inside file names as separators between names.
Run a command on each match
The action -exec command {} \; runs the command once per matched item. find replaces {} with that item's path. For example:
find project -name 'main.c' -exec wc -l {} \;
If the match is project/src/main.c, this runs wc -l project/src/main.c. The escaped semicolon ends the action. The backslash keeps the shell from using the semicolon to end the entire shell command first.
Use {} + to pass batches of paths to fewer command invocations:
find notes -type f -name '*.txt' -exec grep -l 'TODO' {} +
grep receives the selected file paths as arguments and lists the ones containing "TODO". Both forms preserve a path such as "notes/team plan.txt" as one argument. The + form reduces startup work when many files match.
Combine file selection with text selection
find tests file properties; grep examines contents. Add -mtime -7 before the action to search only recently modified text files:
find notes -type f -name '*.txt' -mtime -7 -exec grep -l 'TODO' {} +
First the name, type, and age tests select files. Then grep searches their text. grep -r alone can search a tree's contents, but it does not replace those file-property tests.
Apply deletion only after checking the selection
-delete removes matches without confirmation or undo. First run the selection with no deleting action and read the paths. Only then add -delete at the end, as in find /tmp/build -type f -name '*.tmp' -delete. For a per-item confirmation prompt, -ok rm {} \; asks before running the removal command.
Actions run as the expression is evaluated. Putting -delete before -name '*.tmp' lets deletion happen before that name test can restrict it. A later test cannot protect an item from an action already taken.
Choose a faster lookup when it fits
locate guide.txt searches an index of stored paths containing "guide.txt". It can respond quickly without walking the disk. Because the database is refreshed periodically, a new file may be missing or a deleted path may remain. updatedb rebuilds the index; a system-wide update commonly needs administrator rights. Some distributions require an optional package such as plocate.
which grep looks through the folders in PATH for an executable program file, often reporting /usr/bin/grep. It does not describe every way Bash resolves a command. type echo, for example, can identify a built-in even when an external program named "echo" also exists. Use type commandname for Bash's interpretation, find for a current tree search, and locate for a quick indexed name lookup.
Key Takeaways
-exec command {} \;runs once per path;{} +batches paths.- Use
findfor file properties and grep for contents, combining them when needed. - Preview deletion matches, then put
-deleteafter the restricting tests. locatesearches an index that can be out of date.whichfinds executable files in PATH;typeexplains Bash command resolution.
6Edit text files in nano and vi
Terminal editors let you change notes and configuration files during a remote session or on a system without a graphical desktop. nano lets you type immediately and shows shortcuts at the bottom of the screen. vi assigns different jobs to keys in different modes, so changing modes is part of editing. Practice opening, finding text, making a small change, and saving or leaving deliberately; these steps matter as much as knowing a delete command.
6.1 Edit and save a file in nano
nano notes.txt opens text in a terminal editor. You can move with arrow keys and type directly into the text. If the file does not exist, nano starts an empty buffer, the working text held in memory; saving creates the file on disk.
Read the shortcut labels
The bottom lines show key combinations. ^ means Ctrl, so ^X means Ctrl+X. M- means Meta, usually Alt, so M-U means Alt+U. Ctrl+G opens help. These labels describe keys to press together, not characters to type into the file.
Make and save a small change
To change "port=8080" to "port=9090" in app.conf:
- Run
nano app.conf. - Press Ctrl+W, enter "8080", and press Enter.
- At the match, delete the four digits and type "9090".
- Press Ctrl+O, then Enter to confirm the file name and save.
- Press Ctrl+X to leave.
Back at the shell, grep 'port' app.conf should show "port=9090". This checks the saved file, not just what was visible in the editor.
Ctrl+O is Write Out, not an open-file command. Ctrl+R reads another file's contents into the current buffer at the cursor. Confusing those actions can insert unwanted text into the file you are editing.
Search, replace, and move
Ctrl+W searches for text. Alt+G goes to a requested line number, useful when an error message identifies a particular line. Ctrl+C shows the current line and column.
Alt+R, also available as Ctrl+, starts replacement. Enter the text to find and then its replacement. For each match, answer Y to replace or N to skip; A replaces all. Choosing individual replacements helps when the same digits appear in several settings and only one should change.
Move text and undo changes
| Shortcut | Action |
|---|---|
| Ctrl+K | Cut the current whole line |
| Ctrl+U | Paste cut text at the cursor |
| Alt+A | Start marking a selection |
| Alt+U / Alt+E | Undo / redo |
Repeated Ctrl+K presses collect consecutive lines. With a marked selection, Ctrl+K cuts the marked text instead of the whole line. Ctrl+U then inserts that text where the cursor is placed. Undo is useful immediately after an accidental cut or replacement.
Leave deliberately
Ctrl+X exits. If the buffer has unsaved changes, nano asks whether to save it. Y saves, followed by Enter to confirm the name; N leaves without saving those changes; Ctrl+C cancels the exit and returns to editing. Answering N is a decision to discard work, not a way to continue editing.
nano -l app.conf shows line numbers, and nano +12 app.conf starts at line 12. If saving fails because you lack permission, the edited buffer has not been saved to that path. System files require appropriate rights, for example sudo nano /etc/hosts when an authorized system change needs administrator access. For practice, use a file in your own folder.
Key Takeaways
- nano accepts typed text immediately; a new file is created when saved.
^means Ctrl andM-usually means Alt in shortcut labels.- Ctrl+O and Enter save; Ctrl+X exits and asks about unsaved changes.
- Ctrl+W searches, Alt+R replaces, and Alt+G goes to a line.
- Ctrl+K cuts, Ctrl+U pastes, and Alt+U undoes.
6.2 Work in vi's modes
vi notes.txt opens a terminal editor in normal mode. A key such as x is a command there. To type text, first enter insert mode. On many Linux systems, vi starts Vim or another vi-compatible editor.
Keep track of the mode
| Mode | Job | How to enter |
|---|---|---|
| Normal | Move, delete, copy, and issue commands | Esc |
| Insert | Type text into the buffer | i, a, o, or another insert command |
| Command-line | Enter commands such as saving | : from normal mode |
Vim shows "-- INSERT --" while inserting. Esc returns to normal mode. From normal mode, i inserts before the cursor and a after it. I starts before the line's first nonblank character, preserving indentation; A appends at the end. o opens a line below and O a line above.
Move without typing text
In normal mode, h, j, k, and l move left, down, up, and right. Arrow keys also work. w moves to the next word, 0 to the line's beginning, and $ to its end. G jumps to the last line. 1G jumps to the first; Vim also offers gg. A number before G selects a line, so 12G jumps to line 12. :12 and Enter does the same job. Counts also repeat movements: 5j moves down five lines.
Edit from normal mode
x deletes the character under the cursor. dd cuts a line; 3dd cuts three lines. yy copies a line. For whole-line text, p pastes below the current line and P above it. For character text, p pastes after the cursor and P before it.
cw replaces the rest of the current word and enters insert mode. r followed by a character replaces just one character. u undoes a change; Vim's Ctrl+R redoes it. A dot . repeats the last change.
Find and change a setting
To replace "8080" with "9090" in app.conf, run vi app.conf, then type /8080 and Enter. At the first "8", type cw, type "9090", and press Esc. Type :wq and Enter to save and leave.
Search backward with ?text and Enter. n repeats the current search direction and N reverses it. Searches accept regular expressions. To replace every occurrence throughout a file, :%s/8080/9090/g uses % for all lines and g for all occurrences on each line. :set number shows line numbers while editing.
Save or quit on purpose
From normal mode, :w and Enter saves; :w newname.txt saves under a new name. :q quits only if no unsaved changes prevent it. :wq, :x, or ZZ saves and quits. :q! discards unsaved changes and quits.
If typing unexpectedly moves the cursor or deletes text, press Esc to establish normal mode, then use u to undo unwanted edits before continuing. Enter insert mode before typing the replacement text. When leaving an editing attempt, choose whether to save or discard; the exclamation mark in :q! authorizes discarding unsaved work.
Key Takeaways
- vi starts in normal mode; enter insert mode to type and press Esc to return.
i,a,o, and their uppercase forms choose where insertion begins.ddcuts a line,yycopies it, andpputs whole-line text below or character text after the cursor.uundoes, and.repeats the last change./textsearches forward;nrepeats andNreverses the search direction.:wqsaves and quits;:q!discards unsaved changes and quits.
Linux Text Processing examples
1Read a file without losing your place
10 steps
- 1IntroductionChoose how much text to see
A terminal can show an entire file, a screen at a time, or just a selected part. The choice changes how easily you can find what matters.
Use a short note to learn
cat, then move tolessfor a file too long to keep on screen. Finally, useheadandtailwhen you know the part you need. These commands read text without editing the stored file. - 2ExampleJoin two short notes
a.txtcontains "a" and "b" on separate lines.b.txtcontains "c d". Runcat -n a.txt b.txt.The output has three numbered lines, in that order. Numbering continues from the first file into the second; the numbers are added to the output only. To save the original joined text without numbering, run
cat a.txt b.txt > ab.txt. - 3ExplanationOpen a reading view
Run
less /etc/servicesto read a long file one screen at a time. The program keeps the view open and waits for navigation keys.Space goes forward a screen,
bgoes back, and arrow keys move through lines.gjumps to the first line;Gto the last. Pressqto close the view and return to the shell. A missing shell prompt is normal while the pager is open. - 4ExampleSearch and continue
In
less, type/sshand press Enter to search forward for "ssh". Pressnfor the next match in that direction orNto go the opposite way.A backward search starts with
?ssh. After that search,ncontinues backward. The direction comes from the search you started.less -N fileadds line numbers if you need to note the location of a match. - 5ChallengeFind the earlier match
You have searched forward for "listen" in a long file with less. You want the previous match, then want to leave the pager. What keys do you press?
---reveal---
Press
Nto move opposite to the forward search direction, thenqto quit. Lowercasenwould continue forward. Quitting is what restores the shell prompt; reaching a match or the end of the file does not close the pager. - 6ExplanationChoose a count or a starting line
head -n 4 fileprints the first four lines.tail -n 4 fileprints the last four. Without-n, each defaults to ten lines.A plus sign changes tail’s job:
tail -n +4 filebegins at line 4 and prints everything after it. For a file with one header line,tail -n +2 fileremoves that header. It keeps all data records, not just the final two. - 7ExampleSelect lines 11 through 20
Suppose a file contains the numbers 1 through 100, one per line. Start with
head -n 20 numbers.txt, which keeps 1 through 20.Add
| tail -n 10. The last ten lines of the selected input are 11 through 20. Both endpoints count, so this range contains ten lines. Asking tail for eleven lines would include line 10 as well, shifting the start backward. - 8ChallengeChoose another middle range
A file has at least 17 lines. Write a pipeline using head and tail that prints lines 12 through 17, including both ends.
---reveal---
Use
head -n 17 file | tail -n 6. First keep lines 1–17, then retain the last six of those: 12, 13, 14, 15, 16, and 17. Using a count of five would omit line 12 because the endpoints are included in the requested range. - 9ChallengeInspect the latest text and keep watching
You need to see the latest entries of a long application file and continue watching as new entries arrive. Which command fits, and how do you leave it?
---reveal---
Use
tail -f app.log, then Ctrl+C when finished. Plaintailwould show the current ending and exit.tail -Fis useful when the program replaces the file during rotation, because it follows the file name through replacement. - 10RecapUse the view that fits the task
For a whole short file, use
cat; for navigation and searching, openless. A pager remains open until you quit it.Use
headandtailfor the beginning, ending, or a middle range. Read a plus sign in tail’s line count as a starting position. For a live view, follow appended text withtail -fand stop with Ctrl+C. The selection you make also works when the text arrives through a pipe.
2Trace input, results, and errors
11 steps
Save this lesson to follow every Walkthrough step by step.
- 1IntroductionDraw three separate routes
- 2ExplanationName the standard streams
- 3ExampleFeed the same data through another route
- 4ChallengeComplete keyboard input
- 5ExampleSupply several lines below a command
- 6ExplanationChoose replacement or addition
- 7ExampleCombine results and diagnostics
- 8ChallengeReverse the connections
- 9TipKeep the input intact
- 10ChallengeCount input and separate a diagnostic
- 11RecapTrace the shell’s setup
3Build a useful text report
14 steps
Save this lesson to follow every Walkthrough step by step.
- 1IntroductionGive each stage one job
- 2ExplanationConnect the two commands
- 3ChallengeChoose a pipe or a file
- 4ExplanationKnow what the counter measures
- 5ExampleSeparate sorting from counting
- 6ChallengeExplain a duplicate that survives
- 7ExplanationSelect the column that matters
- 8ExampleRemove the label and inspect the values
- 9ChallengeCheck a separator assumption
- 10ExampleGroup the selected values
- 11ExampleRank the counts
- 12ConnectionKeep evidence before the final count
- 13ChallengeDiagnose an extra category
- 14RecapCheck what travels between stages
4Build patterns that match the intended text
15 steps
Save this lesson to follow every Walkthrough step by step.
- 1IntroductionDecide what counts as a match
- 2ExplanationChoose a search report
- 3ExampleDistinguish a word from a line
- 4ChallengeSearch for an exact version label
- 5TipSearch a folder or ask a quiet question
- 6ExplanationPlace a match within the line
- 7ExampleSelect one character from a set
- 8Key InsightAllow zero copies
- 9ChallengeMatch a literal filename ending
- 10ExplanationTurn on extended operators
- 11ExampleCheck an entire code
- 12ExampleGroup a choice or a repeated part
- 13ChallengeExtract parts and inspect their limits
- 14ChallengeKeep only active settings
- 15RecapCheck the pattern and the report separately
5Select files before acting on them
12 steps
Save this lesson to follow every Walkthrough step by step.
- 1IntroductionSeparate names from contents
- 2ExplanationChoose the starting folder
- 3ExampleCombine type with name
- 4ChallengeKeep the wildcard for find
- 5ExplanationCompare rounded size units
- 6ChallengeRead the day boundary
- 7ExplanationPass paths as arguments
- 8ExampleSearch only selected contents
- 9TipPut a destructive action after its tests
- 10ExplanationChoose an index or a command lookup
- 11ChallengeChoose a current content check
- 12RecapMatch the search to the evidence you need
6Make and save a small terminal edit
13 steps
Save this lesson to follow every Walkthrough step by step.
- 1IntroductionDistinguish working text from saved text
- 2ExplanationRead nano’s controls
- 3ExampleFind and replace one value
- 4ChallengeMove a line and recover from a mistake
- 5TipHandle the exit prompt
- 6ExplanationEnter vi’s typing mode
- 7ExampleMove to the text you need
- 8ExampleChange the setting in vi
- 9ChallengePaste the kind of text you copied
- 10TipUse small edits and undo
- 11ChallengeChoose whether the vi edit should survive
- 12ChallengeCompare two unfinished edits
- 13RecapFinish with a deliberate save or discard
Linux Text Processing practice questions
Set 1: Read text and follow its flow10 Practice Questions
Question 1 of 10
docs/a.txt, a directory docs/b.txt, and a regular file docs/c.TXT. What does find docs -type f -name '*.txt' select?Show answerHide answer
a.txt meets both. b.txt fails the type test, while c.TXT would need the case-insensitive -iname test to qualify under the same suffix pattern.Question 2 of 10
Show answerHide answer
Question 3 of 10
less. What single key returns you to the shell?Show answerHide answer
q to quit the pager and return to the shell prompt. g moves to the beginning of the file but keeps the pager open, so it cannot restore the shell prompt. The absence of that prompt while reading means less is still handling your keys.Question 4 of 10
tail -n +3 records.txt print?Show answerHide answer
tail -n 3 would select the final three lines. Removing two opening lines and selecting three ending lines are different operations.Question 5 of 10
> report.txt, but its diagnostics remain visible. Which stream carries those diagnostics?Show answerHide answer
> operator redirects stdout alone, which explains why errors can remain on the screen. Sharing the terminal by default does not make the two output streams the same stream.Question 6 of 10
Show answerHide answer
>> operator appends stdout, preserving the file’s existing text and creating the file if needed. A single > opens the file after emptying its old contents. 2> redirects diagnostics instead of normal results, so it also selects the wrong stream for this task.Question 7 of 10
command | wc -l report?Show answerHide answer
wc -l, giving a count of 5. The two stderr lines retain their separate route and do not enter the counter. A count of 7 would require combining stderr with stdout before the pipe, which this command line does not do.Question 8 of 10
sort -n produce?Show answerHide answer
-n option changes the comparison; it does not merely reverse the default text order.Question 9 of 10
supplies.txt contains "clip", "ring", "clip", "pad", "clip", "ring", and "clip", each newline-terminated. What two rows lead sort supplies.txt | uniq -c | sort -rn | head -n 2, ignoring count padding?Show answerHide answer
Question 10 of 10
grep -ic 'fail' file count?Show answerHide answer
-w, which this command does not include.Set 2: Choose commands and interpret matches10 Practice Questions
Save this lesson to see every answer and practice every set.
Question 1 of 10
?queue in less, what does n do?Show answerHide answer
Save this lesson to see this answer.
Question 2 of 10
labels.txt has eight newline-terminated lines. How does the output of wc -l < labels.txt differ from wc -l labels.txt?Show answerHide answer
Save this lesson to see this answer.
Question 3 of 10
command 2>&1 > result.txt?Show answerHide answer
Save this lesson to see this answer.
Question 4 of 10
teams.csv contains "name,team", "Uma,Blue", "Zed,Gold", "Ivo,Blue", and "Moe,Gold", with final newlines. What does cut -d, -f2 teams.csv | sort | uniq | wc -l report?Show answerHide answer
Save this lesson to see this answer.
Question 5 of 10
build.txt for the literal text "v3.8", including the real dot?Show answerHide answer
Save this lesson to see this answer.
Question 6 of 10
grep '^d.g$' file?Show answerHide answer
Save this lesson to see this answer.
Question 7 of 10
grep '^ba*t$' file among "bt", "bat", "baat", and "boat"?Show answerHide answer
Save this lesson to see this answer.
Question 8 of 10
.txt files containing "REVIEW". What command applies all three restrictions?Show answerHide answer
Save this lesson to see this answer.
Question 9 of 10
Show answerHide answer
Save this lesson to see this answer.
Question 10 of 10
yy and press p. Where does the copied line appear?Show answerHide answer
Save this lesson to see this answer.
Set 3: Trace pipelines and correct mistakes10 Practice Questions
Save this lesson to see every answer and practice every set.
Question 1 of 10
runs.txt contains "b", "a", "b", "a", and "a", each on its own line. What leading count-value pair does uniq -c runs.txt | sort -rn produce?Show answerHide answer
Save this lesson to see this answer.
Question 2 of 10
cut -d: -f3,1 print from the line "Nia:ops:42"?Show answerHide answer
Save this lesson to see this answer.
Question 3 of 10
grep -E '^(go)+$' file?Show answerHide answer
Save this lesson to see this answer.
Question 4 of 10
Show answerHide answer
Save this lesson to see this answer.
Question 5 of 10
echo done > state.txt writes its output?Show answerHide answer
Save this lesson to see this answer.
Question 6 of 10
parts.txt contains "pin", "bolt", "pin", "nut", and "bolt", each newline-terminated. What does sort parts.txt | uniq | tee distinct.txt | wc -l do?Show answerHide answer
Save this lesson to see this answer.
Question 7 of 10
grep -Eo '[0-9]{1,2}' produce from that line?Show answerHide answer
Save this lesson to see this answer.
Question 8 of 10
-size -1M test?Show answerHide answer
Save this lesson to see this answer.
Question 9 of 10
locate plan.txt show a path that was deleted this morning?Show answerHide answer
Save this lesson to see this answer.
Question 10 of 10
Show answerHide answer
Save this lesson to see this answer.
Set 4: Diagnose file and text workflows10 Practice Questions
Save this lesson to see every answer and practice every set.
Question 1 of 10
Show answerHide answer
Save this lesson to see this answer.
Question 2 of 10
ls -lt produces one "total" heading followed by file entries. How many entries does ls -lt | head -n 6 display when at least six entries exist?Show answerHide answer
Save this lesson to see this answer.
Question 3 of 10
sort interactively, what signals end of input at the start of an empty line?Show answerHide answer
Save this lesson to see this answer.
Question 4 of 10
command 2>&1 | tee combined.txt | wc -l produce?Show answerHide answer
Save this lesson to see this answer.
Question 5 of 10
cut -d' ' -f2 print?Show answerHide answer
Save this lesson to see this answer.
Question 6 of 10
grep -icw 'alert' file report?Show answerHide answer
Save this lesson to see this answer.
Question 7 of 10
Show answerHide answer
Save this lesson to see this answer.
Question 8 of 10
find . -type f -name '*.cfg' -mtime +30 select?Show answerHide answer
Save this lesson to see this answer.
Question 9 of 10
Show answerHide answer
Save this lesson to see this answer.
Question 10 of 10
I begin insertion?Show answerHide answer
Save this lesson to see this answer.
Set 5: Combine rules in unfamiliar cases10 Practice Questions
Save this lesson to see every answer and practice every set.
Question 1 of 10
work, including no files from deeper folders. What search does that?Show answerHide answer
Save this lesson to see this answer.
Question 2 of 10
x deletes one character. You then press p. Where is that character inserted?Show answerHide answer
Save this lesson to see this answer.
Question 3 of 10
echo?Show answerHide answer
Save this lesson to see this answer.
Question 4 of 10
Show answerHide answer
Save this lesson to see this answer.
Question 5 of 10
log.txt and err.txt already contain older text. What happens with command >> log.txt 2> err.txt?Show answerHide answer
Save this lesson to see this answer.
Question 6 of 10
sizes.csv contains "name,size", "alpha,8", "beta,110", and "gamma,24". What pipeline prints the two complete data records with the greatest numeric sizes first?Show answerHide answer
Save this lesson to see this answer.
Question 7 of 10
jobs.csv contains "id,zone", "1,North", "2,South", "3,North", "4,East", and "5,North". What first count-value pair results from tail -n +2 jobs.csv | cut -d, -f2 | sort | uniq -c | sort -rn?Show answerHide answer
Save this lesson to see this answer.
Question 8 of 10
Show answerHide answer
Save this lesson to see this answer.
Question 9 of 10
grep '^r[^ae]d$' file among "rad", "red", "rid", and "road"?Show answerHide answer
Save this lesson to see this answer.
Question 10 of 10
grep -Ev '^[[:space:]]*(#|$)' file?Show answerHide answer
Save this lesson to see this answer.
Stuck on your own question? Homework Help
Key Terms
A
Alternation
| in extended syntax, often with parentheses to limit the choice’s scope.^(east|west)$, both alternatives must fill the whole line because they share the anchors.Anchor
^ for the start or $ for the end of a line in ordinary grep searches.d.g excludes a longer line such as "undog" even though it contains "dog".Append
>> for a command’s standard output.>> report.txt collect their results in one file instead of replacing it.B
Bracket expression
b[ae]t matches "bat" or "bet", while b[^a]t excludes "a" from the middle position.Buffer
C
cat
cat is a command that prints the contents of one or more files to standard output, following the order of its file arguments.cat -n a.txt b.txt numbers their combined output without editing either input file.Character class
[[:digit:]] selects a digit; both pairs of square brackets are needed.Command-line mode
: is typed in normal mode, running it when Enter is pressed.:wq saves the buffer and quits after Enter; a plain wq in normal mode does not do this.Concatenation
cat.cat first.txt second.txt > combined.txt, text from the first file precedes text from the second.cut
cut is a filter that selects fields or fixed character positions from each input line, preserving the selected parts in their original order.cut -d: -f1,6 /etc/passwd pairs each account name with its home folder field.D
Delimiter
cut -d selects the character used to split each line.Descriptor duplication
2>&1 does for stderr and stdout.command > all.txt 2>&1, stderr receives the file connection that stdout already has.E
Empty field
Escaping
\.txt$ requires a literal ".txt" ending, while .txt$ permits another character before "txt".Exit status
grep -q uses its status to report a result without printing matching lines.Extended regular expression
?, +, |, braces, and parentheses their special meanings without backslashes.grep -E before a grouped pattern such as ^(red|blue)$ to enable the alternative.F
Field
cut and selected according to the delimiter used in that record.File descriptor
2 in 2> errors.txt selects the error stream and must touch the redirection operator.Filter
sort, cut, and grep each act as filters when they receive text from an earlier command.find
find searches a directory tree from specified starting paths, tests each reached item’s properties, and prints or acts on selected paths.find notes -type f -name '*.txt' selects regular text-named files throughout that folder tree.find -exec
find -exec runs another command on selected paths, substituting {} and using an escaped semicolon or plus sign to end the action.-exec wc -l {} + passes batches of matching paths to a command that counts their lines.Fixed-string search
-F.grep -F "v2.4" releases.txt to require the actual dot in the version label.Fixed-width text
cut -c1-5 even if later text varies.Follow mode
tail waiting for appended data after its initial output, so new text can appear while a program writes to the file.tail -f app.log during an application check, then press Ctrl+C when you finish watching.Frequency count
sort and counting runs with uniq -c.sort -rn after uniq -c to put the largest counts first when inspecting common entries.G
grep
grep is a text-search command that normally prints each input line containing a match for a pattern, with options to change the report.grep -n WARN app.log to show matching lines together with their positions in the file.Group
(ab)+ permits "ab" and "abab" because the plus repeats the full two-character group.H
head
head is a filter that prints the beginning of its input, selecting ten lines by default or a different line or byte count when requested.head -n 4 notes.txt before deciding whether you need the whole file.Here document
cat <<EOF, the lines entered before the closing EOF are data, while that final line is a marker.Here string
wc -w <<< "red blue green" receives three words without needing a separate input file.I
Insert mode
i, a, or o and ending with Esc.cw removes the rest of a word, type its replacement in insert mode and then press Esc.L
less
less is a pager that lets you move forward and backward, search text, and view files or piped output without editing the text.less -N notes.txt when you want line numbers while locating a particular passage in a file.Line range
head, then the last ten of that selection.locate
locate is a fast path-name lookup that searches a prepared database, whose contents may lag behind changes to the live file system.M
Marked selection
Match extraction
-o option does for each nonempty match.grep -Eo '[0-9]+' prints the two numbers on separate lines.Metacharacter
* from the shell, but grep still interprets it as a regex operator when it receives it.Modal editor
Modification time
-mtime tests elapsed age in completed 24-hour periods.-mtime +30 because only 30 complete day periods have passed.N
Name test
-name distinguishes case while -iname ignores it.-name 'plan*' accepts "plan.txt", whereas -name 'plan' requires that exact base name.nano
nano is a terminal text editor that accepts typed text directly and displays shortcuts for saving, searching, cutting, and leaving.nano +12 app.conf when a program reports that a setting needs attention on line 12.Newline
wc -l counts these markers, so an unfinished final line does not add to its reported total.Noclobber
> redirection onto existing regular files, helping prevent accidental replacement.set -o noclobber, use >| only when you intend to override the existing-file protection.Normal mode
dd to cut the current line without accidentally typing those two letters.Null device
/dev/null, is a special file that discards written data, allowing a chosen output stream to be deliberately hidden.2>/dev/null hides diagnostics from a file search, including messages about unreadable folders.Numeric sort
sort -n to put "9" before "100" when the lines represent quantities instead of labels.P
Pager
less /etc/services, then press Space for the next screen or q to quit.PATH
PATH is the shell’s list of folders to search for external executable commands, which a tool such as which uses for a program-file lookup.which grep searches command folders instead of scanning every directory on the computer.Pipe
grep -i error app.log | wc -l, selected lines flow into the command that counts them.Pipeline
sort values.txt | uniq | wc -l, the final count measures distinct grouped values.Q
Quantifier
? for optional text or {4} for exactly four repeats.[0-9]{4} requires four digits; anchors can make that the entire line’s content.R
Recursive search
grep -r searches text in files across that directory tree.grep -rl TODO notes lists names of matching files without printing all their matching text.Redirection
sort < names.txt > ordered.txt, the shell supplies one file as input and opens another for output.Regular expression
^web[0-9]$ describes a whole line containing "web" followed by one digit.S
Search depth
-maxdepth limits how far traversal may descend into subdirectories.-maxdepth 1 to inspect the starting folder and its immediate entries without entering deeper folders.Sort key
sort -k and interpreted using the chosen separator.-k3,3nr compares field 3 only, numerically and in reverse order, in a score ranking.Standard error
> can leave a missing-path message visible on standard error.Standard input
sort < names.txt, the shell connects the file to the command’s standard input before it runs.Standard output
wc -l < names.txt > count.txt, the line count travels on standard output into the report file.Substitution
:%s/old/new/g replaces every matching "old" on every line of the file.T
tail
tail is a filter that prints the end of its input, or starts at a numbered line when the line count is preceded by a plus sign.tail -n +2 scores.csv to omit a one-line header while keeping all remaining data records.tee
tee is a filter that copies its input both to a named file and to standard output, saving a result while allowing a pipeline to continue.tee selected.txt before a counting stage when you also need to inspect the selected lines.Traversal
| sort to a plain path search when the order of its printed results needs to be predictable.Truncation
> redirection, before the command begins reading input.sort names.txt > names.txt loses the original data because truncation happens first.type
type is a Bash built-in that describes how a command name is interpreted, including an alias, function, built-in, or executable path.type echo can report a built-in even when which echo points to an external executable file.Type test
f for a regular file, d for a directory, or l for a link.-type f prevents a directory named "old.txt" from qualifying in a search for text files.U
uniq
uniq is a filter that detects adjacent equal lines, removing repeated copies or reporting selected runs and their counts through options.uniq -c makes each value’s full frequency appear as one count.updatedb
updatedb rebuilds the database used for indexed path lookups, making its stored view reflect the paths included during the refresh.sudo updatedb when an immediate system-wide refresh is appropriate.W
wc
wc is a counting command that reports newlines, words, and bytes by default, with options to select one measure or count characters.wc -c with wc -m to distinguish bytes from characters.Whole-word match
grep -w error file matches "error found" but rejects the substring inside "terror".Write Out
Y
Yank
yy does for a whole line without removing that line from the working buffer.yy followed by p to duplicate a line below its original position in the file.About this lesson
Make your own Study Materials
Turn your notes, slides and documents into Study Guides, Practice Questions, Flashcards and 6 more tools.