Text processing

grep, sed, awk, sort, uniq, cut, tr.

The real power of the Linux command line is text processing — small tools that filter, search, and transform text, combined with pipes. Master a handful and you can extract any answer from a log or a data file without writing a program. Each tool does one job; the art is chaining them.

grep — search for a pattern

grep prints the lines of a file that match a pattern. It is the single most-used text tool. -i makes it case-insensitive, -r searches a whole directory recursively, -n shows line numbers, -v inverts the match (lines that do NOT match), and -c counts matches. Point it at a file or pipe into it, and it finds the needle in the haystack.

$ grep error app.log
2026-07-03 10:14 ERROR db connection refused
2026-07-03 10:15 ERROR retry failed
$ grep -in error app.log          # -i case-insensitive, -n with line numbers
42:2026-07-03 10:14 ERROR db connection refused
43:2026-07-03 10:15 ERROR retry failed
$ grep -rn "TODO" ./src           # recursive search across a directory tree
./src/api.py:88:    # TODO: add rate limiting
$ grep -v "200" access.log        # every line that is NOT a 200 (i.e. the errors)

sort and uniq — order and deduplicate

sort orders lines alphabetically (or -n numerically, -r reversed). uniq collapses adjacent duplicate lines — and because it only looks at adjacent lines, you almost always sort first. The classic combination sort | uniq -c counts how many times each unique line appears, which is how you turn a raw log into a ranked summary ("which IP hit us most?").

# the classic "top talkers" pipeline: extract IPs, sort, count, rank
$ cut -d' ' -f1 access.log | sort | uniq -c | sort -rn | head -3
    842 10.0.2.1
    317 10.0.1.9
     45 10.0.1.5
#   └ count  └ the unique value (here, the IP)

cut — pull out columns

cut extracts pieces of each line. -d sets the delimiter (what separates the fields) and -f picks which field(s) to keep — so cut -d: -f1 /etc/passwd pulls the username (the first colon-separated field) from every account. It is the quick way to grab one column out of structured text.

$ cut -d: -f1 /etc/passwd | head -3     # first field, delimiter = colon
root
daemon
bin
$ echo "2026-07-03 10:14:22 ERROR" | cut -d' ' -f3   # the 3rd space-separated field
ERROR

sed — find and replace in a stream

sed edits text as it flows through. Its most common use is substitution: sed 's/old/new/g' replaces every occurrence of "old" with "new" (the g means "global", all matches on each line). By default it prints the changed text to the screen; -i edits the file in place. It is how you do a find-and-replace across a file from the command line, without opening an editor.

$ echo "log_level: info" | sed 's/info/debug/'
log_level: debug
$ sed -i 's/8080/9090/g' config.yml     # replace 8080 with 9090 IN the file (all occurrences)
$ sed -n '10,20p' app.log                # print only lines 10–20

awk — column-aware processing

awk is a small language for working with columns. It splits each line into fields ($1 is the first, $2 the second, and so on) and lets you print, filter, and compute on them. awk '{print $1, $4}' prints the first and fourth fields of every line; you can add conditions and even sum a column. When cut is not enough, awk is the next step up.

$ awk '{print $1, $3}' access.log | head -2   # print 1st and 3rd fields
10.0.1.5 GET
10.0.1.9 GET
$ awk '$3 == "500"' access.log                # only lines where field 3 is 500
$ awk '{sum += $5} END {print sum}' sizes.txt  # sum the 5th column