Basic Text Processing

The scripts we built from Chapter 5 through Chapter 8 can already read input, repeat processes via loops, and wrap repetitive logic into functions. However, one requirement that almost always arises as soon as scripts touch real-world scenarios has not been fully covered: how to search, filter, and transform text itself, whether stored in log files, command outputs, or strings inside variables. Sysadmins need this to scan thousands of log lines searching for a specific error. Developers need this to validate input formats before further processing. Regular users need this simply to find which files contain a specific keyword. This chapter covers the foundations of text processing in Bash: a conceptual map of the three primary tools that will accompany us throughout the rest of this series (grep, sed, awk), how to search for patterns using grep along with its most commonly used options, essential basic regular expressions for daily search needs, and Bash's built-in string manipulation without invoking external tools. In-depth discussions of sed and awk follow in Chapter 10 and Chapter 11.

9.1 Conceptual Map of Text Processing Tools in Bash

Before diving into practice, we need to understand the big picture of the three text processing tools most frequently used alongside Bash. This enables us to select the right tool for a specific task rather than forcing a single tool onto every case. None of the three are Bash builtins; rather, they are external utilities available on virtually all Linux distributions, macOS, and WSL, but they are routinely called from within Bash scripts as an integral part of daily workflows.

9.1.1 grep for Pattern Searching

grep is a tool with a single primary job: search for lines matching a pattern and display those lines. The name grep originates from the early Unix ed editor command g/re/p (globally search for a regular expression and print), and its behavior remains faithful to those origins today. grep is best suited when you only need to know "which lines contain X" without modifying their content.

9.1.2 sed for Modifying Text as a Stream

sed (stream editor) processes text line by line like grep, but its primary task is altering text content rather than merely finding it, such as replacing all occurrences of a word with another word in a configuration file. sed is covered thoroughly in Chapter 10.

9.1.3 awk for Processing Structured Data

awk is a dedicated text-processing language operating on column fields. It excels when data possesses a repetitive structure, such as CSV files or log files where each line is delimited by spaces or commas into meaningful columns. awk is covered thoroughly in Chapter 11.

All three are frequently combined using pipes (Chapter 5.3) within a single workflow, for instance filtering relevant lines first with grep, then processing them further with awk. The following table summarizes when each tool is most appropriate.

9.2 grep: Searching for Patterns in Files and Output

This section covers how to use grep to search for patterns, both directly within files and in the output of other commands via pipes, along with the options most frequently used in practice.

9.2.1 Basic Syntax and Searching Within Files

The most basic form of grep accepts two arguments: the search pattern and the target file.

grep "error" /var/log/aplikasi.log

The command above displays every line in /var/log/aplikasi.log that contains the text error, performing a case-sensitive search by default (uppercase and lowercase letters are treated as distinct). grep can also accept multiple files simultaneously. When more than one file is provided, grep automatically prefixes each matching line with the filename so you know which file the result came from.

grep "error" /var/log/aplikasi-1.log /var/log/aplikasi-2.log

9.2.2 Searching Within Output of Other Commands

grep also reads from stdin (Chapter 5.2) if no file argument is provided, allowing it to be piped directly from the output of another command, one of the most common usage patterns in production.

journalctl -u nginx --no-pager | grep -i "failed"
ps aux | grep "nginx"

The journalctl command for inspecting systemd logs will be discussed further in Chapter 19. The second example, ps aux | grep "nginx", contains a classic gotcha every Sysadmin should know: the grep process itself appears in the ps aux output because its own command-line argument contains the word nginx. As a result, the output sometimes shows a "ghost" line that is actually just the running grep process. A common traditional trick to avoid this is wrapping one character of the pattern in a character class (discussed in 9.3.2), such as grep "[n]ginx", because the grep command line itself contains the literal string [n]ginx which does not match the pattern. A cleaner alternative is using pgrep nginx, a separate utility specifically designed to search for processes by name without this issue.

9.2.3 Options -i, -v, and -c

These three options are the most frequently used in daily work. -i makes the search case-insensitive (ignoring uppercase and lowercase differences).

grep -i "error" /var/log/aplikasi.log

The command above matches error, Error, ERROR, and other capitalization variants simultaneously. -v inverts the search logic, displaying lines that do not match the pattern, which is useful for stripping comment lines from configuration files.

grep -v "^#" /etc/ssh/sshd_config

The pattern ^# above means "lines starting with the # character", a regex detail discussed fully in 9.3.2. -c displays the count of matching lines rather than the line contents.

grep -c "error" /var/log/aplikasi.log

One important detail regarding -c: this option counts the number of lines containing the pattern, not the total occurrences of the pattern. A line containing the word error three times is still counted as one matching line, not three.

9.2.4 Options -n, -r/-R, -w, -l, and -o

-n displays the line number before each result, useful for pinpointing the exact location of a line before editing it further, such as with sed in Chapter 10.

grep -n "error" /var/log/aplikasi.log

-r (or -R) enables grep to search recursively into a directory and all of its subdirectories. This is used when searching for a pattern across many files at once, such as scanning all scripts in a project directory.

grep -r "TODO" ~/scripts/

A subtle difference between -r and -R lies in how they handle symbolic links: -r follows symbolic links only if the link itself is explicitly passed as a command-line argument, whereas -R always follows all symbolic links encountered during recursive traversal. This distinction rarely affects daily usage, but it is important to keep in mind when working with directory structures that rely heavily on symlinks, such as the /etc directory on certain distributions.

-w matches the pattern only as a whole word, not as part of another word.

grep -w "cron" /var/log/aplikasi.log

Without -w, the pattern cron would also match words like crontab or anacron because both contain the substring cron. Adding -w ensures that only standalone occurrences of cron count as matches. -l outputs only the names of files containing the pattern without showing line contents, making it ideal to combine with -r to find which files require further inspection.

grep -rl "deprecated" ~/scripts/

-o outputs only the specific portion of text matching the pattern rather than the entire line. This is exceptionally useful when paired with regex in 9.3 to extract specific data snippets from a line, such as an IP address from a long log line.

9.2.5 Troubleshooting: Quoting Patterns to Prevent Shell Processing

Search patterns in grep must always be enclosed in quotes, adhering to the same variable quoting rules discussed in Chapter 3.4. Without quotes, the shell might interpret characters like *, $, [, and ? as filename expansion patterns (Chapter 2.2.5) before passing them to grep as search patterns.

grep [error] /var/log/aplikasi.log
grep "[error]" /var/log/aplikasi.log

The first line risks failing or producing unexpected results if a file matching the pattern [error] happens to exist in the working directory, because the shell expands it before grep ever sees it. The second line is safe because quotes prevent the shell from processing characters inside them altogether. This habit should be strictly maintained, especially when search patterns originate from variables with dynamic content, such as user input.

grep "$pola_pencarian" /var/log/aplikasi.log

Quoting patterns also serves as a primary defense mechanism against unexpected behavior when input originates from untrusted sources (such as an attacker attempting to inject special characters), a topic covered in detail under script security in Chapter 23.

9.3 Basic and Extended Regular Expressions

This section covers regular expressions (often abbreviated as regex), a sequence of characters defining a text search pattern sufficient for daily search tasks using grep. Regex is also the foundation used by sed and awk in Chapter 10 and Chapter 11, so understanding it here will be directly applicable to the next two chapters.

9.3.1 What is a Regular Expression

Unlike standard literal text searches that only match exact characters, regex uses special characters called metacharacters to describe more flexible patterns, such as "lines starting with a digit" or "any word containing three consecutive letters". grep uses regex for pattern matching by default rather than plain literal search, although for patterns without metacharacters, the result is identical to a literal search.

9.3.2 Core Metacharacters: . * ^ $ and [ ]

The following metacharacters are those most frequently used for daily search tasks.

Character classes can also be negated using ^ as the first character inside the brackets, meaning "any single character except those specified".

grep "[^0-9]" file_angka.txt

The pattern above matches lines containing at least one non-numeric character, which is useful for validating whether a file expected to contain only numerical data actually contains unexpected characters.

9.3.3 BRE vs ERE and the -E Option

grep defaults to a regex dialect called BRE (Basic Regular Expressions), one of two regex dialects defined by POSIX specifications. The other dialect, ERE (Extended Regular Expressions), is enabled via the -E option. The most practical difference between the two lies in the metacharacters +, ?, |, (, ), {, and }: in BRE, these characters are treated as literals unless preceded by a backslash to escape them into metacharacters. In ERE, the opposite applies: these characters act as metacharacters directly without backslashes.

grep 'gagal\|error' /var/log/aplikasi.log
grep -E 'gagal|error' /var/log/aplikasi.log

Both lines above produce identical search results: the first line uses BRE with an escaped \|, while the second uses ERE with -E without backslashes. The egrep variant command commonly found in older tutorials is actually just an alias for grep -E, and GNU grep considers it deprecated. Thus, grep -E is the recommended form for new scripts. In practice, choose ERE via -E whenever your pattern involves alternation (|) or repetitions (+, ?, {n,m}), as the syntax is far easier to read compared to the backslash-heavy BRE version.

9.3.4 Case Study: Filtering Configuration and Log Lines

A very common pattern used by Sysadmins is filtering configuration files to display only active lines, removing comment lines and blank lines simultaneously.

grep -v "^#" /etc/ssh/sshd_config | grep -v "^$"

The pattern ^$ above means "a line that ends immediately as it begins", in other words, a blank line containing no characters. Another example is extracting log lines that resemble IPv4 addresses using ERE and the -o option from 9.2.4.

grep -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' /var/log/akses.log

Admittedly, the pattern above only checks for four groups of digits separated by dots, rather than performing full validation that each group falls strictly within the 0-255 range per true IPv4 rules. Consequently, strings like 999.999.999.999 still match. For quick searching and filtering, a simple pattern like this is usually sufficient, but for strict IPv4 validation, an awk-based approach or additional checks in Bash itself (Chapter 11) are far more appropriate.

9.4 Built-in Bash String Manipulation

This closing section covers Bash's native ability to manipulate strings without invoking external tools like grep, using parameter expansion syntax (some forms of which were briefly introduced in Chapter 3.4). For simple string operations, using built-in Bash features is much faster than spawning external processes, as there is no overhead from executing a separate program.

9.4.1 String Length with ${#var}

The syntax ${#variable_name} returns the character count in a string.

nama_host="server-produksi"
echo "${#nama_host}"

The command above prints 16, the character count in server-produksi. A common real-world use case is validating user input length before further processing, such as ensuring a hostname does not exceed a specific character limit.

if [ "${#nama_host}" -gt 63 ]; then
    echo "Error: nama host maksimal 63 karakter" >&2
    exit 1
fi

9.4.2 Substring Extraction with ${var:offset:length}

The syntax ${var:offset:length} extracts a substring starting from offset (0-indexed) for length characters. The :length part is optional; if omitted, the result is the remainder of the string to the end.

tanggal="2026-08-30"
tahun="${tanggal:0:4}"
bulan="${tanggal:5:2}"
hari="${tanggal:8:2}"

echo "Tahun: $tahun, Bulan: $bulan, Hari: $hari"

The script above prints Tahun: 2026, Bulan: 08, Hari: 30, extracting each date component based on character offset in a YYYY-MM-DD formatted string. One important gotcha: extracting characters from the end of a string using a negative offset requires a space before the minus sign, or parentheses around it.

echo "${tanggal: -2}"
echo "${tanggal:(-2)}"

Without a space, ${tanggal:-2} is interpreted as a completely different "default value" syntax (as used in the Chapter 6.4.4 case study, ${1:-status}), meaning "use value 2 if tanggal is empty or unset", rather than "extract the last 2 characters". These two syntaxes look similar but have entirely different meanings, so the space (or parentheses) before a negative offset must not be omitted.

9.4.3 Replacing Text with ${var/pattern/replacement}

The syntax ${var/pattern/replacement} replaces the first occurrence of pattern in var with replacement, whereas ${var//pattern/replacement} (double slashes) replaces all occurrences.

path_file="/home/user/laporan-lama.txt"
echo "${path_file/lama/baru}"

The command above prints /home/user/laporan-baru.txt. For cases requiring replacement of all occurrences, such as normalizing delimiters in a string:

daftar="nginx,mysql,redis"
echo "${daftar//,/ }"

The line above replaces every comma with a space, resulting in nginx mysql redis. Two additional useful variations: ${var/#pattern/replacement} replaces only if pattern matches at the exact beginning of the string, and ${var/%pattern/replacement} replaces only if pattern matches at the exact end of the string.

nama_file="backup_database.tar.gz"
echo "${nama_file/%.tar.gz/.bak}"

9.4.4 Case Conversion: Uppercase and Lowercase

Since Bash 4.0, built-in syntax is available to convert letter casing without calling external utilities like tr. ${var^} capitalizes the first letter, ${var^^} converts all letters to uppercase, ${var,} lowers the first letter, and ${var,,} converts all letters to lowercase.

layanan="NGINX"
echo "${layanan,,}"

nama="produksi"
echo "${nama^}"

The first line outputs nginx, while the second outputs Produksi. A frequent production use case is normalizing user input prior to comparison, such as in y/n confirmation prompts (seen in Chapter 5.1), so that Y, y, or Yes are handled uniformly.

read -r -p "Lanjutkan proses? (y/n): " jawaban
jawaban="${jawaban,,}"

if [ "$jawaban" = "y" ]; then
    echo "Melanjutkan proses..."
fi

Because this feature was introduced in Bash 4.0, scripts requiring compatibility with legacy Bash versions (such as macOS default Bash which remains at version 3.2 as noted in Chapter 1.3.2 unless updated via Homebrew) should use tr '[:upper:]' '[:lower:]' as a more portable alternative.

9.4.5 Troubleshooting: Patterns in Bash are Globs, Not Regex

A common misconception: pattern in ${var/pattern/replacement} is not a regex like those used by grep in 9.3, but rather a glob pattern, the same pattern type used for filename expansion in Chapter 2.2.5 (*, ?, [...]). Regex metacharacters like +, |, or {n,m} are not recognized inside parameter expansion and will be treated as literal characters.

teks="tes123"
echo "${teks/[0-9]+/X}"

The pattern [0-9]+ above does not mean "one or more digits" as it would in regex. Since + is not a glob metacharacter, the pattern reads as "any single digit character followed by a literal + character", which clearly does not match tes123, returning the string unchanged. When text replacement demands true regex capability, such as repeating patterns or alternation, sed in Chapter 10 is the proper choice, as sed is specifically designed for regex-based text substitution rather than simple globs like Bash parameter expansion.