Daily Task Automation

Chapter 12 concludes Part 1 of this series with a comprehensive foundation in Bash scripting, covering variables, branching, loops, functions, and handling script arguments. Part 2 begins here with a more concrete direction: applying that foundation to automate daily tasks on a personal desktop or laptop. Users who frequently type the same commands repeatedly, whether to clean up filenames, back up important folders, or tidy up a cluttered Downloads folder, are performing tasks ideal for script delegation. This chapter covers four of the most common automation scenarios daily users encounter: pattern-based bulk file renaming, automated folder backups using rsync and tar, automated file organization by type, and setting up aliases and shell functions in .bashrc/.bash_profile so favorite commands do not need to be typed from scratch every time a new terminal session is opened.

13.1 Pattern-Based Bulk File Renaming

This section explains how to rename multiple files simultaneously using a consistent pattern, eliminating the need to click and retype filenames individually through a file manager.

13.1.1 Bulk Renaming Scenarios and Risks

A common scenario involves hundreds of photos transferred from a camera or smartphone still named IMG_2044.JPEG, IMG_2045.jpeg, with inconsistent case extensions or requiring a date prefix for easier sorting. Because Bash lacks a built-in bulk rename command, the most portable approach combines a for loop (Chapter 7.1) with mv (Chapter 2.2.4) executed sequentially inside the loop. While some distributions provide an additional rename tool, its syntax varies across implementations (the Perl-based version and the util-linux version operate differently). Therefore, this series avoids relying on it, maintaining compatibility across any Bash environment by using the standard for and mv combination.

Bulk renaming carries significant risks that must be understood before execution: by default, mv overwrites target files without confirmation if a file with the destination name already exists. There is no built-in trash bin or undo mechanism at the shell level to revert this operation. A safe practice, demonstrated in section 13.1.4, is to dry-run rename scripts in simulation mode before running them on actual files.

13.1.2 Renaming with For Loops and Parameter Expansion

The simplest scenario is standardizing file extensions, such as converting all .jpeg files to .jpg. Chapter 12.3.3 introduced ${var#pattern} to remove the prefix of a string. The current task requires removing the suffix of a string using its complementary operator, ${var%pattern}.

#!/bin/bash
for file in *.jpeg; do
    [[ -f "$file" ]] || continue
    mv -- "$file" "${file%.jpeg}.jpg"
done

${file%.jpeg} removes the shortest matching .jpeg suffix from the end of the file variable, leaving the base filename without its extension, onto which the new .jpg extension is appended. The line [[ -f "$file" ]] || continue utilizes the file test operator from Chapter 6.2.3 and continue from Chapter 7.3.2 as a safety mechanism in case the *.jpeg glob pattern matches no actual files, preventing issues caused by unmatched globs as detailed in Chapter 7.1.5. The double hyphen -- before "$file" informs mv that subsequent arguments are not flags, which protects against filenames starting with a hyphen.

For more advanced modifications, two additional parameter expansion forms are useful: ${var##pattern} and ${var%%pattern}. The difference between single (#/%) and double (##/%%) syntax lies in matching greediness: single operators remove the shortest match, whereas double operators remove the longest possible match. The table below summarizes all four variations.

The differences become apparent when the pattern contains a wildcard * and the string contains multiple potential matching points, such as a file named report.data.tar.gz.

name="report.data.tar.gz"
echo "${name%.*}"
echo "${name%%.*}"

${name%.*} evaluates to report.data.tar because the single % strips only the shortest matching suffix from the back (.gz). Conversely, ${name%%.*} evaluates to report because the greedy %% strips as far back as possible, leaving only the portion before the first period. Keeping this distinction in mind is crucial when handling files with compound extensions like .tar.gz, a concept revisited in section 13.3.2 when extracting extensions.

13.1.3 Complex Pattern Renaming with sed

While parameter expansion (13.1.2) covers basic patterns, tasks involving true regular expressions, such as replacing multiple spaces or special characters in filenames, are better handled using sed (Chapter 10). This is because Bash parameter expansion uses glob patterns rather than regex, as highlighted in Chapter 9.4.5.

#!/bin/bash
for file in *.txt; do
    [[ -f "$file" ]] || continue
    new_name=$(echo "$file" | sed -E 's/[[:space:]]+/_/g')
    mv -n -- "$file" "$new_name"
done

The pattern [[:space:]]+ combines a POSIX character class (Chapter 9.3.2) with the + quantifier (one or more), which, per Chapter 9.3.3, requires the -E flag to act as an unescaped metacharacter. Here, sed acts as a pure string transformer via a pipeline (Chapter 5.3.5) rather than modifying file contents in-place (such as with the -i flag in Chapter 10.3.1), since the operation targets the filename itself rather than its content. If this script needs to run on default macOS Bash installations prior to updating via Homebrew (Chapter 1.3.2), keep in mind the GNU sed vs BSD sed considerations from Chapter 10.3.2. While the -E flag is supported by both variants, target environments should always be verified before deployment.

Adding a date prefix to filenames does not require sed at all; standard command substitution (Chapter 3.4.4) using date is sufficient.

prefix="$(date +%Y%m%d)_"
mv -n -- "$file" "${prefix}${file}"

13.1.4 Dry-Run Verification Prior to Execution

A crucial best practice before running bulk renaming scripts on target files is performing a dry-run test: executing the script logic while displaying proposed changes without applying them to the filesystem. The simplest implementation of a dry-run mode in custom scripts is replacing the mv command with echo.

for file in *.jpeg; do
    [[ -f "$file" ]] || continue
    echo "mv -- '$file' '${file%.jpeg}.jpg'"
done

Execute this echo version first to review the output line by line. Once verified, replace echo with mv for actual execution. As an added defensive layer, the -n (--no-clobber) flag on mv prevents overwriting existing destination files, while -i (--interactive) prompts for user confirmation before overwriting, per official mv documentation. Incorporating either option into production scripts provides protection against unexpected naming collisions, as demonstrated in 13.1.3.

Final verification can be conducted using ls (Chapter 2.2.2) upon completion to ensure file counts and naming patterns match expectations. If a pattern error occurs before overwriting critical files, manually revert the improperly named files, refine the pattern logic in the script, and re-run the test.

13.2 Automated Backup of Critical Folders

This section explores two primary tools for automating desktop/laptop directory backups via shell scripts: rsync for file synchronization and tar for generating compressed archive files. Server-side usage, SSH integration, and retention rotation strategies for both tools are covered in Chapter 21; this section focuses on local daily backup requirements.

13.2.1 rsync vs tar: Selecting a Backup Strategy

rsync is a file synchronization utility: it copies a source directory to a target directory, and on subsequent runs, transfers only modified files. This makes routine backups significantly faster than recopying all data from scratch. tar (tape archive) operates differently: it packs the contents of an entire directory into a single archive file, typically compressed using gzip, creating a standalone snapshot easily transferable to external hard drives or cloud storage services.

13.2.2 Folder Synchronization with rsync

The basic syntax for performing local backups with rsync involves copying the contents of one directory to another.

rsync -av ~/Documents/ ~/Backup/Documents/

The -a (archive mode) flag is essential for backup operations. According to official rsync documentation, it acts as a shorthand for -rlptgoD: recursively scanning subdirectories, preserving symbolic links as links, and maintaining permissions, timestamps, group ownership, and file ownership. Note that -a does not preserve Access Control Lists (ACLs) or extended attributes by default (requiring -A and -X respectively), nor does it preserve hard links (which requires -H). Therefore, -a alone should not be assumed to cover advanced metadata requirements for high-security backup environments. The -v (verbose) option logs processed files, allowing real-time monitoring of the backup process.

The trailing slash on the source path (such as ~/Documents/ above) is significant under official rsync specifications: it instructs rsync to copy the contents of Documents directly into ~/Backup/Documents/ rather than creating a nested Documents directory inside the destination. Omitting the trailing slash on the source (rsync -av ~/Documents ~/Backup/) also results in ~/Backup/Documents/, but only because the Documents directory itself is copied as a directory entity into ~/Backup/. Explicitly including trailing slashes on both source and destination paths ensures predictable, consistent behavior.

To keep the destination synchronized with the source, including deleting files from the destination that no longer exist in the source, pass the --delete flag.

rsync -av --delete ~/Documents/ ~/Backup/Documents/

The --delete option carries operational risk: official rsync documentation recommends pairing it with --dry-run (or -n) prior to live execution, ensuring exact files targeted for removal in the target directory are reviewed beforehand.

rsync -av --delete --dry-run ~/Documents/ ~/Backup/Documents/

13.2.3 Compressed Archives with tar

For single-run snapshots, such as backing up a project directory prior to major changes, compressed tar archives are more practical than file synchronization. Combining tar with date via command substitution (Chapter 3.4.4) ensures each generated archive has a unique timestamped filename.

#!/bin/bash
timestamp="$(date +%Y%m%d-%H%M)"
tar -czf ~/Backup/documents-"$timestamp".tar.gz -C ~/ Documents

The -c flag creates a new archive, -z applies gzip compression, and -f sets the output filename. The uppercase -C ~/ option changes the working directory to ~/ before archiving Documents, ensuring relative paths (Documents/...) are stored in the archive rather than absolute paths like /home/user/Documents/.... Using relative paths provides flexibility when extracting archives to alternative locations, avoiding fixed absolute paths that force extraction back to the original filesystem locations.

13.2.4 Backup Rotation and Archive Verification

Daily backup scripts left running indefinitely will eventually consume available disk space with obsolete archive files. Implementing a retention rotation mechanism using find with the -mtime option resolves this issue.

find ~/Backup/ -name "documents-*.tar.gz" -mtime +30 -delete

Per GNU find documentation, -mtime +30 matches files modified more than 30 x 24 hours ago, calculated in 24-hour intervals. Because -delete permanently removes files, temporarily replace it with -print to inspect the targeted list before applying deletions, following the dry-run methodology covered in sections 13.1.4 and 13.2.2.

find ~/Backup/ -name "documents-*.tar.gz" -mtime +30 -print

Once an archive is created, verify that its contents can be read without errors rather than relying solely on file size checks.

tar -tzf ~/Backup/documents-"$timestamp".tar.gz

The -t flag lists archive contents without extracting files, providing a quick check for archive integrity before data recovery is needed. Effective backups run automatically without manual intervention; scheduling rsync or tar scripts via cron is covered in Chapter 15.1.

13.3 Automated File Organization

This section explores techniques for automatically categorizing files by extension, focusing on managing a Downloads directory where downloaded files typically accumulate.

13.3.1 Scenario: Managing an Unstructured Downloads Folder

The Downloads directory on personal machines often becomes a catch-all location containing .deb installers, .png images, .pdf documents, and .zip archives mixed together. Manually organizing these files is repetitive, but the pattern is consistent: specific extensions belong in designated subdirectories. This task is ideal for automation through scripting.

13.3.2 Categorizing Files by Extension

Associative arrays (introduced in Chapter 4.2) map file extensions to target subdirectories. Adding new categorization rules requires adding a key-value pair without modifying the processing loop, following design patterns from Chapter 4.4.3.

#!/bin/bash
source_dir="$HOME/Downloads"

declare -A target_folders=(
    ["pdf"]="Documents"
    ["docx"]="Documents"
    ["jpg"]="Images"
    ["png"]="Images"
    ["zip"]="Archives"
    ["tar"]="Archives"
    ["gz"]="Archives"
)

for file in "$source_dir"/*; do
    [[ -f "$file" ]] || continue

    filename=$(basename "$file")
    extension="${filename##*.}"
    extension="${extension,,}"

    destination="${target_folders[$extension]:-Others}"
    mkdir -p "$source_dir/$destination"
    mv -n -- "$file" "$source_dir/$destination/"
done

${filename##*.} uses greedy matching (section 13.1.2) to extract the text after the final period, matching the behavior described for report.data.tar.gz. As a result, files like archive.tar.gz yield gz as the extension rather than tar.gz. The line extension="${extension,,}" normalizes the extension string to lowercase using case conversion (Chapter 9.4.4), matching .PDF and .pdf to the same rule. The parameter expansion ${target_folders[$extension]:-Others} combines associative array access (Chapter 4.2.3) with a default fallback value (Chapter 6.4.4): if an extension is unlisted in target_folders, the file is routed to the Others folder instead of causing script errors or returning an empty string. Finally, mkdir -p (Chapter 2.2.3) creates target directories as needed without throwing errors if they already exist.

Grant execution permissions using the standard procedure from Chapter 1.4.3, then run the script and verify output via ls (Chapter 2.2.2):

chmod +x organize-downloads.sh
./organize-downloads.sh

ls ~/Downloads/Documents ~/Downloads/Images ~/Downloads/Archives

To run this process automatically (such as daily), schedule the script using cron (introduced for desktop environments in Chapter 15.1 and expanded for server management in Chapter 18).

13.3.3 Edge Cases: Files Without Extensions and Name Collisions

Two common edge cases should be addressed before running this script regularly on live directories. First, files lacking extensions entirely (such as a README file without periods). Because the *. pattern within ${filename##*.} finds no match, the expansion returns filename unchanged. Consequently, README is evaluated as an extension string named readme and routed to the default Others directory. This behavior handles non-extension files safely without causing script failures.

Second, file name collisions can occur if a file with the exact same name exists in the destination folder. The script in 13.3.2 uses mv -n (13.1.4) to prevent overwriting existing target files. As a result, source files encountering a name collision remain in the Downloads root directory rather than being overwritten. If business logic requires preserving both files with automatic renaming, append a timestamp to the filename before moving, using the archiving pattern from section 13.2.3.

13.4 Aliases and Shell Functions in .bashrc/.bash_profile

This final section covers configuring custom shell commands persistently so they are available across interactive terminal sessions without requiring manual redefinition or script execution.

13.4.1 Aliases vs Shell Functions: Usage Guidelines

An alias provides a shortcut mapping a short string to a longer command sequence using the alias builtin command.

alias ll='ls -la'
alias update='sudo apt update && sudo apt upgrade'

The update alias above uses apt, the standard package manager for Debian-based distributions like Ubuntu. Adapt alias definitions to match the target system package manager, such as dnf on Fedora or pacman on Arch Linux.

Because aliases perform simple text substitution before command line evaluation, they cannot process positional arguments or execute complex control flow. When requirements involve positional parameters ($1, $2 from Chapter 8.2), conditional logic (Chapter 6), or multi-step execution, use a shell function (Chapter 8.1) defined within shell configuration files rather than separate script files.

mkcd() {
    mkdir -p "$1" && cd "$1" || return 1
}

The mkcd function (make and change directory) creates a directory and navigates into it in one step. This combined behavior cannot be handled by a standard alias because it requires dynamic argument processing. The control operators && and || (Chapter 6.4.2) ensure cd executes only if mkdir -p succeeds, returning exit code 1 if either step fails.

13.4.2 Persistent Custom Aliases and Functions

Commands, aliases, and functions entered directly in a terminal persist only for the duration of that specific shell session. To persist across sessions, define them inside Bash initialization files loaded automatically on startup. Bash documentation distinguishes initialization files based on session type: a login shell loads ~/.bash_profile (or ~/.bash_login, or ~/.profile, using the first file found), while an interactive non-login shell loads ~/.bashrc.

This behavior varies across terminal applications: Linux terminal emulators typically launch interactive non-login shells that load .bashrc, whereas the default macOS Terminal application opens new windows as login shells, reading only .bash_profile by default. To ensure configurations load consistently across both shell types without duplicate entries, standard practice dictates sourcing .bashrc inside .bash_profile.

if [ -f ~/.bashrc ]; then
    . ~/.bashrc
fi

The syntax . ~/.bashrc (a dot, followed by a space, followed by the file path) is equivalent to the source command, executing the file contents within the current shell context rather than launching a subshell. Maintaining all custom aliases and functions inside .bashrc while sourcing it from .bash_profile guarantees configurations apply across both session types.

echo "alias ll='ls -la'" >> ~/.bashrc

The command above appends a new alias definition to .bashrc using redirection (Chapter 5.3.1). While useful for quick additions, editing configuration files directly with a text editor is recommended for managing multiple definitions.

13.4.3 Advanced Shell Functions

Functions placed in .bashrc use the same syntax as script-defined functions, including local variable scoping (Chapter 8.4.2). The example below defines a quick-backup function wrapping the tar operations described in 13.2.3, making it callable from any working directory.

quick-backup() {
    local target="$1"
    local timestamp

    if [[ -z "$target" ]]; then
        echo "Usage: quick-backup TARGET_FOLDER" >&2
        return 1
    fi

    timestamp="$(date +%Y%m%d-%H%M)"
    tar -czf ~/Backup/"$(basename "$target")"-"$timestamp".tar.gz \
        -C "$(dirname "$target")" "$(basename "$target")"
    echo "Backup completed: $target"
}

Using local variables ensures target and timestamp do not override variables in the interactive shell scope (Chapter 8.4.2). After adding this function to .bashrc, run source ~/.bashrc or open a new terminal window to execute the command directly, such as quick-backup ~/Documents.

13.4.4 Troubleshooting Unloaded Aliases

Updating configuration files does not instantly update active shell sessions, as configuration files are read only upon session startup. Reload configuration changes manually using source or open a new terminal window.

source ~/.bashrc

The type builtin checks whether a command string is registered as an alias, function, builtin, or external executable.

type ll
type mkcd

If type returns "not found" after updating and sourcing .bashrc, verify whether the definitions were added to the correct configuration file for the current shell type (login vs non-login shell, as discussed in 13.4.2). The shopt builtin (Chapter 7.1.5) includes the login_shell option to inspect session state, set automatically by Bash during initialization.

shopt login_shell

If output shows login_shell off, the current shell is an interactive non-login shell. Aliases defined exclusively inside .bash_profile without sourcing .bashrc will not be loaded. To temporarily remove an alias without editing configuration files, use the unalias builtin command within the current session.

unalias ll

The techniques presented in this chapter (bulk renaming, automated backups, file organization, and persistent alias/function configuration) form core daily workflow automation strategies for personal environments. Chapter 14 expands on these concepts, focusing on multimedia file operations, batch conversion workflows, and media archive management.