Automation with Cron & Systemd

The closing case study of Chapter 17 produced audit-server.sh, a script that prints complete server conditions with just a single manual command. The problem is that Sysadmins managing production servers rarely have time to log in every morning just to run a single script; the actual need is for that script to run on its own, right on schedule, without any human intervention. This chapter discusses the two most common ways to achieve this in Linux: cron, the classic task scheduler that has been the de-facto standard since the early Unix era, and systemd timer, a more modern alternative fully integrated with systemd as the init system in almost all contemporary Linux distributions. The discussion begins with crontab syntax and environment traps that often confuse beginners, followed by an introduction to systemd timer units, how to run scripts as services via unit files, and closes with a case study of rescheduling the audit script from Chapter 17 using both methods simultaneously.

18.1 Task Scheduling with cron and crontab

Cron is a task scheduling daemon that runs continuously in the background from system boot, reading schedule lists from one or many crontab files, then executing the listed commands precisely at the specified time. No active login session or open terminal is needed; cron is specifically designed for tasks that run without human presence. Its implementation varies across distributions, commonly cronie in RHEL/Fedora or the Vixie cron-based cron package in Debian/Ubuntu, but the crontab format discussed in this sub-chapter is consistent across all these implementations, adhering to the man 5 crontab specification.

18.1.1 Crontab Format and Syntax

A single line in a crontab consists of five schedule fields followed by the command to be executed, separated by spaces.

* * * * * command_to_execute

An asterisk (*) means "every possible value" in that field. Three other special characters expand schedule expressions to be much more flexible: comma (,) for value lists, hyphen (-) for ranges, and slash (/) for step values.

*/15 * * * * /usr/local/bin/check-connection.sh
0 2 * * * /usr/local/bin/daily-backup.sh
0 9 * * 1-5 /usr/local/bin/morning-report.sh
30 3 1 * * /usr/local/bin/monthly-log-rotate.sh

The first line runs check-connection.sh every 15-minute interval. The second line runs daily-backup.sh every day at 02:00. The third line runs morning-report.sh at 09:00 but only from Monday through Friday (1-5), useful for tasks relevant only on workdays. The last line runs monthly-log-rotate.sh at 03:30 on the 1st of every month.

Most modern cron implementations, including Vixie cron and cronie, also support seven shortcut keywords that replace the five schedule fields at once for common patterns: @reboot (runs once whenever the cron daemon starts, usually meaning every system boot), @yearly or @annually, @monthly, @weekly, @daily or @midnight, and @hourly. These keywords are not part of the POSIX cron standard, so they are not guaranteed to be available on all Unix-like systems, but they are safe to use in practically all modern Linux distributions.

@daily /usr/local/bin/daily-backup.sh
@reboot /usr/local/bin/mount-additional-disk.sh

18.1.2 Editing Crontab with crontab -e

Every user has their own crontab, and the official way to edit it is not by opening the file directly with a text editor, but through the crontab -e command.

crontab -e

This command opens the logged-in user's crontab using the editor specified by the VISUAL or EDITOR variable (falling back to system default, usually vi, if neither is set). Once saved, cron automatically re-reads its content without requiring a manual restart of the daemon. To view active crontab contents without opening it for editing, use the -l option.

crontab -l

Each user's crontab file is actually stored in a special spool directory whose exact location varies between distributions, such as /var/spool/cron/crontabs/ in Debian/Ubuntu or /var/spool/cron/ in RHEL/Fedora. This path is mentioned purely as context, not as an invitation to edit it directly; always use the crontab -e command, and never open or overwrite files in that spool directory directly via regular text editors. Besides missing the trigger to notify the cron daemon of changes, spool file formats and permissions are strict, and cron typically rejects files with incorrect permissions for security reasons.

Root, or any user permitted via /etc/cron.allow//etc/cron.deny, can edit another user's crontab using the -u option.

sudo crontab -u deploy -e

One option that must be used with caution is -r, which deletes the user's entire crontab without any confirmation in most cron implementations. Make a habit of running crontab -l > backup-crontab.txt prior to experimenting with -r, ensuring existing schedules are not lost due to a typo or target user error.

18.1.3 Environment and PATH in Cron

The most classic trap experienced by almost everyone using cron for the first time: a script runs smoothly when tested manually from terminal, but fails completely once scheduled via cron, without a clear error message. The cause is almost always the same: cron runs every command within a much more minimal environment compared to a regular interactive shell. Cron does not read ~/.bashrc, ~/.bash_profile, or /etc/profile, because the executed commands are neither a login shell nor an interactive shell as distinguished in Chapter 1.4. Consequently, aliases, functions in .bashrc, and additional PATH variables commonly used daily are entirely unavailable within cron's context.

Cron's default PATH variable is much shorter than an interactive PATH, generally containing only /usr/bin:/bin, according to default behavior documented in man 5 crontab. Commands located outside those two directories, such as binaries installed via third-party package managers in /usr/local/bin or /opt, will not be found unless invoked using their full absolute path.

0 2 * * * daily-backup.sh
0 2 * * * /usr/local/bin/daily-backup.sh

The first line risks failing with errors like command not found if daily-backup.sh does not happen to exist in /usr/bin or /bin. The second line, using an absolute path, is significantly safer and forms an essential habit for every command scheduled via cron, not only for the script itself but also for any commands called within it (tar, rsync, curl, and similar tools).

If writing absolute paths on every line feels repetitive, PATH can also be redefined directly at the top line of the crontab, applying to all schedule lines underneath it.

PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/bin
SHELL=/bin/bash

0 2 * * * daily-backup.sh

The SHELL variable above specifies which shell cron uses to execute commands, defaulting to /bin/sh instead of /bin/bash. This is particularly important if crontab commands rely on Bash-specific syntax like arrays or [[ ]] without executing scripts through their own #!/bin/bash shebang; explicitly setting SHELL=/bin/bash removes this ambiguity.

18.1.4 Cron Job Output and Logs

By default, cron does not discard output generated by executed commands. According to man 5 crontab, stdout and stderr from every job are captured and sent via email to the crontab owner using a local Mail Transfer Agent (MTA) such as sendmail or postfix. In practice, most modern production servers do not install a local MTA at all, causing these emails to fail silently with no visible error anywhere. New Sysadmins often assume jobs "run fine" when in reality they fail completely and error messages vanish into undelivered local mail.

A far more reliable practice is explicitly redirecting job output to a log file.

0 2 * * * /usr/local/bin/daily-backup.sh >> /var/log/daily-backup.log 2>&1

This >> file 2>&1 pattern is identical to the concept covered in Chapter 5.3, appending both stdout and stderr into a single log file without overwriting previous content across executions. If output is intentionally unnecessary, redirecting to /dev/null communicates intent far clearer than defaulting to potentially unhandled emails.

The MAILTO variable at the top of a crontab controls email delivery destinations when an MTA is present and operational; setting it to an empty string disables email delivery entirely, pairing well with explicit log redirection practices.

MAILTO=""
0 2 * * * /usr/local/bin/daily-backup.sh >> /var/log/daily-backup.log 2>&1

Job output and cron daemon activity logs are two distinct entities. Cron records execution start timestamps (without output body) to syslog, searchable via journalctl on systems running systemd-journald.

journalctl -u cron
journalctl -u crond

Service unit names differ across distributions, using cron on Debian/Ubuntu and crond on RHEL/Fedora; check actual names using systemctl status cron or systemctl status crond if one is missing. Activity logs confirm whether cron triggered jobs on schedule, independent of job execution success or failure, which is inspected via output log files.

18.1.5 System Crontab and /etc/cron.d

All examples so far utilized per-user crontab files. For system-wide automation requirements, a system crontab exists at /etc/crontab. Its format is similar but inserts an additional field, the username, right after the five schedule fields and before the command. This additional field distinguishes it from user-level crontabs: user crontabs implicitly know job ownership (the user itself), while system crontabs can execute tasks as any system user from a unified file.

# m h dom mon dow user command
0 3 * * * root /usr/local/bin/system-backup.sh

The /etc/cron.d/ directory holds separate files adhering strictly to the /etc/crontab format, including user fields. Package managers frequently employ this mechanism to ship application-specific schedules (e.g., mysql-server or certbot) without editing shared /etc/crontab files directly, mitigating multi-package configuration conflicts. Both Debian/Ubuntu and RHEL/CentOS/Fedora fully support this /etc/cron.d/ framework.

Most Vixie cron derivative distributions provide four ready-to-use directories: /etc/cron.hourly/, /etc/cron.daily/, /etc/cron.weekly/, and /etc/cron.monthly/. Dropping executable scripts into these directories causes cron to run them at configured intervals without writing custom schedule lines; execution relies on base /etc/crontab entries that invoke run-parts to process directory contents sequentially. Exact execution hours vary across distributions; verify server settings inside /etc/crontab or anacron configurations when exact timing precision is required.

18.2 Systemd Timers as an Alternative to Cron

While cron is adequate for basic scheduling needs, its limitations become apparent on complex server infrastructures: lack of structured logging, no automated catch-up mechanisms when system downtime coincides with execution schedules, and lack of integration with systemd resource controls (CPU, memory) used across other services. Systemd timers represent systemd's solution for task scheduling, and because almost all modern Linux distributions (Ubuntu, Debian, Fedora, RHEL, Arch, openSUSE, etc.) adopt systemd as their default init system, timer capabilities are natively available without supplementary installations.

18.2.1 Why Systemd Timers

The following table summarizes core operational differences between cron and systemd timers relevant to daily Sysadmin routines.

The advantages of systemd timers do not render cron obsolete. For simple single-line schedules that do not require resource controls or catch-up executions, cron remains quicker to draft. Systemd timers excel for critical tasks that must not be missed (such as daily backups on laptops frequently powered down), or jobs requiring monitoring via structured logging.

18.2.2 Anatomy of a Timer Unit

A systemd timer is always paired with a service unit: the timer dictates when, while the service dictates what is executed. By convention, name.timer automatically triggers name.service sharing the exact base name, unless overridden using the Unit= option.

[Unit]
Description=Daily server audit schedule

[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true

[Install]
WantedBy=timers.target

OnCalendar= defines calendar-based schedules, utilizing syntax documented in man 7 systemd.time. The format *-*-* 02:00:00 translates to "every day, any date, any month, any year, at 02:00:00", equivalent to 0 2 * * * in cron. systemd.time also provides shorthand aliases for common schedules: OnCalendar=daily, OnCalendar=weekly, OnCalendar=monthly, and OnCalendar=hourly.

Persistent=true (defaulting to false) enables catch-up executions: if the system is powered off when a timer is scheduled to trigger, systemd executes the service immediately upon the next boot instead of waiting for the subsequent scheduled interval. Notably, this directive applies exclusively to calendar-based timers (OnCalendar=); it exerts no effect on relative time timers like OnBootSec= (delay post-boot) or OnUnitActiveSec= (delay post-last activation), which suit periodic recurring tasks without fixed wall-clock requirements.

18.2.3 Creating and Enabling Timers

Creating a new timer always requires two files: a service file defining execution parameters and a timer file defining scheduling rules. Service unit structure is expanded in Section 18.3; the example below uses minimal declarations to focus on scheduling mechanics. First, create the service file.

sudo nano /etc/systemd/system/check-disk.service

[Unit]
Description=Quick disk health check

[Service]
Type=oneshot
ExecStart=/usr/local/bin/check-disk.sh

Next, create the corresponding timer file, maintaining an identical base file name.

sudo nano /etc/systemd/system/check-disk.timer

[Unit]
Description=Disk check schedule every 30 minutes

[Timer]
OnBootSec=5min
OnUnitActiveSec=30min

[Install]
WantedBy=timers.target

Save these unit files in /etc/systemd/system/, the local administrative directory prioritized by systemd over distribution defaults in /usr/lib/systemd/system/. Once saved, instruct systemd to reload configurations and enable the timer unit (not the service unit) via systemctl.

sudo systemctl daemon-reload
sudo systemctl enable --now check-disk.timer

Execute daemon-reload whenever unit files are created or edited so systemd registers definitions before invocation. Using enable --now accomplishes two actions simultaneously: enable registers the timer to start automatically at system boot (via WantedBy=timers.target in the [Install] block), while --now starts the timer immediately without awaiting reboot.

Verify registered timers and upcoming execution schedules using list-timers.

systemctl list-timers check-disk.timer

Output columns detail NEXT (next scheduled execution) and LAST (last trigger execution), providing actionable context compared to static crontab -l listings.

18.3 Running Scripts as Services

Section 18.2 introduced service units briefly as timer targets. This section explores service unit anatomy in depth, as service units extend beyond timer targets: they form the standard mechanism for daemonizing custom applications on modern servers, featuring automatic restart options upon application crashes.

18.3.1 Service Unit Anatomy

A .service unit file consists of three primary sections: [Unit] for metadata and dependencies, [Service] for execution behavior, and [Install] for enablement rules.

[Unit]
Description=Internal web application
After=network.target

[Service]
Type=simple
User=deploy
WorkingDirectory=/opt/application
ExecStart=/opt/application/run.sh
Restart=on-failure

[Install]
WantedBy=multi-user.target

The Type= directive defines process tracking for systemd. Type=simple (default when omitted) assumes the process invoked by ExecStart= remains in the foreground as the primary active process, suited for persistent daemons like web applications. Conversely, Type=oneshot (used in check-disk.service in 18.2.3) expects the process to execute and terminate, fitting single-run automation scripts triggered by timers or manual execution.

The User= directive drops privileges to run as a non-root user, maintaining least privilege access principles referenced in Section 17.3.3. Restart=on-failure instructs systemd to automatically restart processes exiting with non-zero exit codes, a feature natively absent in cron. The WantedBy=multi-user.target directive inside [Install] hooks service enablement to standard multi-user system boot targets.

18.3.2 Managing Services with systemctl

systemctl is the single unified utility for managing unit lifecycles across services and timers.

sudo systemctl start web-app.service
sudo systemctl stop web-app.service
sudo systemctl restart web-app.service
systemctl status web-app.service

The start, stop, and restart commands modify temporary operational state without altering boot startup configurations. The status subcommand displays execution summaries, active states (active (running) or failed), main Process IDs (PIDs), and recent log entries. Managing boot startup behavior uses distinct subcommands.

sudo systemctl enable web-app.service
sudo systemctl disable web-app.service

enable generates symlinks within target directories matching WantedBy= declarations to start services on boot; disable removes symlinks without stopping active runtime processes. To enable and immediately start units in a single operation, use enable --now as shown in Section 18.2.3.

Logs from systemd-managed services are ingested directly into the system journal without manual configuration, leveraging advantages detailed in Section 18.2.1.

journalctl -u web-app.service
journalctl -u web-app.service -f
journalctl -u web-app.service --since today

The -f (follow) flag streams live entries similarly to tail -f, while --since today filters entries for current-day events, bypassing manual log file maintenance.

18.3.3 User Services vs System Services

Service units discussed up to this point are system services, located in /etc/systemd/system/, managed with root administrative privileges via sudo, and executed independently of user login states. For user-level automation, systemd offers user services operating without administrative privileges.

mkdir -p ~/.config/systemd/user
nano ~/.config/systemd/user/reminder.service

[Unit]
Description=Hourly water drinking reminder

[Service]
Type=oneshot
ExecStart=/usr/bin/notify-send "Time to drink water"

User units reside in user home directories, managed by appending the --user flag to systemctl commands.

systemctl --user daemon-reload
systemctl --user enable --now reminder.timer

By default, user service managers terminate when the user logs out. To allow background user services to persist on servers after SSH sessions disconnect, administrators can enable lingering for the user account.

sudo loginctl enable-linger deploy

enable-linger maintains the user's systemd service manager instance across disconnects, granting system-service durability while preserving non-root management access.

18.4 Case Study: Scheduled Server Maintenance Script

This closing case study reschedules the audit-server.sh script introduced in Section 17.4 using both cron and systemd timers, highlighting differences in implementation. Scheduling audit tasks establishes the groundwork for automated alerting workflows explored in Chapter 20, where non-zero script exit statuses trigger notifications upon filesystem threshold breaches.

18.4.1 Scheduling Design

Assume audit-server.sh has been placed in /usr/local/bin/audit-server.sh with execution permissions granted. The deployment design targets daily execution at 02:00, when system load is typically low. The cron implementation writes results to dated log files as demonstrated in Section 17.4.3, whereas the systemd timer relies directly on journal logging without explicit -o file output flags, utilizing automatic timestamping covered in Section 18.3.2.

18.4.2 Implementing Cron and Systemd Timers

The cron version is registered via crontab -e.

MAILTO=""
0 2 * * * /usr/local/bin/audit-server.sh -t 80 -o /var/log/audit-server/audit-$(date +\%Y\%m\%d).log

Note the escaping backslashes preceding % characters in date +\%Y\%m\%d. According to man 5 crontab, unescaped percent signs (%) within crontab command fields are converted into newline characters, passing remaining text to commands as standard input (stdin). Omitting backslashes breaks string formatting and causes execution errors. This is a common cron pitfall because commands function normally when executed interactively inside standard terminal sessions.

The systemd timer implementation relies on two unit files.

sudo nano /etc/systemd/system/audit-server.service

[Unit]
Description=Daily server audit check

[Service]
Type=oneshot
ExecStart=/usr/local/bin/audit-server.sh -t 80

sudo nano /etc/systemd/system/audit-server.timer

[Unit]
Description=Daily server audit schedule

[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true

[Install]
WantedBy=timers.target

Persistent=true is explicitly set here, unlike interval-based timers (e.g., OnUnitActiveSec=30min). Daily audit operations benefit from catch-up executions following server downtime or maintenance windows occurring at 02:00.

sudo systemctl daemon-reload
sudo systemctl enable --now audit-server.timer

18.4.3 Verification and Troubleshooting

To verify the cron setup, ensure the entry is written correctly, then test manual execution matching the cron line (without backslashes required by interactive shells).

crontab -l
/usr/local/bin/audit-server.sh -t 80 -o /var/log/audit-server/audit-$(date +%Y%m%d).log
cat /var/log/audit-server/audit-*.log

If scheduled outputs do not generate, verify daemon status (systemctl status cron or systemctl status crond), inspect invocation history via journalctl -u cron, and verify absolute binary paths within environment variables.

For systemd timer implementations, use list-timers to verify active schedules, and query journalctl to inspect execution logs.

systemctl list-timers audit-server.timer
journalctl -u audit-server.service --since today

If timer units trigger but associated services fail, invoke systemctl status audit-server.service to inspect process exit codes and error output. Syntax errors inside unit definitions can be validated prior to execution using systemd-analyze verify.

systemd-analyze verify /etc/systemd/system/audit-server.service

Both concise cron configurations and resilient systemd timer units represent valid automation patterns for production deployments. Chapter 19 builds on these foundation topics by exploring advanced log management using journalctl.