Linux Server Hardening

Chapter 29 closed Part VII with security details that we intentionally noted as preparation for this chapter, namely LXD containers running in an unprivileged state by default in Section 29.1.3. Since Chapter 1 and Chapter 2, we have also repeatedly touched on the term attack surface without ever thoroughly discussing it, including an explicit promise in Chapter 2 that its complete discussion awaited in this Chapter 30. Part VIII, which begins with this chapter, finally fulfills that promise, opening a new chapter of the series fully focused on server security, starting from the most fundamental principles to firewalls, intrusion prevention, mandatory access control, and auditing across the next four chapters. Hardening is the systematic process of reducing vulnerabilities that an Attacker could exploit to breach or damage a system, and this chapter builds its foundation upon four pillars: the principle of minimal attack surface, disabling unnecessary services, maintaining regular updates as the primary line of defense, and understanding the concept of TPM-backed disk encryption for both physical server and cloud scenarios.

30.1 Principles of Minimal Attack Surface and Defense in Depth

Before disabling or changing anything, we first need to understand what we are actually reducing and where it stands among the other defense layers that we will study throughout Part VIII.

30.1.1 What Is an Attack Surface

An attack surface is the aggregate of potential entry points that an Attacker could exploit to access, disrupt, or take over a system, ranging from listening network ports, running services, installed packages, to user accounts that have access to the server. The more components that are active, the larger the surface that a Sysadmin must maintain, and the greater the likelihood that one of them contains an unpatched vulnerability. The principle of a minimal attack surface, which we have touched on since Chapter 1 and Chapter 2, is ultimately simple: do not run or install anything that is not genuinely required by the server's role. A server dedicated solely to acting as a web server, for example, does not need to run printing services, Bluetooth, or local network discovery commonly found in a desktop installation.

30.1.2 Defense in Depth: A Map of Hardening Layers in This Series

Minimizing the attack surface is the first layer, not the sole layer of defense. Defense in depth is a strategy of arranging multiple defense layers that are independent of one another, so that a failure in one layer does not immediately mean the entire system is breached. This series structures these layers progressively throughout Part VIII, as summarized in the following table.

This Chapter 30 focuses on the first layer in the table above, namely shrinking what can be attacked via active services and closing known vulnerabilities through routine updates, concluded with an introduction to disk encryption as an additional defense in case all layers above it are still breached and the server disk falls into the wrong hands.

30.1.3 Calculating Our Server's Baseline Attack Surface

Before disabling anything in Section 30.2, it is good practice to measure how large our managed server's attack surface currently is, so that a comparison figure is available after the hardening process is completed.

Practical Steps

1. Count the number of services currently running via systemd.

   systemctl list-units --type=service --state=running --no-legend | wc -l

2. Count the number of fully installed packages on the system.

   dpkg -l | grep -c ^ii

Verification and Troubleshooting

• Record both numbers as a baseline immediately after the initial installation is completed in Chapter 2, then compare them periodically. A significant surge without a clear reason warrants investigation, whether because a Developer installed a debug tool and forgot to remove it, or worse, because there is an unfamiliar package that we never installed ourselves.
• There is no universally applicable "safe" number for both metrics. Every server's requirements differ; it is normal for a database server to run fewer services compared to a server performing multiple roles simultaneously, so use these numbers as a benchmark against the server itself over time, not against other servers.

30.2 Disabling Unnecessary Services

Section 30.1 merely measured the attack surface numerically. This section transitions into concrete action: identifying which services are not actually required by our server's role, and then disabling them properly.

30.2.1 Identifying Active Services and Ports

Every service listening on a port represents an open communication pathway that could potentially be accessed from the outside, whether by legitimate Users or an Attacker performing port scanning.

Practical Steps

1. View all listening TCP/UDP ports along with their owner processes.

   sudo ss -tulnp

2. Compare this with the list of running services via systemd, using the command we introduced in Chapter 5.

   systemctl list-units --type=service --state=running

Verification and Troubleshooting

• The Process column in the output of ss -tulnp displays the binary name and PID owning the port. If the service name is unfamiliar, trace its package origin using dpkg -S /path/to/binary before deciding whether it is safe to disable.
• The netstat command, once popular for similar purposes, has long been superseded by ss, part of the iproute2 package, because net-tools (the source of netstat) is no longer actively developed across many modern distributions including Ubuntu.
• systemctl list-units --type=service --state=running only shows services active at the present moment, not services that will launch on subsequent boots. Check the list of enabled services via systemctl list-unit-files --type=service --state=enabled, as a currently stopped but enabled service will automatically activate after the next reboot, a condition easily overlooked during a casual audit.

30.2.2 Disabling, Masking, and Removing Services

Systemd provides three levels of action against an unnecessary service, each with a different level of severity.

Practical Steps

1. Stop and disable the service from the boot process simultaneously.

   sudo systemctl disable --now service-name

2. To ensure the service cannot be started by any means, mask the service.

   sudo systemctl mask service-name

3. If the associated package is completely unnecessary, remove it entirely along with its dependencies.

   sudo apt purge package-name
   sudo apt autoremove

Verification and Troubleshooting

• Confirm the final status of the service; it should report masked or inactive (dead), not failed.

  systemctl status service-name

• The systemctl mask command technically creates a symlink pointing the unit file to /dev/null inside /etc/systemd/system/, which can be verified directly via ls -la /etc/systemd/system/service-name.service.
• A candid note on risk: apt purge also removes configuration files, not just the binaries. For services that might be reinstalled later, such as an unused database that could be required again, use systemctl disable --now or mask without purge, ensuring old configurations are preserved.
• If something turns out to depend on a newly disabled service, restore it using systemctl unmask service-name followed by systemctl enable --now service-name.

30.2.3 Case Study: Pruning Frequently Left-Behind Services

The following services are often installed via recommends dependencies of other packages despite rarely being needed on a production server. Always check whether the service is actually installed on your server before disabling it, as not all installations include them.

Practical Steps

1. First check which of the above services are installed on your server.

   dpkg -l | grep -E 'avahi-daemon|cups-browsed|modemmanager|bluetooth|rpcbind'

2. Disable installed services one by one if they are irrelevant to your server's role, as demonstrated below for avahi-daemon.

   sudo systemctl disable --now avahi-daemon.service avahi-daemon.socket

Verification and Troubleshooting

• After disabling, test the main functionality of the server (e.g., web access via Nginx, SSH connections) to ensure no hidden dependencies were impacted.
• In practice, rpcbind is the candidate most frequently disabled by mistake without prior verification, because Sysadmins forget that the server still acts as an NFS client for shared storage mounts, as discussed in Chapter 25. Always check /etc/fstab and showmount before touching this service on a long-running server.

30.3 Regular Updates as the Primary Line of Defense

Disabling unneeded services closes vulnerabilities known from the start. Regular updates address newly discovered vulnerabilities after the server is operational, and because new vulnerabilities are continuously published, this practice often determines whether an incident actually occurs or is successfully prevented in advance.

30.3.1 Why Updates Are the Highest-Impact Security Control

The majority of server security incidents stem not from sophisticated zero-day exploits unknown to anyone, but from legacy vulnerabilities that already have official patches that were not applied in time. Attackers generally do not target a specific server manually; instead, they run automated scanners searching for software versions with vulnerabilities widely published in CVE databases. A server lagging behind on updates, even if protected by a strict firewall in Chapter 31, remains exposed to exploitation via vulnerabilities in intentionally exposed services, such as the web server or SSH itself. Section 6.3 provided us with two automation mechanisms for this: unattended-upgrades in Section 6.3.1 for routine package updates, and Livepatch in Section 6.3.2 for kernel patches without rebooting. This section does not repeat their installation, but rather adds audit routines that must run alongside that automation.

30.3.2 Quick Audit of Security Patch Status

Automation like unattended-upgrades reduces manual workload, but a responsible Sysadmin must still periodically verify manually that the server is truly up to date, rather than relying on the assumption that automation always runs smoothly.

Practical Steps

1. Refresh repository metadata, then list packages that have newer versions available.

   sudo apt update
   apt list --upgradable

2. Filter only updates coming from security origins, matching origins allowed in the unattended-upgrades configuration in Section 6.3.1.

   apt list --upgradable 2>/dev/null | grep -i security

Verification and Troubleshooting

• An empty output on step two indicates no pending security updates, either because the server is fully updated or because unattended-upgrades has already applied them automatically.
• A list containing many lines without the word security generally represents standard feature updates or minor versions, not urgent vulnerabilities. Prioritize -security origins first, and schedule non-security updates during a separate maintenance window so they do not mix with patch urgency.
• For servers with active Ubuntu Pro as discussed in Section 6.5, the pro fix tool can directly target a specific CVE, useful when a critical vulnerability is newly announced and we want to ensure the server is patched without waiting for the next unattended-upgrades cycle.

  sudo pro fix CVE-YYYY-NNNNN

30.3.3 Scheduled Reboots for Kernels and Critical Components

Certain updates, particularly kernel updates and core libraries like glibc, only become fully active after processes loading them into memory are restarted. Having a package installed does not mean its patch has taken effect.

Practical Steps

1. Check whether the server requires a reboot for critical patches to take effect.

   cat /var/run/reboot-required

2. View which specific packages triggered the reboot requirement.

   cat /var/run/reboot-required.pkgs

Verification and Troubleshooting

• Both files above are generated by hooks from the update-notifier-common package whenever specific package installations require a reboot. A No such file or directory message on a server not requiring a reboot is normal behavior, not an error.
• For non-kernel components using old libraries in memory without requiring a full system reboot, needrestart (covered in Section 6.3.1) remains the most practical tool for restarting only the affected processes without rebooting the entire server.
• Decisions regarding automated versus manual reboots were discussed via the Unattended-Upgrade::Automatic-Reboot parameter in Section 6.3.1. For production database servers or high-uptime services, monitor /var/run/reboot-required via monitoring (Chapter 38) and schedule manual reboots during agreed maintenance windows, rather than letting pending reboots accumulate indefinitely.

30.4 TPM-backed Disk Encryption for Servers and Cloud

The preceding three sections closed vulnerabilities at the software level, from running services to installed patches. This closing section addresses a defense layer of a different nature that remains relevant even if an Attacker manages to bypass all software layers above: encrypting data stored physically on disk.

30.4.1 The Classic LUKS Passphrase Problem on Headless Servers

LUKS (Linux Unified Key Setup) is the standard full-disk encryption specification in Linux, operating over dm-crypt at the kernel level. Its classic approach requires someone to manually type a passphrase every time the server boots, before the operating system fully initializes and network access becomes available. This approach is logical for laptops always in front of their owners, but poses a real challenge for headless servers as discussed since Chapter 1. A remotely rebooted server, such as after a kernel patch in Section 30.3.3, will stall waiting for a passphrase that no one is present to type, unless Sysadmins set up extra workarounds like SSH access inside initramfs via dropbear-initramfs, which adds operational complexity.

30.4.2 How TPM Seals Encryption Keys

A TPM (Trusted Platform Module) is a specialized security chip embedded on physical motherboards or provided as a virtual TPM (vTPM) on modern virtualization platforms. It is capable of securely storing cryptographic keys and sealing them against specific boot chain conditions. Its mechanism relies on measured boot: every stage of the boot process, from firmware to bootloader to kernel, records its hash value to specialized TPM registers called Platform Configuration Registers (PCRs). The LUKS encryption key is sealed so that it can only be decrypted if these PCR values match the state recorded when sealing occurred. As long as the boot chain remains untampered with, the TPM releases the key automatically without requiring a manual passphrase, resolving the remote reboot problem from Section 30.4.1. Conversely, if the boot chain changes—such as firmware modified by an Attacker or a disk moved to different hardware—the unseal process fails, causing the system to fall back to a recovery key mechanism, while signaling that something has changed and requires inspection.

In the Linux ecosystem, binding LUKS to TPM2 is commonly managed via systemd-cryptenroll, a tool integrated into systemd in recent releases. As an illustration of the underlying mechanism, the following two commands list detected TPM devices and bind a LUKS partition to PCR 7, which records the Secure Boot state.

systemd-cryptenroll --tpm2-device=list
sudo systemd-cryptenroll /dev/sdXN --tpm2-device=auto --tpm2-pcrs=7

Before relying on this mechanism, ensure that your hardware or virtualization platform provides a functional TPM chip, whether physical or virtual.

ls /dev/tpm*

The presence of /dev/tpm0 or /dev/tpmrm0 indicates the kernel has detected and loaded drivers for the TPM device. If the command above returns No such file or directory, the server lacks a usable TPM, either because physical hardware is not equipped with the chip or because the virtual instance has not enabled vTPM options at the hypervisor or cloud provider level.

To remain accurate to our series standard, full support for TPM-backed disk encryption via installation wizards matured earlier in Ubuntu Desktop than in Ubuntu Server across recent releases. Exact implementation details on Ubuntu Server 26.04 LTS may evolve across point releases, so the safest approach before relying on it in production servers is to verify its status and activation methods via official Ubuntu Server release notes, rather than treating the example command above as the definitive sole method.

30.4.3 Relevance for Physical Servers vs. Cloud Instances

The relevance of TPM-backed disk encryption varies depending on whether you manage physical hardware or cloud instances.

For physical servers located in an on-premise server room or hosted at a colocation facility, the primary threat is physical theft or disk removal. Anyone who extracts a server disk and attaches it to another computer can directly read its contents if unencrypted. TPM-backed disk encryption mitigates this threat without sacrificing remote reboot capability, making it far more practical than manual LUKS passphrases for rarely visited physical servers.

For cloud instances, the scenario is slightly different. Major cloud providers like AWS, Azure, and GCP typically enforce data-at-rest encryption at their storage backend level (e.g., EBS encryption in AWS or default persistent disk encryption in GCP), meaning physical disk theft by third parties is effectively addressed without extra Sysadmin configuration. What keeps TPM relevant in the cloud is a shift toward boot chain integrity rather than simple data confidentiality. Features like NitroTPM on AWS, Trusted Launch on Azure, and Shielded VMs on GCP provide a vTPM that enables verification that the VM booted using untampered components. This adds a defense layer against bootkit or rootkit attacks in multi-tenant environments and helps fulfill compliance requirements that explicitly mandate guest-level disk encryption regardless of provider-level guarantees.

In summary, TPM-based disk encryption is most valuable for physical servers facing physical theft risks alongside frequent remote reboot needs. For cloud instances, its value centers on boot chain integrity and compliance fulfillment rather than mere data confidentiality already handled by cloud providers.

So far, we have built the foundation of server hardening on four pillars: understanding and measuring the attack surface, disabling unnecessary services properly, maintaining regular updates as a high-impact security control rather than a administrative routine, and understanding TPM-backed disk encryption as an extra defense layer for physical and cloud scenarios. The following Chapter 31 proceeds to the second layer of our defense-in-depth map from Section 30.1.2: advanced firewalls, ranging from legacy iptables, UFW as a user-friendly frontend, to nftables as the modern architecture now powering both.

```