AppArmor on the Server

Chapter 32 concludes with an honest admission: Fail2ban ceases to be effective precisely at the point an attacker successfully gains access through another loophole, as its job is indeed to prevent repeated login attempts, not to restrict what a process can do once it is already running inside the system. This chapter enters the fourth layer of the defense in depth roadmap built in Section 30.1.2, which is AppArmor, operating under a working assumption distinct from the previous three layers: not preventing the attacker from getting in, but rather limiting the damage a process can inflict once running, even if that process runs as root.

The scenario is concrete and frequently encountered in the field. A security vulnerability in a PHP application managed by a developer, such as an arbitrary file upload vulnerability, can be exploited by an attacker to execute arbitrary commands through the PHP-FPM process from Chapter 14. Without AppArmor, that PHP-FPM process running as the www-data user is free to read or write any file accessible under standard Linux permissions for the www-data user, including directories of other applications that happen to grant read access to the same group. AppArmor closes this gap by restricting the process to access only the directories, files, and kernel capabilities explicitly registered in its profile, ensuring that even permissive Linux permissions no longer serve as a free pass for a hijacked process.

We will start with a review of Mandatory Access Control concepts and the anatomy of an AppArmor profile, followed by inspecting default profiles protecting BIND9 since Chapter 9 and MariaDB since Chapter 20, confirming firsthand that Nginx and PHP-FPM run without default profiles, seeing how Docker and LXD rely on AppArmor from the ground up to isolate containers, understanding the difference between enforce and complain modes through hands-on mode switching, and concluding by building a simple custom profile for Nginx from scratch using aa-genprof.

33.1 AppArmor Concept Review

Before moving into the hands-on practice of inspecting and creating profiles, it is beneficial to align our understanding of what AppArmor actually does behind the scenes, as the terms in this section will recur throughout the chapter.

33.1.1 Mandatory Access Control: Why Standard Linux Permissions Are Not Enough

Standard Linux permissions, namely the combination of owner, group, and mode familiar since Linux 101, fall under the Discretionary Access Control (DAC) model. It is termed discretionary because control resides in the hands of the file or process owner, and crucial to note, processes running as root can almost always bypass any DAC restriction. This is its fundamental flaw: once a process is hijacked by an attacker and happens to run as root, such as certain database daemons or DNS servers requiring high privileges at startup, DAC is entirely powerless to limit what that process does next.

Mandatory Access Control (MAC) takes a different approach. Access policies are set by administrators through kernel-enforced rules, regardless of the process owner's privileges, including root. AppArmor is an implementation of MAC in Linux in the form of a Linux Security Module (LSM), an official kernel framework allowing security modules to hook into crucial system call execution points before they are carried out. AppArmor is not the only LSM implementation; SELinux, used in the Red Hat/Fedora distribution family, is another MAC implementation with a label-based philosophy that is considerably more complex to configure. Ubuntu and the broader Debian family choose AppArmor because its path-based model is much easier for humans to read and write compared to SELinux labels, and it comes installed and active by default starting from the initial installation in Chapter 2.

Practical Steps

1. Confirm that the AppArmor module is truly active at the kernel level.

   cat /sys/module/apparmor/parameters/enabled

2. Check the status of the service managing profile loading at boot.

   sudo systemctl status apparmor

Verification and Troubleshooting

• A healthy output for the first step displays the letter Y, indicating that the kernel is indeed loading AppArmor as an active LSM. Ubuntu Server 26.04 loads AppArmor by default upon installation, so this step should never output N unless someone deliberately disabled it via the apparmor=0 parameter in GRUB, an action that rarely has a valid reason on production servers.
• A normal systemctl status apparmor displays active (exited) rather than active (running). This is not a sign of failure; this unit is responsible for loading all profiles into the kernel once at boot time and then terminating, as subsequent policy enforcement is handled entirely by the kernel itself, not by a persistent daemon process in userspace.

33.1.2 Profile Anatomy: Path Entries, Capability Entries, and Locations in /etc/apparmor.d/

A single AppArmor profile is a plain text file defining what a specific executable is allowed to access. Profiles contain two primary types of rules. Path entries define which files or directories a process can access along with their access modes, such as read (r), write (w), or execute (x). Capability entries define which Linux kernel capabilities the process is allowed to use, such as net_bind_service, which allows a non-root process to bind to ports below 1024 like port 80 used by Nginx since Chapter 12. Profiles are stored in the /etc/apparmor.d/ directory, using a legacy naming convention that replaces every / character in the executable path with a dot. For example, the profile for /usr/sbin/named belonging to BIND9 is stored as /etc/apparmor.d/usr.sbin.named.

The commands aa-status, aa-genprof, aa-logprof, aa-complain, and aa-enforce used throughout this chapter are not part of the base apparmor package active since Chapter 2, but are provided via a separate package named apparmor-utils which is not installed by default.

Practical Steps

1. Install apparmor-utils to obtain all aa-* commands.

   sudo apt install apparmor-utils

2. View a summary of all profiles currently loaded by the kernel along with their modes.

   sudo aa-status

Verification and Troubleshooting

• The output of aa-status on a fresh Ubuntu Server 26.04 installation typically shows several system default profiles such as /usr/sbin/tcpdump or profiles related to snapd, looking roughly like this, although the exact number of profiles varies depending on installed packages.

  apparmor module is loaded.
  15 profiles are loaded.
  12 profiles are in enforce mode.
     /usr/sbin/tcpdump
     ...
  3 profiles are in complain mode.
     ...
  0 processes have profiles defined.

• The line processes have profiles defined showing zero is normal at this stage, as active services like Nginx or PHP-FPM on our server do not yet have profiles defined, which is the exact issue dissected in Section 33.2.

33.2 AppArmor Profiles for Services: Nginx, PHP-FPM, and Databases

The conceptual foundation is ready. This section directly proves something promised since Chapter 9, while correcting a common misconception: not all services are automatically protected by AppArmor simply because the module is active in the kernel.

33.2.1 Services Protected from the Start: BIND9 and MariaDB

Section 9.2.1 mentioned that BIND9 on Ubuntu runs under an AppArmor profile restricting which directories the named process can access, promising a deeper discussion in this chapter. The bind9 package includes a ready-to-use profile in /etc/apparmor.d/usr.sbin.named that automatically activates in enforce mode upon installation, requiring no extra configuration steps.

MariaDB from Chapter 20 has a newer story worth knowing for sysadmins. Since MariaDB 11.8, the MariaDB community re-released the AppArmor profile for mariadbd, making it much more mature than the old profile. Every rule line is tested against over 7,000 test cases from the official mariadb-test-run test suite, significantly more than the roughly 1,000 tests typical in Debian/Ubuntu packaging CI pipelines. The result is a profile approximately 200 lines long stored in /etc/apparmor.d/mariadbd, omitting the dot convention in its name in favor of a cleaner naming style, unlike BIND9's legacy usr.sbin.named convention.

Practical Steps

1. Confirm that the named profile for BIND9 is running in enforce mode.

   sudo aa-status | grep -A1 named

2. Do the same for the mariadbd profile.

   sudo aa-status | grep -A1 mariadbd

3. Check the line count of the MariaDB profile without opening it line by line.

   sudo wc -l /etc/apparmor.d/mariadbd

Verification and Troubleshooting

• Both aa-status | grep commands above should display the profile path followed by an indication that the named or mariadbd process is running in enforce mode. If BIND9 or MariaDB has not been installed following Chapter 9 or Chapter 20, these lines will not appear at all.
• The mariadbd profile is relatively newly re-released by the MariaDB community, so it is possible that the Ubuntu 26.04 package revision installed on your server marks it in complain mode instead of enforce as a temporary precaution. This does not indicate an error. If so, manually switch it to enforce via sudo aa-enforce /etc/apparmor.d/mariadbd following the pattern in Section 33.3.2.
• An honest note from the field: default profiles like mariadbd above are designed to support default Debian/Ubuntu packaging configurations, including the standard data directory location at /var/lib/mysql as discussed in Section 20.1.1. If a sysadmin later moves the MariaDB data directory to a custom location outside that standard path, such as an additional disk mount point from Chapter 7, this profile might deny access to that new location despite valid Linux permissions, a symptom identical to the warning given for BIND9 in Section 9.2.1.

33.2.2 Nginx and PHP-FPM: Services Running Without Default Profiles

Unlike BIND9 and MariaDB, neither the Nginx nor PHP-FPM packages from Chapter 12 and Chapter 14 include any AppArmor profiles by default as of this writing. This does not mean they are inherently more vulnerable by design, as Nginx and PHP-FPM remain protected by standard Linux permission layers and the principle of least privilege via the www-data user active since installation. However, without an AppArmor profile, there is no additional MAC layer restricting these processes if they are compromised, a gap we address in Section 33.4 via custom profiles.

Practical Steps

1. Search for profiles for Nginx or PHP-FPM in the AppArmor configuration directory.

   ls /etc/apparmor.d/ | grep -iE "nginx|php"

2. Confirm via aa-status that neither process is listed as profiled.

   sudo aa-status | grep -iE "nginx|php-fpm"

Verification and Troubleshooting

• Both commands above should return no output, confirming that Nginx and PHP-FPM operate without an additional MAC layer. This is not an error, but confirmation of the baseline state that makes practicing Section 33.4 relevant.
• The same note applies to Apache from Chapter 13, as touched upon in Section 17.3.3 during certificate troubleshooting: Apache on Ubuntu also runs without a strict path-restricting AppArmor profile, meaning Apache errors stem far more frequently from incorrect Linux permissions than from AppArmor denials.

33.2.3 AppArmor Behind Container Engines: Docker and LXD

A reasonable misconception following the previous sections is assuming AppArmor is only relevant for services running directly on the host. In reality, container technologies used since Section VII, namely Docker in Chapter 26 and LXD in Chapter 29, rely on AppArmor as a primary isolation layer, often without direct visibility.

Docker automatically applies a default profile named docker-default to every running container unless overridden via the --security-opt option. This profile restricts containers from performing high-risk actions against the host, such as loading kernel modules or writing directly to most paths under /proc and /sys. LXD takes a similar but stricter approach: every container instance launched via lxc launch since Section 29.3 is automatically confined by its own AppArmor profile, combined with user namespaces in unprivileged containers (the LXD default), preventing one container from reading processes or sending signals to another container or the host.

Practical Steps

1. Confirm the active AppArmor profile on a running Docker container, replacing container_name with an actual container on your server.

   docker inspect --format '{{.AppArmorProfile}}' container_name

Verification and Troubleshooting

• A healthy output displays docker-default. If the result is empty or shows unconfined, the container was run with --security-opt apparmor=unconfined or the --privileged flag, meaning all MAC layers from AppArmor are removed for that container. Sysadmins must ensure there is a compelling reason for this configuration, such as temporary debugging, rather than using it as a shortcut to bypass permission errors that could be solved with custom profiles.
• An important point to emphasize: the docker-default profile and default LXD confinement are designed to isolate the container from the host and other containers, not to tightly restrict specific applications running inside the container. Nginx or PHP-FPM running inside a Docker container still does not automatically receive a detailed profile like the one built manually in Section 33.4, because the docker-default profile operates at the container level as a whole, rather than at the individual process level inside it.

33.3 Enforce Mode vs Complain Mode

Previous sections repeatedly mentioned "enforce". This section completes the concept and demonstrates how to safely switch modes, an essential prerequisite before building custom profiles in Section 33.4.

33.3.1 Fundamental Differences Between the Two Modes

An AppArmor profile operates in one of two modes. Enforce mode means profile rules are strictly enforced by the kernel; unauthorized access is denied and logged. Complain mode means policy is not enforced at all; all access attempts are allowed regardless of profile rules, but policy violations are still logged just like in enforce mode. Complain mode is designed specifically for profile development when required application access is not yet fully known, whereas enforce mode is the target state once the profile is mature enough to protect the service.

33.3.2 Practice: Switching Modes on the MariaDB Profile

The mariadbd profile from Section 33.2.1 serves as a safe exercise target for mode switching because it is mature and tested, minimizing the risk of disrupting database services during temporary experimentation.

Practical Steps

1. Switch the mariadbd profile to complain mode.

   sudo aa-complain /etc/apparmor.d/mariadbd

2. Confirm the mode change using aa-status.

   sudo aa-status | grep -B2 mariadbd

3. Return to enforce mode after completing the test.

   sudo aa-enforce /etc/apparmor.d/mariadbd

Verification and Troubleshooting

• The output line from grep in step two should move from the profiles are in enforce mode section to profiles are in complain mode, and then return after running step three.
• Violations logged by AppArmor, in both complain and enforce modes, enter the kernel audit log rather than standard application logs. Monitor them with the following command to see which lines of the mariadbd profile were triggered.

  sudo journalctl -k | grep -i apparmor | grep -i mariadb

• If enforce mode on a production server blocks legitimate access, do not remove the profile as a quick fix. First switch it to complain mode via aa-complain, review the denial log, and add missing rules via a local override file at /etc/apparmor.d/local/mariadbd. This pattern mirrors the jail.local pattern in Fail2ban from Section 32.2.2: the main file is never edited directly to prevent overwritten changes during package upgrades, while all custom rules reside in a separate local file.

33.4 Building a Simple Custom Profile for Nginx

This final section addresses the gap demonstrated in Section 33.2.2: Nginx running without an AppArmor profile. Rather than writing a profile from scratch line by line, which is error-prone and time-consuming, we use aa-genprof, a tool that monitors actual application behavior and constructs a profile based on observed access patterns.

33.4.1 The aa-genprof Workflow: Observing Nginx in Real Time

aa-genprof works interactively: upon execution, it places Nginx into complain mode and waits for "training" by using the application normally in another terminal window, leveraging tmux sessions introduced in Section 3.3.1. Every access logged during training is evaluated by aa-logprof for inclusion in the profile.

Practical Steps

1. Open a new tmux session and split it into two panes, following the workflow from Section 3.3.1.
2. In the first pane, begin profile creation for the Nginx binary.

   sudo aa-genprof /usr/sbin/nginx

   aa-genprof will display a message stating Nginx has been moved to complain mode and will prompt you to press S to scan logs after completing application training.
3. Without closing the first pane, switch to the second pane and execute standard Nginx operations using the example.local virtual host from Section 12.2.1 and the HTTPS configuration from Section 17.3.2.

   curl http://example.local
   curl -k https://example.local
   sudo systemctl reload nginx
   sudo nginx -t

4. Return to the first pane and press S to scan the recently generated access logs.

Verification and Troubleshooting

• If the first pane does not display the (S)can option, verify that apparmor-utils is installed as described in Section 33.1.2, as aa-genprof is part of that package.
• The more comprehensive the actions taken in the second pane, the more complete the generated profile will be. If other Nginx features are used on the production server, such as the reverse proxy from Section 12.3.2 or log rotation from Section 12.4.2, execute those operations before pressing S to capture relevant access patterns.

33.4.2 aa-logprof: Reviewing Suggestions and Enabling Enforce Mode

After scanning, aa-genprof invokes aa-logprof behind the scenes to present recorded access events as profile proposals, offering options for approval.

Practical Steps

1. Review each proposal as it appears. It will look similar to this for Nginx log file access:

   Profile:  /usr/sbin/nginx
   Path:     /var/log/nginx/access.log
   New Mode: owner w
   
     [1 - owner w]
   (A)llow / [(D)eny] / (I)gnore / Audi(t) / Abo(r)t / (F)inish

   Press A to approve valid access required by Nginx, such as write access to its log directory at /var/log/nginx/, read access to /etc/nginx/ including sites-enabled, and access to the PHP-FPM socket at /run/php/php8.4-fpm.sock from Section 14.3.1 if Nginx passes PHP requests.
2. After reviewing all proposals, press F to complete the process. aa-genprof automatically writes the final profile to /etc/apparmor.d/usr.sbin.nginx and enables enforce mode.

Verification and Troubleshooting

• Confirm that the new profile is loaded and active in enforce mode.

  sudo aa-status | grep -A1 nginx

• Retest the endpoints used during profile training, now operating under real enforcement rather than complain mode.

  curl http://example.local
  curl -k https://example.local

• If a curl request fails or Nginx returns a 500/502 error that was not present before, the generated profile likely missed a required access path. Switch to complain mode to restore service immediately, inspect denial logs, and rerun aa-logprof /usr/sbin/nginx to append missing rules.

  sudo aa-complain /etc/apparmor.d/usr.sbin.nginx
  sudo journalctl -k | grep -i apparmor | grep -i nginx

• Once missing access rules are appended and verified safe, return to enforce mode.

  sudo aa-enforce /etc/apparmor.d/usr.sbin.nginx

• Handle custom profiles with greater care than built-in profiles like mariadbd from Section 33.2.1. The mariadbd profile has been validated across thousands of test cases, whereas the custom Nginx profile only covers manually tested scenarios. If significant Nginx configuration changes occur later, such as adding new virtual hosts with different directory structures, repeat the complain-train-logprof cycle rather than assuming the existing profile automatically covers new paths.

At this point, we have completed the fourth layer of defense in depth from Section 30.1.2: understanding AppArmor as a Mandatory Access Control implementation operating via path entries and capability entries, verifying that BIND9 and MariaDB are protected by default profiles while Nginx and PHP-FPM are not, identifying AppArmor as the underlying container isolation layer in Docker and LXD operating at the container level, practicing safe transitions between complain and enforce modes using the mariadbd profile, and constructing a custom profile for Nginx from scratch using aa-genprof and aa-logprof. AppArmor has clear limitations: it is only as strong as its profile scope, cannot identify specific actor identities or exact event timelines outside technical denial logs, and requires disciplined sysadmins to update profiles whenever application configurations change significantly. Chapter 34 will conclude Section VIII with the fifth and final layer of our defense in depth map: auditing and least privilege, covering auditd for comprehensive system activity logging, least privilege principles in multi-user server environments, and a foundational security checklist to verify before declaring a server production-ready.