Basic Docker Concepts

After Docker is installed and passes verification, the next step is not to jump straight into creating images or running containers without understanding how it works. Docker is not actually a single standalone application, but rather a collection of separate components that communicate with each other via APIs: there is a component that receives commands from us, a component that actually executes containers at the kernel level, and an image registry that can be accessed from any machine. Understanding this component map from the beginning will be very helpful when debugging is needed, whether by Developers whose containers suddenly refuse to start, or Sysadmins/DevOps Engineers who need to trace why the daemon on a production server has stopped responding. This chapter discusses four foundations that will continue to be used throughout all other sub-chapters in this series: Docker architecture along with its main components, the fundamental differences between images and containers, the container lifecycle from creation to deletion, and an introduction to the Dockerfile as a way to define images as code.

3.1 Docker Architecture

Docker uses a client-server architecture. There are two core components always involved every time we type a docker command: the Docker client which receives commands from us via the terminal, and the Docker daemon (dockerd) which actually performs those requests, such as building images, running containers, or managing networks and volumes. The client and daemon communicate via a REST API, either through Unix sockets on the same machine (default on Linux and macOS), named pipes on Windows, or through network interfaces if the daemon resides on another machine.

Because communication is based on a REST API, the Docker client is actually just one way to access the daemon, not the only way. We can prove this directly by sending a request to the daemon Unix socket using curl, without involving the Docker CLI at all.

curl --unix-socket /var/run/docker.sock http://localhost/version

The command above returns API version information in JSON format, exactly like what is displayed by docker version, because both call the exact same endpoint in the Docker Engine API. This trick is useful in the field when Sysadmins/DevOps Engineers need to integrate monitoring or automation tools into Docker without installing the full Docker CLI on that machine.

Behind the daemon, there are two more layers that are rarely touched directly but are important to understand. containerd is a high-level container runtime invoked by dockerd to manage the container lifecycle: pulling images, creating containers, starting, and stopping them. Docker donated containerd to the Cloud Native Computing Foundation, so the same runtime is also used by many other container platforms outside of Docker. Beneath containerd is runc, a low-level runtime that actually creates namespaces and cgroups in the kernel to isolate container processes. runc follows the Open Container Initiative (OCI) specification, so containers run by Docker essentially comply with the same open standards used by other runtimes.

The final component in this architecture is the registry, which is where images are stored and distributed. Docker Hub is the default public registry used by the Docker client when we run docker pull or docker push without explicitly specifying another registry. Beyond Docker Hub, organizations also commonly run their own private registries to store internal images that must not be publicly accessible, a common scenario in work environments for Sysadmins/DevOps Engineers handling enterprise applications.

One thing that Sysadmins/DevOps Engineers managing multi-tenant servers should note: anyone who can access the daemon socket essentially has full control over the host, because the daemon runs with high privileges to create namespaces and attach mounts to the host filesystem. Restricting access to the daemon Unix socket, or not exposing the Docker Engine API over the network without TLS, is a basic security practice that must be addressed before the server is used in production. For an Attacker, this socket is an easy target: once access is gained, whether through a container that intentionally or unintentionally mounts /var/run/docker.sock into itself, or via an API exposed to the network without authentication, the Attacker can instruct the daemon to create a new container with a bind mount to the host root filesystem, which in practice is equivalent to gaining full root access to the machine.

3.2 Image vs Container

An Image is a read-only template containing all the instructions needed to create a container: application filesystem, dependencies, environment variables, and the command executed when the container starts. An image is composed of multiple layers stacked on top of each other, and each layer is immutable, never changing after creation. A Container is a runnable instance of an image. When a container is created, Docker adds a thin writable layer on top of all the image layers, using a copy-on-write mechanism via a storage driver such as overlay2. All changes occurring during container execution, whether new files, configuration modifications, or application logs, are recorded in this writable layer rather than modifying the original image.

An analogy often used to explain this relationship is classes and objects in object-oriented programming. An image is equivalent to a class, a definition specifying structure and behavior. A container is equivalent to an object, a concrete instance created from that class. Just as a single class can create many independent objects, a single image can be used to run multiple containers simultaneously, each with its own writable layer, without interfering with each other.

A practical consequence of this architecture is that containers are disposable. Because the container writable layer is separate from the image, deleting a container will not alter its underlying image, but all data written by the container to that writable layer is also lost once the container is deleted, unless saved via a volume. Let us prove this directly through practice. Run a container from the alpine image that writes a file to its writable layer.

docker run --name demo-layer alpine sh -c "echo 'container data' > /tmp/data.txt"

The container above stops immediately after the echo command finishes running, but the container and its writable layer remain saved until explicitly removed. The docker diff command displays all changes made to the writable layer compared to the original image.

docker diff demo-layer

The output shows the path /tmp/data.txt marked with A (added), proving that the file exists only in the container writable layer, not in the alpine image itself. If you attempt to re-run the docker run --name demo-layer ... command above without removing the old container, Docker will reject it with the error "Conflict. The container name is already in use", because container names must be unique on a single host; remove the old container before creating a new one with the same name. Compare this with the following commands to view locally stored images and created containers.

docker image ls
docker container ls -a

docker image ls displays images as static units, whereas docker container ls -a displays containers as units with states (running, exited, and so forth). Clean up this test container when finished, as exited containers continue to consume disk space until deleted.

docker rm demo-layer

In practice, misunderstandings regarding images versus containers are a common source of confusion for beginner Developers: modifying a file inside a running container via docker exec, then wondering why those changes disappear after the container is recreated. Those changes do not magically disappear; rather, they were never written to the image in the first place, only to the writable layer of the old container that was replaced by the new container.

3.3 Container Lifecycle

Containers do not simply have two basic states (running or not running). Throughout its lifecycle, a container transitions through several distinct Status values recorded and inspectable via docker inspect. Understanding these state transitions is essential for Sysadmins/DevOps Engineers to manage containers properly, especially when designing restart strategies or troubleshooting unexpected container behavior.

The seven statuses above are official values that can appear in the Status field. Once a container is completely removed via docker rm, the removing status exists only for a moment, after which the container disappears entirely; there is no "removed" status to check because no container remains to inspect. The following practical steps focus on the most common everyday transitions: created, running, paused, and exited.

Let us practice each transition step by step. Start by creating a container without running it immediately, using docker create.

docker create --name lifecycle-demo alpine sleep 300

This command only prepares the container in the created state without running anything. Check its status.

docker ps -a --filter name=lifecycle-demo

The STATUS column will show "Created". Start the container with docker start, then re-check its status.

docker start lifecycle-demo
docker ps --filter name=lifecycle-demo

As a note, docker run used daily is actually a combination of two commands, docker create followed by docker start, wrapped into one step for convenience. Next, freeze the running container using docker pause.

docker pause lifecycle-demo
docker ps --filter name=lifecycle-demo

The STATUS column will append "(Paused)" behind the container uptime, indicating that processes inside are frozen. Docker achieves this effect using the freezer cgroup, which freezes all processes inside the container without actually stopping them, similar to pressing a pause button on an application rather than closing it. Return the container to the running state with docker unpause, then stop the container.

docker unpause lifecycle-demo
docker stop lifecycle-demo

docker stop sends a SIGTERM signal to the main container process, giving the application a chance to shut down gracefully, such as closing database connections or finishing active requests. If the process has not stopped after the timeout period (defaulting to 10 seconds for Linux containers), Docker forces termination using SIGKILL. This timeout period can be extended via the -t flag, an important setting for applications requiring more time for a graceful shutdown, such as processes finalizing database transactions.

docker stop -t 30 lifecycle-demo

If a container must be stopped immediately regardless of graceful shutdown, such as a process that is hung and unresponsive to SIGTERM, use docker kill which sends SIGKILL directly without a grace period.

docker kill lifecycle-demo

Finally, remove the container that is no longer needed.

docker rm lifecycle-demo

A container still in running status cannot be removed directly and will return an error; stop it first via docker stop, or force-remove it using the -f flag on docker rm if the container must be cleaned up immediately. Note that docker rm -f bypasses the graceful shutdown process entirely and sends SIGKILL before removing the container, making it suitable for emergency situations but carrying the risk of leaving unwritten disk data if used indiscriminately on containers handling critical production transactions.

On production servers, Sysadmins/DevOps Engineers rarely stop and start containers manually like this. Instead, restart policies are used to determine whether the Docker daemon should automatically restart a container when it stops. There are four options: no (default, never restart automatically), on-failure[:max-retries] (restart only if the container exits with a non-zero exit code, with an optional retry limit), always (always restart a stopped container, including after the Docker daemon itself restarts), and unless-stopped (similar to always, but does not restart after a daemon reboot if the container was manually stopped beforehand).

docker run -d --name web-service --restart unless-stopped nginx

unless-stopped is the most common choice for long-running production services, as containers automatically recover from crashes or server reboots without restarting aggressively if intentionally stopped for maintenance. Restart policies on running containers can also be updated without recreating the container.

docker update --restart unless-stopped web-service

A common pitfall for Developers learning container lifecycles: a container automatically stops as soon as its main process (PID 1) finishes or exits, even if other background processes are still running inside it. This differs from a VM, which remains running as long as the operating system has not shut down. A container essentially wraps a single primary process; if that process exits, the container stops, regardless of any other remaining background tasks.

3.4 Dockerfile Basics

A Dockerfile is a text file containing a sequence of instructions used by Docker to build an image automatically. Each instruction line essentially forms a new layer in the resulting image, and Docker caches these layers so subsequent builds run faster as long as instructions and related files remain unchanged.

The four most fundamental instructions that appear in nearly every Dockerfile are FROM, RUN, COPY, and CMD. FROM specifies the base image as the starting point and must be the first instruction in a Dockerfile (unless preceded by an ARG instruction to define build arguments before FROM). RUN executes shell commands during image build time, typically used to install packages or prepare the environment. COPY copies files from the build context on the host machine into the image filesystem. CMD specifies the default command executed when a container is created from the image, though this command can be overridden when docker run is called with additional arguments.

Let us practice directly by building a simple image. Prepare a new working directory, then create a hello.sh file inside it.

mkdir docker-basics-demo && cd docker-basics-demo

#!/bin/sh
echo "Hello from Docker container!"

Save the content above as hello.sh. Next, create a file named exactly Dockerfile (without an extension) in the same directory.

FROM alpine:3.20
COPY hello.sh /hello.sh
RUN chmod +x /hello.sh
CMD ["/hello.sh"]

FROM alpine:3.20 uses Alpine Linux as the base image due to its small footprint, a common choice for examples and applications that do not require heavy system dependencies. COPY hello.sh /hello.sh copies the script to the root filesystem of the image. RUN chmod +x /hello.sh grants execution permissions to the script during the build process. CMD ["/hello.sh"] executes this script by default whenever a container is created from the image, written in exec form (JSON array), which is recommended over shell form because it avoids an intermediate shell process to execute the command.

Build the image from this Dockerfile using docker build, assigning a name and tag via the -t flag.

docker build -t hello-docker:1.0 .

The dot at the end of the command points to the build context, which is the directory whose contents are sent to the daemon for building, including all files accessible by COPY or ADD instructions. If Docker displays the error "COPY failed: file not found in build context", the most common cause is that hello.sh is not stored in the same directory as the Dockerfile, or the docker build command was executed from the wrong directory; ensure your current working directory is docker-basics-demo before re-running the build command. Once the build completes, run a container from the newly created image to verify the result.

docker run hello-docker:1.0

If the message "Hello from Docker container!" appears in the terminal, the image was successfully built and executed according to the Dockerfile instructions. This build command uses BuildKit by default, the stable backend build engine set as default in both Docker Engine and Docker Desktop, identified by build output showing parallel progress per layer. If you ever need to compare with the legacy builder for troubleshooting purposes, it can still be enabled via the environment variable DOCKER_BUILDKIT=0.

A good practice to establish from your first Dockerfile is creating a .dockerignore file, which operates similarly to .gitignore but serves to exclude specific files or directories from the build context. Without this file, the build context can bloat unnoticed if the working directory contains large folders like node_modules or .git, slowing down the docker build process as the entire directory content is transferred to the daemon even if never referenced by any COPY instruction.

node_modules
.git
*.log

The four instructions above, FROM, RUN, COPY, and CMD, are sufficient to build a fully functional image, but they do not represent the full set of Dockerfile capabilities. ENTRYPOINT is similar to CMD, but is not easily overridden from the outside, making it better suited for fixing the main container command. ENV defines environment variables available inside the container. WORKDIR sets the default working directory for subsequent instructions. EXPOSE documents ports used by the application inside the container. The multi-stage build technique separates compilation stages from runtime stages within a single Dockerfile, ensuring build dependencies like compilers are not carried over into the final production image.