Operating Systems

Your process runs in user mode and cannot touch hardware, other processes' memory, or the page tables. Everything it needs from the outside world goes through a system call: it puts a number in rax, arguments in registers, and executes the syscall instruction. The CPU switches to ring 0, jumps to a fixed kernel entry point, and the kernel does the work on the process's behalf.

The base cost is roughly 50–100 ns, meaningfully more since Spectre/Meltdown mitigations added page-table isolation. That is cheap once and ruinous a million times a second — which is the entire reason io_uring, batched writes and buffered I/O exist.

The vDSO shortcut

Some calls don't need the kernel at all. gettimeofday() and clock_gettime() are served from the vDSO — a small shared page the kernel maps into every process, containing data the kernel keeps updated. The call becomes an ordinary function call at a few nanoseconds instead of a trap. If you ever wonder why timestamping every log line is affordable, this is why.

Linux schedules tasks. A process and a thread are both a task_struct; the only difference is how much they share. fork() creates a task with a copy of the address space, clone() with CLONE_VM creates one that shares it. "Thread" is a userspace word for a task that shares memory with its siblings.

Copy-on-write fork

fork() does not copy your 8 GB heap. It marks every page read-only in both parent and child and copies a page only when one of them writes to it. This is why forking a large process is fast — and why a forked child can still trigger an out-of-memory kill minutes later, when the writes finally arrive. Redis's background save is the canonical example.

The scheduler

Since kernel 6.6 the default is EEVDF (Earliest Eligible Virtual Deadline First), replacing CFS. Both are fair-share designs: each runnable task accrues virtual runtime, and the one that has had least gets the CPU. EEVDF adds an explicit deadline so latency-sensitive tasks can be served ahead of throughput-hungry ones rather than merely fairly.

What matters operationally is unchanged: fair-share means nobody is starved and nobody is guaranteed. If you need a guarantee, you need SCHED_FIFO, cgroup CPU bandwidth, or pinning — not a nice value.

The kernel uses every spare byte of RAM as page cache — a cache of file contents. A box showing 200 MB free and 60 GB cached is not short of memory; it is doing exactly what it should. Cache is reclaimable on demand. The number to watch is available in /proc/meminfo, which accounts for that.

Reading the memory columns

• VSZ — everything mapped, including memory never touched and files mapped but not read. Nearly meaningless for capacity.
• RSS — resident pages, but shared pages are counted in full against every process sharing them. Sum the RSS of 20 workers and you'll double-count libc twenty times.
• PSS — proportional set size; shared pages divided among sharers. This is the number you want when asking "how much is this process really costing me".

Writeback

A buffered write returns as soon as the page is marked dirty. Flushing happens later, governed by vm.dirty_ratio and vm.dirty_background_ratio. When dirty pages exceed dirty_ratio, writers are throttled synchronously — the application stalls in the kernel until writeback catches up. On a box with lots of RAM and a slow disk, the defaults let gigabytes accumulate and then stall everything at once. Lowering the ratios trades a little throughput for far less latency variance.

A file descriptor is a small integer indexing a per-process table. Behind it the VFS presents one interface — read, write, seek, close — over regular files, sockets, pipes, devices, epoll instances, timers, even other processes' memory. That uniformity is why strace is so useful: almost everything a process does is a read or a write to some fd.

Descriptors are a hard-limited resource, and the limit is per-process, inherited at exec, and usually far lower than people expect. A service that leaks one fd per request will run for hours and then fail all at once with EMFILE — accept() starts failing while the process looks perfectly healthy.

A container is a normal Linux process with two things done to it. Namespaces change what it can see; cgroups limit what it can use. There is no container object in the kernel, which is exactly why containers start in milliseconds and why a container escape is a kernel bug rather than a hypervisor bug.

The namespaces

• pid — its own PID 1 and process tree. Your process is 1 inside, 48122 outside.
• net — its own interfaces, routes, iptables rules, port space.
• mnt — its own mount table; the root filesystem it sees.
• uts — its own hostname.
• ipc — its own shared memory and semaphores.
• user — UID mapping; root inside, unprivileged outside. The one that makes rootless containers possible.
• cgroup — hides the host's cgroup hierarchy from the process.

Why PID 1 matters inside

PID 1 has special duties: it reaps orphaned children and it does not get default signal handlers. Run an application as PID 1 without thinking about it and you get two classic bugs — zombie processes accumulating because nothing reaps them, and SIGTERM being ignored so every deploy waits the full termination grace period and then gets SIGKILLed. That is what --init and tini exist to fix.

Two different OOM paths exist and they behave differently. Global OOM fires when the whole machine is out and the kernel picks a victim by oom_score — roughly, whoever is using the most, adjusted by oom_score_adj. cgroup OOM fires when a single cgroup hits its memory.max, and kills something inside that cgroup only. The host has plenty of free memory; the container still dies.

This is the entire explanation for the most confusing Kubernetes failure mode: a pod is OOMKilled while the node's memory graph shows 40% used. The node was never the constraint. The container's limit was.

What counts against the limit

Under cgroup v2, the charge includes anonymous memory and page cache the cgroup caused. A process that reads a lot of files can be OOMKilled for cache it does not need and would happily give back — the kernel does try to reclaim first, but a fast enough reader can outrun reclaim. This is why a batch job that streams large files needs a limit well above its apparent working set.

Signals are the kernel's interrupt mechanism for processes. Most have a default disposition: SIGTERM terminates, SIGKILL terminates and cannot be caught, SIGSEGV dumps core. A process can install a handler for anything except SIGKILL and SIGSTOP.

PID 1 is the exception. The kernel does not apply default dispositions to it. A process running as PID 1 with no explicit SIGTERM handler simply ignores the signal. Kubernetes sends SIGTERM, waits terminationGracePeriodSeconds (30 by default), then SIGKILLs. If your entrypoint is the application itself and it has no handler, every single pod deletion takes the full 30 seconds and ends in a hard kill — connections dropped, in-flight requests lost, and a rolling deploy that takes half an hour.

Buffered I/O is the default: reads are served from page cache when possible, writes return once dirty. Great throughput, unpredictable latency, and a copy between kernel and user buffers on every call.

Direct I/O (O_DIRECT) bypasses the page cache entirely and DMAs straight into your buffer. Databases use it because they maintain a better cache than the kernel can — they know which pages matter. It demands aligned buffers and aligned offsets, and it is slower for anything that would have hit cache.

io_uring replaces the syscall-per-operation model with two shared ring buffers between userspace and kernel. You write submission entries into a ring, the kernel writes completions into another, and with polling mode you can do sustained I/O with zero syscalls. For anything issuing hundreds of thousands of operations a second, this is the difference between spending 40% of your CPU in kernel entry overhead and spending almost none.