The Operating System

The operating system is the software layer that sits between your program and the physical hardware. It plays two fundamental roles: abstraction — hiding the messy details of hardware behind clean, uniform APIs — and multiplexing — safely sharing hardware resources among many programs running simultaneously.

Without an OS, every program would need to know the exact hardware commands for every keyboard model, every disk controller, every network chip ever manufactured. The OS solves that problem once, for everyone.

Grand Analogy: The OS is a government. The kernel is the executive branch. Processes are citizens. System calls are official request forms. Drivers are embassy translators for foreign devices. The file system is the land registry. The scheduler is traffic management. The memory manager is the housing authority.

User Mode vs. Kernel Mode

Modern CPUs enforce two privilege levels in hardware. User mode (Ring 3) is where all application code runs — it cannot directly touch hardware, peek at other processes' memory, or execute privileged CPU instructions. Kernel mode (Ring 0) is where the OS kernel runs — it has unrestricted access to everything.

This boundary isn't a software convention. The CPU physically refuses to execute privileged instructions from Ring 3 code.

Analogy: User mode is being a regular citizen. You can't walk into the power plant and flip switches. You submit a request (a system call) to the government (the kernel), which does it on your behalf — safely.

System Calls

When your program needs something privileged — reading a file, sending network data, allocating memory — it issues a system call (syscall). A syscall is a controlled, hardware-mediated jump from user mode into the kernel. The kernel performs the work, then returns the result and switches back to user mode.

This is the fundamental security boundary of computing. A buggy or malicious application cannot corrupt the kernel or another process's memory because the hardware prevents it. The only path through is a controlled syscall.

A program is bytes on disk — a recipe. A process is the OS running that program — the act of cooking. You can have many processes running the same program simultaneously (every open tab in your browser is a separate process running the same browser binary).

Process Memory Layout

Every process gets its own private virtual address space, organized into distinct regions:

The text segment holds the compiled instructions (read-only). The data segment holds global and static variables. The heap grows upward as the program dynamically allocates memory. The stack grows downward for function calls and local variables. The OS kernel metadata (PID, state, open files, signal handlers) lives outside the user-visible address space.

The Process Control Block

For every process, the kernel maintains a Process Control Block (PCB) — a data structure tracking everything needed to manage and resume the process:

• PID — unique process identifier
• State — running, ready, blocked, or terminated
• Saved registers — the CPU's register values at the last context switch
• Memory maps — which virtual pages are allocated and where they point
• Open file descriptors — which files, sockets, and pipes are open
• Accounting — CPU time consumed, priority, resource limits

Creating Processes: fork() and exec()

On Unix/Linux, processes are created via fork() — it clones the calling process into a parent and child. The child is an exact copy (PCB, memory, file descriptors). The child then calls exec() to replace its code image with a new program.

pid_t pid = fork();
if (pid == 0) {
    // child process
    execv("/usr/bin/ls", args);
} else {
    // parent — pid is the child's PID
    waitpid(pid, &status, 0);
}

This fork-then-exec pattern seems roundabout, but it's elegant: the child inherits the parent's open files, environment variables, and signal handlers for free, then replaces only its code image.

Process States

A process moves through several states during its lifetime:

Analogy: Think of a restaurant kitchen. Ready = orders waiting to be cooked. Running = the chef is actively cooking this dish. Blocked = waiting on an ingredient from the pantry (I/O) — the chef moves on to another order. Terminated = dish served and done.

Context Switching

Each CPU core can run exactly one process at a time. The OS creates the illusion of simultaneous execution by rapidly switching between processes — a context switch.

A context switch costs 1–10 microseconds and happens thousands of times per second. The cost is real but small: it's why a system with 500 processes doesn't feel 500× slower than one with a single process running.

The Scheduler

The scheduler decides which ready process gets the CPU next, and for how long.

Algorithm	Strategy	Trade-off
FIFO / FCFS	First come, first served	Simple; long jobs starve short ones
Shortest Job First	Shortest task runs first	Optimal avg wait; requires knowing job length
Round Robin	Each gets a time slice, rotate	Fair; lots of context switches
Priority	Highest priority first	Responsive; risk of starvation for low-priority tasks
MLFQ	Multiple queues, dynamic priority	Best real-world balance

Real OS schedulers — Linux's CFS (Completely Fair Scheduler), macOS, Windows — use variants of MLFQ. A time slice (quantum) is typically 1–10 ms. If a process doesn't voluntarily yield the CPU before its slice expires, a timer interrupt preempts it — the OS forcibly takes the CPU back.

A thread is a unit of execution within a process. Every process starts with one thread and can create more. All threads in a process share the same code, heap, and open file descriptors — each thread only has its own stack, CPU registers, and program counter.

Creating a thread is 10–100× cheaper than forking a new process, because there's no need to duplicate the address space.

Analogy: A process is a house (its own address space, utilities, front door). A thread is a person living in the house — they share the kitchen and living room, but each has their own daily schedule and to-do list.

Why Threads?

Three reasons. Parallelism: spread CPU-bound work across multiple cores. Concurrency: keep a UI responsive while a background thread does network I/O. Efficiency: threads communicate through shared memory — no serialization, no copying data between address spaces.

Thread Pools

Creating a thread is cheap, but not free — there's kernel overhead, and unbounded thread creation under load can exhaust system resources. High-throughput servers use a thread pool: a fixed set of pre-created threads that pull tasks from a shared queue. This bounds the overhead and prevents runaway thread creation.

The Danger: Shared State and Race Conditions

Since threads share memory, they can corrupt each other's data. Consider two threads both incrementing a counter:

Thread 1: read counter  → 0
Thread 2: read counter  → 0
Thread 1: write counter → 1
Thread 2: write counter → 1   ← lost update!

Expected: 2    Actual: 1  ← RACE CONDITION

The problem is that read-modify-write is not atomic. The OS can context-switch between any two instructions. You need synchronization.

Synchronization Primitives

Mutex (Mutual Exclusion Lock) — only one thread can hold the lock at a time. All others block until it's released.

pthread_mutex_lock(&mutex);
    counter += 1;   // critical section
pthread_mutex_unlock(&mutex);

Analogy: A mutex is a bathroom with one lock. One person at a time; everyone else waits outside.

Semaphore — a generalized counter that allows up to N threads through simultaneously. A mutex is a semaphore with N=1. Used for resource pools: "at most 10 database connections at once."

Condition Variable — lets a thread block until some condition becomes true, then get notified. The classic producer-consumer pattern:

// Consumer
pthread_mutex_lock(&mutex);
while (queue_empty()) {
    pthread_cond_wait(&cv, &mutex);  // atomically release lock + sleep
}
item = dequeue();
pthread_mutex_unlock(&mutex);

// Producer
pthread_mutex_lock(&mutex);
enqueue(item);
pthread_cond_signal(&cv);  // wake one waiter
pthread_mutex_unlock(&mutex);

Analogy: A condition variable is like a "Now Serving" display at the DMV. You sit and wait until your number is called — you don't busy-poll the counter.

Read-Write Lock — many readers can hold it simultaneously, but a writer gets exclusive access. Optimal for read-heavy workloads like in-memory caches.

Deadlock

Deadlock occurs when two or more threads wait for each other in a cycle — none can proceed.

Four conditions must all hold simultaneously: mutual exclusion, hold-and-wait, no preemption, and circular wait. The practical fix is to always acquire locks in a fixed global order — this eliminates the circular wait condition. Alternatively, use a timeout on lock acquisition and retry if you can't get both locks.

User-Level vs. Kernel Threads

In the 1:1 model (Linux default, Windows), each user thread maps to one kernel thread — the OS scheduler sees each thread and can run them on separate cores.

In the M:N model (Go goroutines, Erlang processes), a runtime multiplexes many lightweight threads onto a smaller pool of kernel threads. This allows millions of goroutines without per-goroutine kernel overhead. The runtime implements its own scheduler to decide which goroutine runs on which kernel thread. The trade-off: the runtime adds complexity, and blocking syscalls require careful handling to avoid stalling the entire kernel thread.

Every process believes it has exclusive access to a vast, contiguous block of memory starting at address 0. This belief is correct — but the address space is virtual, not physical. The OS and hardware collaborate to maintain this illusion while safely sharing real RAM across dozens of processes.

Virtual Memory

Each process has its own virtual address space. Every memory address the CPU generates is virtual. The OS, with help from dedicated hardware, translates each virtual address to a physical RAM address on every memory access — invisibly, in hardware.

Two processes can both have a pointer to address 0x1000. Those virtual addresses map to different physical frames — they never collide.

Pages, Frames, and the Page Table

Memory is divided into fixed-size blocks. A page is a block in virtual memory (typically 4 KB). A frame is a block in physical RAM (same size). Each process has a page table maintained by the OS — a mapping from virtual page numbers to physical frame numbers.

The MMU and TLB

The Memory Management Unit (MMU) is dedicated hardware that performs every virtual→physical translation. Without caching, each memory access would require two memory reads: one to consult the page table, one for the actual data. The Translation Lookaside Buffer (TLB) solves this: a small, very fast cache of recent page table lookups, typically holding 64–1024 entries.

A TLB miss means the CPU must "walk" the page table in memory to find the mapping — expensive. Context switches often flush the TLB, which is one reason they're not free.

Analogy: The page table is a phone book. The TLB is your recent calls list. You check recent history first; you only open the phone book for a number you haven't called before.

Page Faults

Not every page needs to live in RAM. Pages can be stored on disk (swap space) and loaded on demand. When the CPU accesses a virtual address whose page isn't in RAM, the MMU fires a page fault trap:

When RAM fills up and the OS must constantly swap pages to and from disk, the system spends more time moving pages than doing real work — a condition called thrashing. This is why adding RAM to an overloaded system can dramatically improve performance.

Page Replacement

When RAM is full and a new page must be loaded, the OS must evict an existing page:

Algorithm	Strategy	Note
FIFO	Evict the oldest page	Simple; can evict frequently-used pages
LRU	Evict least recently used	Good in practice; expensive to implement exactly
Clock (Second Chance)	Approximates LRU efficiently	Used in real kernels
Optimal (Bélády's)	Evict the page not needed longest	Theoretical ideal; requires predicting the future

malloc and the Heap

When your code calls malloc(n), the allocator finds a free block on the heap. If the heap needs to grow, it calls mmap() or brk() — syscalls that ask the OS for additional pages.

Fragmentation is a chronic problem. External fragmentation: free memory exists but is scattered in small, non-contiguous chunks — a large allocation fails even when total free memory is sufficient. Internal fragmentation: the allocator rounds up to a size class, wasting padding bytes inside each allocation.

Modern allocators (jemalloc, tcmalloc, mimalloc) use size-class bins and per-thread caches to minimize fragmentation and reduce lock contention in multithreaded workloads.

Analogy: External fragmentation is like having 10 empty seats in a theater, but none adjacent — a party of 4 can't sit together. Internal fragmentation is booking a table for 4 when only 3 show up.

Memory Safety

These bugs are the root cause of the vast majority of security vulnerabilities:

• Buffer overflow — writing past an array's end; can overwrite return addresses (the classic exploit vector for shellcode injection)
• Use-after-free — accessing memory after it's been freed; that memory may now contain attacker-controlled data
• Double free — freeing the same pointer twice; corrupts the allocator's internal free list
• Memory leak — forgetting to free; memory accumulates until the process is killed

Garbage-collected languages (Java, Go, Python) prevent these by automating reclamation — at a GC pause and throughput cost. Rust prevents them at compile time through its ownership and borrow-checking system, with zero runtime overhead.

Address Space Layout Randomization (ASLR)

Modern OSes randomize the base addresses of the stack, heap, and shared libraries on every process launch. An attacker who knows a buffer overflow exists still can't hardcode the target address — it changes every run. Combined with stack canaries (a sentinel value placed before the return address, checked on function return) and non-executable stack (NX/XD bit), ASLR forms the modern memory safety baseline.

RAM is volatile — power off and data vanishes. File systems organize data on persistent storage so it survives reboots, crashes, and power cuts.

Storage Media

HDD (Hard Disk Drive): spinning magnetic platters with a read/write head that physically moves across the surface. Sequential access is fast; random access is slow because the head must seek to a new position (~5–10 ms latency, ~100–200 MB/s throughput). The mechanical seek is the bottleneck — touching many small files is dramatically slower than one large file.

SSD (Solid State Drive): flash memory — electrons trapped in transistor gates store bits. No moving parts. Any cell is equally fast to access (~0.05–0.1 ms latency, 500 MB/s–7 GB/s). Random access nearly matches sequential. SSDs have limited write cycles per cell; wear leveling distributes writes evenly across cells to extend lifespan.

Analogy: HDD is a vinyl record player — you must physically move the needle to the right track. SSD is a USB flash drive — any address is equally instant.

File System Structure

At the top level, a formatted disk is divided into regions. The boot block holds bootstrap code loaded by the firmware. The superblock describes the file system itself: total size, block size, counts of free inodes and free blocks. The inode table holds one inode per file. The rest is data blocks — the actual file contents.

Inodes and Directories

An inode stores file metadata: size, permissions, owner, timestamps, and pointers to data blocks. Crucially, it does not store the filename.

A directory is a special file that maps filenames to inode numbers. The filename lives in the directory entry. This means renaming a file within the same directory is instantaneous — only the directory entry changes; the inode and data are untouched.

To resolve the path /home/alice/resume.txt, the OS walks: root inode → root directory → find "home" → follow to inode 2 → find "alice" → inode 100 → find "resume.txt" → inode 500 → read data blocks 1001 and 1002.

Analogy: The inode system is a library. The card catalog (directories) maps book titles (filenames) to catalog numbers (inode numbers). The catalog number points to the shelf location (data block pointers). The actual book (file data) is on the shelf.

Hard Links and Symbolic Links

A hard link is a second directory entry pointing to the same inode. The inode keeps a reference count — the data is freed only when all hard links are deleted. A symbolic (soft) link is a special file containing a path string. If the target is deleted, the symlink dangles — it points to nothing.

Analogy: Hard link = two entries in the phone book for the same person (same actual person). Soft link = a sticky note saying "call Alice's number" — useless if Alice changes her number.

File System Types

File System	Platform	Strengths
ext4	Linux	Journaling, widely supported, reliable
XFS	Linux	High throughput, large files
Btrfs	Linux	Copy-on-write, snapshots, checksums
NTFS	Windows	Journaling, permissions, encryption
APFS	macOS / iOS	Copy-on-write, SSD-optimized
ZFS	Solaris / BSD / Linux	Enterprise: checksums, RAID-Z, snapshots
FAT32	Universal	Simple; 4 GB file limit; USB interop

Journaling

Without protection, a crash mid-write can leave the file system in an inconsistent state — a partially written inode, a block allocated but not linked, a directory with a dangling entry. Journaling writes a log of intended changes before making them. On reboot, the journal is replayed or rolled back atomically, restoring a consistent state without a full fsck scan.

Analogy: Journaling is a surgeon's checklist. Before the first incision, document the plan. If something goes wrong mid-operation, the team knows exactly where to pick up or what to undo.

VFS — Virtual File System

Linux routes all file I/O through the Virtual File System layer — a common interface that dispatches to the specific file system driver below. This is why open(), read(), and write() work identically whether your file is on ext4, an NFS network share, or /proc (a pseudo-filesystem that exposes live kernel state). The application never needs to know which storage backend it's talking to.

Everything is a File

Unix's "everything is a file" principle means the same read()/write() interface works for regular files, directories, devices (/dev/sda), process memory (/proc/1234/mem), network sockets, and pipes. Redirection, shell pipelines, and tools like /dev/null work elegantly because of this uniformity — add a new device, expose it as a file, and every existing tool immediately works with it.

The OS must communicate with hundreds of distinct hardware devices: disks, keyboards, network cards, GPUs, USB peripherals, sensors, sound cards. Each device speaks a different protocol, uses different registers, and has different timing requirements. The challenge is providing a uniform interface without baking in the specifics of every device ever manufactured.

The I/O Stack

When your program calls read(), the request descends through layers. The OS kernel handles buffering, caching, and scheduling (the I/O scheduler reorders disk requests to minimize seek time). A device driver translates the generic kernel request into device-specific commands. A DMA controller moves the data directly between device and memory without involving the CPU for each byte.

Three Ways to Do I/O

1. Polling (Programmed I/O) — the CPU continuously checks a status register to see if the device is ready. Simple to implement but wastes CPU cycles spinning in a loop.

2. Interrupt-Driven I/O — the device raises a hardware interrupt when it's ready. The CPU does other work in the meantime and handles the interrupt when it fires. This is far more efficient — the CPU is useful during the wait.

3. DMA (Direct Memory Access) — a dedicated DMA controller transfers data between device and memory without the CPU handling each byte. The CPU is only interrupted once, when the entire transfer completes. Essential for high-throughput devices like SSDs and NICs.

Analogy: Polling = standing at the microwave watching the timer. Interrupt-driven = setting a timer and reading a book until the beep. DMA = hiring a moving crew to unload the truck while you go to work — they interrupt you once when the whole job is done.

Interrupts

When a device signals an interrupt, the CPU: finishes its current instruction, saves its complete register state onto the kernel stack, looks up the interrupt handler address in the Interrupt Vector Table (IVT), jumps to that handler, processes the event, then restores the saved state and resumes exactly where it left off. The round-trip is microseconds.

Interrupts are used for far more than just I/O. The scheduler fires a timer interrupt every ~1 ms to preempt processes. Page faults go through the interrupt mechanism. System calls use a software interrupt (trap). Hardware errors like divide-by-zero also trigger interrupts.

Device Drivers

A driver is kernel code that translates between the OS's generic I/O interface and a device's specific protocol.

The OS says: "send these bytes on the network." The Intel e1000 driver knows: which memory-mapped registers to write, how to format DMA ring buffer descriptors, how to poll the transmit status register, how to handle transmit errors.

Analogy: A driver is a translator. The OS speaks one language (generic I/O calls). The device speaks another (hardware registers and protocols). The driver translates between them — which is why you need to install drivers for new hardware.

Drivers run in kernel mode, with full access to everything. A buggy driver that dereferences an invalid pointer doesn't just crash itself — it crashes the entire OS. This is the root cause of Windows Blue Screens and Linux kernel panics. It's also why there's a long-running push to move device logic into user-space drivers (FUSE for file systems, io_uring for async I/O, DPDK for networking) — a crash in user space kills only that process.

Booting is a bootstrapping problem: the OS isn't running yet, so how does the OS start? Each layer is just smart enough to load and hand off to the next.

BIOS / UEFI

The instant power is applied, the CPU begins executing from a hardcoded physical address (on x86, 0xFFFFFFF0). That address maps to firmware stored in flash ROM on the motherboard — the BIOS (Basic Input/Output System) or its modern successor, UEFI (Unified Extensible Firmware Interface).

The firmware runs POST (Power-On Self-Test): verify RAM is present and functioning, enumerate attached storage devices, probe PCI/USB buses, initialize the graphics adapter. If POST fails (bad RAM stick, no bootable device), you get a beep code or a firmware error screen.

UEFI goes further than BIOS: it supports Secure Boot (verifying cryptographic signatures before running any code), a full GUI firmware interface, network booting (PXE), and drives larger than 2 TB (MBR's 32-bit sector addressing tops out at 2 TB; GPT fixes this).

Bootloader

The firmware hands control to the bootloader — a small program that knows how to find and load the OS kernel. On Linux systems this is usually GRUB (Grand Unified Bootloader); on Windows, the Windows Boot Manager.

GRUB itself is split into stages because of a constraint: the legacy boot sector is only 512 bytes. The first-stage loader fits there and does nothing except load the larger second stage, which can read file systems, display a boot menu, and load the kernel image plus an initrd (initial RAM disk) — a small temporary file system containing the drivers needed to mount the real root file system.

Kernel Initialization

The kernel unpacks and begins executing in a strict sequence:

1. Set up the CPU's interrupt descriptor table (so hardware exceptions don't crash immediately)
2. Initialize the memory subsystem — set up page tables, enable virtual memory
3. Probe and initialize CPU cores (multi-core systems come up one core at a time)
4. Load built-in drivers; detect and configure attached hardware
5. Mount the root file system (using the initrd if needed to load the right storage driver)
6. Execute /sbin/init — the first real user-space process, PID 1

Init System (systemd)

PID 1 is special: it is the ancestor of every process, it never exits, and orphaned processes are reparented to it. On modern Linux, PID 1 is systemd.

systemd reads unit files that declare services, their dependencies, and when they should start. It resolves the dependency graph and starts services in parallel where possible — sshd doesn't need to wait for the printer daemon. A full boot to login prompt on modern hardware takes 2–5 seconds; most of that time is systemd starting dozens of services concurrently. systemd-analyze blame will show you exactly which service is slowing yours down.

macOS uses launchd for the same role. Older Linux systems used SysV init — a sequential chain of shell scripts that started services one at a time, which is why those systems took minutes to boot.

Analogy: Booting is waking from a deep sleep. BIOS/UEFI is your nervous system doing basic checks — can I feel my limbs? The bootloader is opening your eyes and orienting yourself. The kernel is your brain coming fully online. Init/systemd is your morning routine: coffee, email, calendar — all the services that make you functional, started in parallel.