Part 5 of 10

Data Structures and Algorithms

The core toolkit for organizing data and solving problems efficiently — and the complexity intuition to choose the right structure.

Ch. 21

Data Structures: Organizing Data

Data structures are the vocabulary of software engineering. Before writing a single line of algorithmic logic, the choice of data structure often determines whether a solution runs in milliseconds or crashes a server. This chapter covers the structures every engineer uses daily — not just what they are, but when and why to reach for each one.

Abstract data type vs. implementation. A stack is an abstract data type (ADT): a contract that says "push, pop, peek, all from one end." A dynamic array or a linked list is a concrete data structure that can satisfy that contract. The ADT is the interface; the data structure is the implementation. This distinction matters because the same ADT (say, a "map" or a "priority queue") can be backed by very different structures with very different performance profiles — and choosing the right backing is most of the engineering. Throughout this chapter, watch for where the abstraction (what you can do) ends and the mechanism (how memory is laid out) begins.

The two questions to ask about any structure: (1) How is it laid out in memory? — this dictates cache behavior, which dominates real-world speed far more than asymptotics suggest. (2) Which operations does it make cheap, and which does it make expensive? — there is no free lunch; every structure trades one for another.

Arrays

Analogy: A row of numbered mailboxes. You know exactly where box #3 is without searching — instant access. But inserting a new box in the middle means physically shifting everything down.

An array stores elements in contiguous memory. That contiguity is the key insight: the CPU computes any element's address as base + index × element_size, giving O(1) random access. Contiguous memory also means arrays are cache-friendly — loading one element often loads neighboring elements into the CPU cache for free.

The cost of this contiguity is rigidity: inserting or deleting in the middle requires shifting all subsequent elements — O(n). Modern languages (Python lists, Java ArrayList, C++ vector) use dynamic arrays that double in capacity when full. Individual resizes are O(n), but they happen infrequently enough that the amortized cost per append is still O(1).

Reach for arrays when you need fast indexed access and iteration, and insertions/deletions are rare or happen only at the end.

Strings

A string is, at its core, an array of characters — but the details are where careers are made and bugs are born. Three points every engineer should internalize:

Encoding. A character is not a byte. In ASCII the two coincided (one byte per character), but modern text is Unicode, usually stored as UTF-8 — a variable-width encoding where a code point takes 1–4 bytes. So len(s) may count code points while the underlying buffer holds more bytes, and indexing "the 5th character" is not necessarily a constant-offset memory read. Emoji and combining accents can even span multiple code points (grapheme clusters). When you slice or reverse a string naively, this is where it breaks.

Immutability. In Python, Java, C#, and JavaScript, strings are immutable — every "modification" allocates a new string. This makes strings safe to share and hash, but it turns naive concatenation in a loop into an O(n²) trap:

# O(n²): each += copies the whole accumulated string
s = ""
for chunk in chunks:
    s += chunk

# O(n): build a list, join once
s = "".join(chunks)

This is the single most common accidental-quadratic bug in production code. Languages that need cheap mutation provide a separate builder type (StringBuilder, io.StringIO, Rust's String vs &str).

Specialized string structures. When text is huge and edited constantly (think a code editor's buffer), a flat array is too expensive to mutate. A rope — a balanced tree of small string chunks — gives O(log n) insert/delete in the middle. For fast substring search and indexing, suffix arrays and suffix trees preprocess a string so that "does pattern P occur?" becomes O(|P| log n) or better; they underpin bioinformatics (DNA alignment) and full-text search.

Linked Lists

Analogy: A scavenger hunt. Each clue tells you where the next clue is. To find clue #5, you must follow 1 through 4 — no shortcut. But inserting a new clue anywhere is just a pointer update.

Each node holds a value and a pointer to the next node. There's no contiguous memory, so indexing requires traversal — O(n). But insertion and deletion at a known position are O(1) since you only update two pointers.

Doubly linked lists add a backward pointer, enabling O(1) removal when you hold a reference to the node itself — without needing to find the previous node. This property makes them ideal for LRU caches, browser history, and the Linux kernel's scheduler.

The hidden cost of linked lists in practice is cache locality: nodes scatter across memory, causing frequent cache misses. For most workloads, a dynamic array wins despite its theoretically worse insert complexity, because fewer cache misses outweigh worse asymptotic behavior on modern hardware.

Reach for linked lists when you have frequent insertions and deletions at arbitrary positions and you hold references to those positions.

Stacks (LIFO — Last In, First Out)

Analogy: A stack of plates. You always add and remove from the top.

Stacks enforce a discipline where the most recently added item is always processed first. Typically backed by a dynamic array or linked list.

Operation Complexity
push O(1)
pop O(1)
peek top O(1)

Used for: function call management (the call stack is a stack), undo/redo operations, expression parsing (checking balanced brackets), depth-first search, backtracking algorithms.

Queues (FIFO — First In, First Out)

Analogy: A line at a grocery store. First in, first served.

  • Deque (double-ended queue): insert and remove from both ends in O(1) — used in sliding window algorithms
  • Priority Queue: dequeue by priority, not arrival order — implemented as a heap (next section)

Used for: breadth-first search, task scheduling, rate limiting, event queues, print spoolers, message brokers.

Hash Tables

Analogy: A coat check. Hand over your coat (value) and get a ticket number (the hash of your key). Retrieving it is direct — no searching. Collisions are two coats assigned the same hook, requiring a strategy to resolve.

Hash tables are arguably the most practically important data structure. They map keys to values with average O(1) lookup, insert, and delete.

How it works: A hash function converts a key into a bucket index. Two strategies for handling collisions:

  • Chaining: each bucket is a linked list of entries
  • Open addressing: on collision, probe adjacent buckets (linear probing, quadratic probing, Robin Hood hashing)

The load factor (entries ÷ buckets) governs performance. When it exceeds ~0.75, the table resizes — typically doubles — and rehashes all entries. This is O(n) work, but it happens infrequently enough that amortized insert stays O(1).

Worst case is O(n) if every key hashes to the same bucket. This is a practical attack vector called hash flooding, which is why Python and Java use randomized hash seeds by default.

Reach for hash tables when you need fast lookup by key, deduplication, or frequency counting.

Hash map vs. hash set. A set is a hash table that stores only keys (membership), a map stores keys with associated values. Same machinery, different payload. A multiset (or "bag," Python's collections.Counter) stores a count per key — the natural tool for frequency analysis.

Sets and Bitsets

A set answers one question fast: "is x a member?" Backed by a hash table it gives O(1) membership, insert, and delete, plus the algebra of sets — union, intersection, difference:

a = {1, 2, 3, 4}
b = {3, 4, 5, 6}
a & b   # intersection → {3, 4}
a | b   # union        → {1, 2, 3, 4, 5, 6}
a - b   # difference   → {1, 2}

A sorted set (backed by a balanced tree, covered below) trades O(1) membership for O(log n) but adds ordered iteration and range queries.

When your universe of possible elements is small and dense — say "which of these 64 flags are set?" — a bitset is dramatically more efficient. Each element is one bit in an integer or array of integers, so a set of thousands of items fits in a cache line or two, and set operations become single CPU instructions:

permissions = 0b0000          # empty set of flags
READ, WRITE, EXEC = 1, 2, 4
permissions |= READ | WRITE   # add elements
has_write = bool(permissions & WRITE)   # membership test

Bitsets power Unix file permissions, feature flags, chess board representations (a 64-bit "bitboard" per piece type), and database bitmap indexes. The whole set fits in registers, so union/intersection of millions of elements runs at memory-bandwidth speed.

Trees

Binary Search Trees (BST)

Analogy: Guessing a number. "Greater than 50? Yes. Greater than 75? No…" Each question eliminates half the possibilities. A balanced BST makes every search like this.

A BST enforces: left child < parent < right child. Search, insert, and delete are O(log n) — if the tree stays balanced. A pathological BST degrades to a linked list (O(n) all operations) if you insert already-sorted data. Deletion is particularly subtle: removing a node with two children requires finding and splicing in the in-order successor.

Balanced BSTs (AVL, Red-Black Trees)

Self-balancing variants guarantee O(log n) for all operations by rotating nodes to maintain balance. A rotation is a local, O(1) pointer rearrangement that pivots a parent and child while preserving the BST ordering — it just redistributes height. The art is detecting imbalance after an insert/delete and applying the right one or two rotations.

  • AVL trees keep every node's two subtrees within height 1 of each other (the "balance factor" is −1, 0, or +1). This tight balance means faster lookups but more rotations on write — ideal for read-heavy workloads.
  • Red-Black trees allow looser balance (the longest root-to-leaf path is at most twice the shortest), which means fewer rotations on write at the cost of slightly taller trees. This is the pragmatic default and the more common choice in standard libraries.

Used in:

  • Linux kernel's scheduler (the completely-fair scheduler uses a red-black tree) and virtual memory subsystem
  • Java's TreeMap and TreeSet
  • C++ std::map and std::set

A Red-Black tree of n nodes has height at most 2 × log₂(n+1), guaranteeing O(log n) always. The reason any of this matters: a plain BST has no such guarantee, and adversarial or simply sorted input silently degrades it to a linked list. Self-balancing is the price of a worst-case guarantee.

Skip Lists

Analogy: Express lanes on a highway. The bottom lane stops at every exit; lanes above skip most exits. To reach exit 47 you ride the highest lane until the next stop would overshoot, drop down a lane, repeat — arriving in a handful of hops instead of crawling exit by exit.

A skip list is a sorted linked list with extra "express" forward pointers. Each node is promoted to higher levels with probability ½, so roughly half the nodes appear in level 1, a quarter in level 2, and so on — a randomized structure that is balanced in expectation rather than by explicit rotations.

Operation Complexity
search O(log n) expected
insert O(log n) expected
delete O(log n) expected

The appeal over a balanced tree is simplicity and concurrency: there are no rotations to reason about, and inserts touch only a few pointers, which makes lock-free and fine-grained-locking implementations far easier. That's why skip lists back the concurrent ordered maps in Java (ConcurrentSkipListMap), the memtables in Redis sorted sets and LevelDB/RocksDB, and many in-memory databases.

Heaps

Analogy: A corporate hierarchy where every manager is always more senior than their direct reports. The CEO (root) is always the most senior person in the company.

A heap is a complete binary tree with the heap property: each parent ≥ its children (max-heap) or ≤ its children (min-heap). Critically, heaps are stored as arrays — children of node at index i live at 2i+1 and 2i+2 — giving cache-friendly access with zero pointer overhead.

Operation Complexity
peek max/min O(1)
insert (bubble up) O(log n)
extract max/min (bubble down) O(log n)
build from n items O(n)

The O(n) build is a non-obvious result: even though each insertion is O(log n), building bottom-up is provably O(n).

Used for: priority queues, Dijkstra's algorithm, heap sort, finding the k-th largest element, median maintenance.

Graphs

Analogy: A city map. Vertices are intersections; edges are roads. A directed graph has one-way streets. A weighted graph shows distances. Google Maps is essentially a massive weighted directed graph.

Types:

  • Directed vs Undirected: one-way vs two-way connections
  • Weighted vs Unweighted: edges carry costs (distance, latency, bandwidth) or not
  • Cyclic vs Acyclic: may or may not contain loops
  • DAG (Directed Acyclic Graph): directed with no cycles — the data model for dependency graphs, build systems, and task pipelines

Representation — the choice matters for algorithm performance:

Adjacency List Adjacency Matrix
Space O(V + E) O(V²)
Edge lookup O(degree) O(1)
Iterate neighbors O(degree) O(V)
Best for Sparse graphs Dense graphs
Real example Social networks Flight connections

For most real-world graphs (road networks, the web, social graphs), adjacency lists win because the graph is sparse — most nodes connect to a tiny fraction of all other nodes.

Union-Find (Disjoint Set Union)

Analogy: Friend groups at a party. To check if two people are in the same group, you each point to your group's "representative." Merging two groups is just having one representative point at the other.

Union-Find tracks a collection of disjoint sets and answers one question blazingly fast: are these two elements in the same set? It supports two operations — find(x) (which set does x belong to?) and union(x, y) (merge two sets). Each set is a tree; the root is the set's representative.

Two optimizations make it astonishingly efficient:

  • Union by rank/size: always attach the smaller tree under the larger, keeping trees shallow.
  • Path compression: during find, point every node visited directly at the root, flattening the tree for next time.
parent = list(range(n))
rank = [0] * n

def find(x):
    while parent[x] != x:
        parent[x] = parent[parent[x]]   # path compression
        x = parent[x]
    return x

def union(a, b):
    ra, rb = find(a), find(b)
    if ra == rb:
        return
    if rank[ra] < rank[rb]:
        ra, rb = rb, ra
    parent[rb] = ra
    if rank[ra] == rank[rb]:
        rank[ra] += 1

With both optimizations, m operations on n elements run in O(m · α(n)), where α is the inverse Ackermann function — effectively a constant (less than 5) for any input that fits in the universe. It is one of the most beautiful results in the field: near-constant amortized time from two tiny tricks.

Used for: Kruskal's minimum-spanning-tree algorithm, detecting cycles as you add edges, connected-components in image processing, network connectivity, and "are these accounts in the same fraud ring?" clustering.

Segment Trees and Fenwick Trees

Suppose you have an array and a stream of two interleaved requests: "what is the sum (or min, or max) of elements between index i and j?" and "update element k." Recomputing a range sum from scratch is O(n) per query; precomputing all sums breaks the moment a value changes. Segment trees resolve the tension — both query and update become O(log n).

The idea: build a binary tree where each leaf is one element and each internal node caches an aggregate over its children's range. A range query stitches together O(log n) precomputed nodes; a point update walks one root-to-leaf path, refreshing the aggregates along the way. With lazy propagation, even range updates ("add 5 to everything between i and j") become O(log n).

A Fenwick tree (Binary Indexed Tree) is a more compact, lower-constant-factor structure for the common case of prefix sums with point updates. It uses the binary representation of indices to navigate, fits in a single array, and is a handful of lines of code — the go-to in competitive programming when you only need sums.

Used for: range queries in analytics and databases, computational geometry (sweep-line algorithms), counting inversions, and any "running aggregate over a changing array" problem.

Tries (Prefix Trees)

Analogy: A choose-your-own-adventure book where each letter is a choice and complete words are endpoints. Sharing a prefix means sharing a path — "car", "cars", and "cat" all travel through c → a before diverging.

A trie is a tree where each node represents a character, and paths from root to marked nodes spell out stored words. Lookup, insert, and delete for a word of length k are all O(k), regardless of how many words the trie contains.

The real advantage is prefix queries: finding all words starting with "car" is a natural tree traversal from the node at depth 3, with no scanning of unrelated words.

Used for: autocomplete (every search box you've used), spell checking, IP routing tables (longest prefix match), word games, dictionary lookups with prefix constraints.

B-Trees and B+ Trees

A B-tree is not a binary tree. Each node can hold hundreds or thousands of keys. This distinction is profound — and it exists because of the physics of disk storage.

Disk reads are slow (milliseconds) but read whole blocks at once (typically 4–16 KB). A B-tree node is sized to fit in exactly one disk block, so each level of the tree costs exactly one disk read. With a branching factor of 1000, a tree of height 3 can index a billion entries with just 3 disk reads.

A binary tree of the same billion entries would need 30 levels — 30 disk reads, 10× slower.

Analogy: If a BST is finding a word by flipping one page at a time, a B-tree is using the table of contents — hundreds of entries per page — to jump directly to the right chapter, then section, then entry.

B+ trees (what databases actually use) store all actual data in leaf nodes and keep internal nodes as pure indexes. Leaf nodes are linked in a sorted list, enabling efficient range scans — WHERE age BETWEEN 25 AND 35 is a B+ tree scan.

Used for: every database index (MySQL InnoDB, PostgreSQL, SQLite), every file system (HFS+, NTFS, ext4, APFS).

Bloom Filters — Trading Certainty for Space

Analogy: A bouncer with a fuzzy memory. Ask "have I seen this name?" and a "no" is always trustworthy, but a "yes" might be a mix-up with someone who shares features. The upside: the bouncer remembers a guest list of millions using a scrap of paper.

A Bloom filter is a probabilistic set that answers membership using a bit array and k hash functions. To add an item, hash it k ways and set those k bits. To test membership, check those same bits: if any is 0 the item is definitely absent; if all are 1 the item is probably present — with a tunable false-positive rate, but never a false negative.

The payoff is memory: about 10 bits per element gives a ~1% false-positive rate, regardless of how large the elements themselves are. A Bloom filter holding a billion URLs fits in roughly a gigabyte; the equivalent hash set would need far more. The cost is that you cannot iterate it, cannot delete from a basic one, and must accept occasional false positives.

Used for: the canonical pattern is "check the cheap filter before the expensive lookup." Databases (Cassandra, HBase, RocksDB) consult a Bloom filter before touching disk, skipping I/O for keys that definitely aren't there. CDNs use them to avoid caching one-hit-wonder URLs; browsers once used them for malicious-URL checks; spell-checkers and dedup pipelines use them to skip work. Variants — Counting Bloom filters (support deletion), Cuckoo filters (deletion + better locality), and HyperLogLog (cardinality estimation) — round out the family of "approximate, but tiny" structures.

Composite Structures in Practice: the LRU Cache

Real systems rarely use a single textbook structure — they compose them so each covers the other's weakness. The canonical example is the LRU (Least Recently Used) cache, which must do two things in O(1): look up a value by key, and evict the least-recently-used entry when full.

No single structure does both. A hash map gives O(1) lookup but has no notion of recency order; a doubly linked list maintains order with O(1) splicing but has no fast lookup. Combine them:

  • A hash map maps each key to its node in the list.
  • A doubly linked list orders nodes from most- to least-recently-used.

On access, the hash map finds the node in O(1) and the list moves it to the front in O(1). On eviction, drop the tail. Python exposes exactly this via functools.lru_cache and collections.OrderedDict. The lesson generalizes: when no one structure fits, the answer is usually a small, disciplined combination of two.

Summary

Structure Access Search Insert Delete Sweet Spot
Array O(1) O(n) O(n) O(n) Random access, iteration
Linked List O(n) O(n) O(1)* O(1)* Frequent mid-list changes
Hash Table N/A O(1) avg O(1) avg O(1) avg Key-value lookup
Set / Bitset N/A O(1) O(1) O(1) Membership, flags, set algebra
BST (balanced) O(log n) O(log n) O(log n) O(log n) Sorted data, range queries
Skip List O(n) O(log n)† O(log n)† O(log n)† Concurrent ordered maps
Heap O(1) top O(n) O(log n) O(log n) Priority queues
Trie O(k) O(k) O(k) O(k) Prefix matching
B-Tree O(log n) O(log n) O(log n) O(log n) Disk-backed storage
Union-Find N/A α(n) α(n) N/A Connectivity / grouping
Segment / Fenwick N/A O(log n) O(log n) O(log n) Range queries on changing data
Bloom Filter N/A O(k) O(k) N/A‡ Approximate membership, tiny space

*If you already hold a reference to the node. †Expected (randomized). ‡Not supported in the basic variant.

How to actually choose. Default to a dynamic array — its cache behavior beats its asymptotics. Reach for a hash table the moment you key by something. Move to a balanced tree or skip list only when you need ordered iteration or range queries. Pull out the specialized structures (trie, segment tree, union-find, Bloom filter) when a problem's shape exactly matches what they make cheap. The trap at every level is reaching for the clever structure before the simple one has actually failed.


Ch. 22

Algorithms: Solving Problems

An algorithm is a precise recipe for solving a problem. But the interesting question isn't just whether it works — it's how efficiently it scales. This chapter covers the tools for measuring algorithms, the essential algorithms every engineer encounters, and the four fundamental problem-solving paradigms.

Big O Notation

Big O describes how runtime or memory grows as input size n grows, ignoring constant factors and lower-order terms. It answers: if I double the input, how much slower does this get?

Phone book analogy:

  • O(1): You have the page memorized — time is constant regardless of book size
  • O(log n): Open to the middle, pick a half, repeat — binary search
  • O(n): Read every entry — linear scan
  • O(n log n): Sort the book first, then search many times
  • O(n²): Compare every entry to every other entry — bubble sort
  • O(2ⁿ): Try every possible subset — brute-force combinations

Two common traps:

  • Space complexity follows the same notation. An algorithm can be O(n log n) time but O(1) extra space (quicksort, in-place) vs O(n) extra space (merge sort's merge buffer). Both time and space matter.
  • Amortized complexity spreads the cost of occasional expensive operations. Dynamic array append is O(1) amortized because resizes (O(n)) happen exponentially rarely — the cost averages out.

Reading Big-O Honestly

Asymptotics describe growth, not speed. Three caveats separate engineers who can apply Big-O from those who merely recite it:

  • Constants and lower-order terms are hidden — and they matter at real sizes. An O(n) algorithm with a huge constant can lose to an O(n log n) one until n is enormous. This is exactly why libraries use insertion sort (O(n²)) for small subarrays inside merge/quick sort.
  • The variable must be named. "O(n)" is meaningless until you say what n is. For graph algorithms it's usually V and E; for string algorithms, the pattern length and text length; for matrix work, often the dimension, not the element count.
  • Best/average/worst are different functions. Quicksort is O(n log n) average but O(n²) worst; hash lookup is O(1) average but O(n) worst. Which one you quote depends on whether you fear the adversary or the average case.

Amortized Analysis — Three Lenses

When a sequence of operations is mostly cheap with rare expensive spikes, amortized analysis proves the average over the sequence is low. Three standard techniques:

  • Aggregate: bound the total cost of n operations, then divide by n. For n dynamic-array appends, total copying work is n + n/2 + n/4 + … < 2n, so each append is O(1) amortized.
  • Accounting (banker's): charge each cheap operation a little extra and "save" the credit to pay for future expensive ones. Each append pays 3 units; 1 for itself, 2 banked to fund the eventual copy.
  • Potential: define a potential function on the data structure's state; an operation's amortized cost is its real cost plus the change in potential. This is the most general method, used to analyze splay trees and Fibonacci heaps.

The practical takeaway: a single O(n) operation does not make an algorithm O(n) per step if those operations are rare enough — but amortized guarantees can be defeated by an adversary who triggers the worst case repeatedly, which is why latency-sensitive systems sometimes prefer structures with good worst-case bounds over good amortized ones.

Recursion — Thinking in Terms of Itself

Recursion is the technique of solving a problem by solving smaller instances of the same problem. It is the backbone of three of the four paradigms below, and the natural way to walk trees and graphs. Every correct recursion needs two parts:

  • A base case that stops the descent (the smallest input, solved directly).
  • A recursive case that reduces the problem toward the base case and combines the results.
def factorial(n):
    if n <= 1:          # base case
        return 1
    return n * factorial(n - 1)   # recursive case

Each call gets its own stack frame holding its local variables and return address (see the call stack in Part 3). That has two consequences engineers must respect:

  • Stack depth is bounded. Deep or infinite recursion overflows the stack — Python caps it near 1000 frames by default. Algorithms that recurse O(n) deep on large n (e.g., walking a million-node linked list) should be rewritten iteratively or made tail-recursive in languages that optimize it.
  • Naive recursion can re-solve the same subproblem exponentially many times. That is precisely the problem dynamic programming exists to fix — recursion plus a cache (memoization) collapses the redundant work.

Analogy: Russian nesting dolls. To "process all dolls," you open the current one and apply the same procedure to the doll inside — until you reach the solid innermost doll (the base case) and start handing results back out.

Sorting Algorithms

Bubble Sort — O(n²): Repeatedly swap adjacent out-of-order elements. Simple to implement; useful only to illustrate the problem — never use in production code.

Selection Sort — O(n²): Repeatedly find the minimum of the unsorted portion and swap it into place. Does the fewest writes of any simple sort (O(n) swaps), which mattered when writing to memory was expensive (e.g., EEPROM).

Insertion Sort — O(n²) worst, O(n) on nearly-sorted: Build the sorted portion one element at a time, sliding each new element back to its place — exactly how most people sort a hand of cards.

def insertion_sort(arr):
    for i in range(1, len(arr)):
        key = arr[i]
        j = i - 1
        while j >= 0 and arr[j] > key:
            arr[j + 1] = arr[j]
            j -= 1
        arr[j + 1] = key

Insertion sort is the workhorse you don't notice: it is adaptive (nearly-sorted input runs in nearly O(n)), stable, in-place, and has tiny constants. Production sorts (Timsort, introsort) switch to it for small subarrays where its low overhead beats the recursion of the "better" algorithms.

Merge Sort — O(n log n):

Analogy: Sort a deck by splitting in half until you have single cards, then merge pairs back in order. Each merge is simple: compare the top card of two piles, take the smaller one. Repeat until merged.

Merge sort always achieves O(n log n) regardless of input order. It requires O(n) extra space for the merge buffer. It is stable — equal elements preserve their original order, which matters when sorting objects by one field while preserving order by another.

Quick Sort — O(n log n) average, O(n²) worst:

Pick a pivot element, partition into elements less than and greater than it, then recursively sort each partition.

[5, 3, 8, 1, 7]   pivot = 5
→ [3, 1] | 5 | [8, 7]
→ [1, 3, 5, 7, 8]

Despite the theoretically worse worst case, quicksort is often faster in practice:

  • Cache-friendly: operates in-place with no allocation
  • Smaller constant factors: fewer memory accesses per comparison
  • Worst case avoidable: randomized pivot selection makes O(n²) astronomically unlikely

What the industry actually uses: Python and Java use Timsort (a hybrid of merge sort and insertion sort that exploits natural runs in data). Java uses a dual-pivot quicksort for primitive arrays. The lesson: O(n log n) is the floor, but constant factors and real-world data distribution determine what wins in practice.

Counting Sort / Radix Sort — O(n + k): These bypass the comparison-based lower bound by exploiting integer structure. Counting sort works when your integers fall in a small range [0, k]; radix sort extends this to large integers by sorting digit by digit.

Heap Sort — O(n log n), in-place: Build a max-heap from the array (O(n)), then repeatedly swap the root to the end and sift down (O(log n) each). Unlike quicksort it has a guaranteed O(n log n) worst case and uses O(1) extra space — but its scattered memory access makes it cache-unfriendly, so it's usually slower in practice than quicksort. Its guaranteed bound makes it the fallback inside introsort (C++ std::sort): start with quicksort, switch to heap sort if recursion goes too deep, defeating the O(n²) adversary.

Quickselect — O(n) average: A cousin of quicksort for when you want the k-th smallest element (or the median) without fully sorting. Partition around a pivot, then recurse into only the side containing position k. Discarding half the work each step gives linear average time; the median-of-medians pivot rule guarantees O(n) even in the worst case. This is how numpy.partition and "top-k" queries work.

Core insight: No comparison-based sort can beat O(n log n). Information theory proves it: distinguishing n! orderings requires at least log₂(n!) ≈ n log n comparisons. But if your data has exploitable structure (integers, fixed-length strings), you can beat it.

Sorting at a Glance

Algorithm Best Average Worst Space Stable Notes
Insertion O(n) O(n²) O(n²) O(1) ✓ Great on small/nearly-sorted
Merge O(n log n) O(n log n) O(n log n) O(n) ✓ Predictable; external sorting
Quick O(n log n) O(n log n) O(n²) O(log n) ✗ Fastest in practice
Heap O(n log n) O(n log n) O(n log n) O(1) ✗ Worst-case guarantee
Counting/Radix O(n+k) O(n+k) O(n+k) O(n+k) ✓ Integers/fixed-length keys only

Stability — preserving the relative order of equal keys — is the property that lets you sort by one field and then another to get a multi-key sort. It's why Python's sorted is stable by contract.

Searching Algorithms

Linear Search — O(n): Scan every element. Works on any data; no preprocessing required.

Binary Search — O(log n): Requires sorted data. Check the middle element and eliminate half the search space each step.

def binary_search(arr, target):
    low, high = 0, len(arr) - 1
    while low <= high:
        mid = (low + high) // 2
        if arr[mid] == target:
            return mid
        elif arr[mid] < target:
            low = mid + 1
        else:
            high = mid - 1
    return -1

Analogy: Binary search on a billion items takes only ~30 steps. Each comparison eliminates exactly half of what remains — the definition of logarithmic growth.

Binary search's power extends beyond sorted arrays. The same "eliminate half the space each step" logic applies to searching optimization spaces — if you can evaluate whether the answer is too high or too low, binary search applies.

Binary search on the answer. Whenever a problem has a monotonic predicate — "is a solution of size x feasible?" where feasibility, once true, stays true — you can binary-search the answer itself rather than an array. "What is the minimum ship capacity to deliver all packages in D days?" or "what is the smallest largest-bucket when splitting an array into k parts?" are searched in O(log(range) × cost-of-check). Recognizing this disguised binary search is a hallmark of an experienced problem solver.

The Two-Pointer and Sliding-Window Patterns

These aren't named algorithms so much as patterns that turn an obvious O(n²) brute force into an O(n) pass — the most common optimization you'll apply in day-to-day coding.

Two pointers: maintain two indices that move toward each other or in the same direction, exploiting order. On a sorted array, finding a pair summing to a target is O(n): start one pointer at each end, and move inward based on whether the sum is too big or too small — no nested loop.

def two_sum_sorted(arr, target):
    lo, hi = 0, len(arr) - 1
    while lo < hi:
        s = arr[lo] + arr[hi]
        if s == target:
            return (lo, hi)
        elif s < target:
            lo += 1      # need a bigger sum
        else:
            hi -= 1      # need a smaller sum
    return None

Sliding window: maintain a contiguous range [left, right] and slide it across the data, expanding to include new elements and shrinking to maintain a constraint. Because each element enters and leaves the window at most once, the whole scan is O(n) even though it conceptually examines many subarrays. This is the tool for "longest substring without repeating characters," "smallest subarray with sum ≥ S," and most fixed/variable-window problems. It pairs naturally with the deque and hash map from the previous chapter.

Graph Algorithms

BFS — Breadth-First Search

Analogy: A ripple expanding on a pond — it spreads outward uniformly, one ring at a time, reaching close points before distant ones.

BFS visits all nodes at distance 1, then all at distance 2, and so on. It uses a queue to track the frontier. Key property: in an unweighted graph, BFS finds the shortest path (by edge count) to every reachable node.

Used for: shortest path in unweighted graphs, "degrees of separation" in social networks, level-order tree traversal, web crawlers that explore by link depth.

DFS — Depth-First Search

Analogy: Exploring a maze by always choosing to go deeper. Only when you hit a dead end do you backtrack and try another branch.

DFS goes as deep as possible along each branch before backtracking. It uses a stack (or recursion, which uses the call stack implicitly).

Used for: cycle detection, topological sort, maze solving, finding connected components, strongly connected components (Tarjan's algorithm), tree serialization.

Dijkstra's Algorithm — Shortest Path in Weighted Graphs

Dijkstra's finds the shortest path from a source node to all other nodes in a graph with non-negative edge weights. The idea: always expand the closest unvisited node, greedily committing to its shortest path.

It uses a min-heap (priority queue) to efficiently find the current nearest node. Time complexity: O((V + E) log V).

Limitation: Dijkstra's fails with negative-weight edges. Use Bellman-Ford (O(VE)) instead — it handles negative weights and also detects negative cycles, which have no well-defined shortest path.

Other essential graph algorithms:

  • A* Search: Dijkstra + a heuristic estimating remaining distance. Explores fewer nodes by biasing toward the goal. Used in GPS routing and game pathfinding.
  • Floyd-Warshall: All-pairs shortest paths in O(V³). When you need distances between every pair of nodes.
  • Kruskal's / Prim's: Minimum spanning tree — the cheapest way to connect all nodes with no cycles.
  • Topological Sort: Order DAG nodes so every directed edge goes "forward." Essential for build systems (compile A before B), course prerequisites, and task scheduling with dependencies.
  • Max-Flow / Min-Cut: Find the maximum flow through a network of capacitated edges (Ford-Fulkerson, Dinic's). The max-flow min-cut theorem ties it to the cheapest set of edges whose removal disconnects source from sink — the basis for bipartite matching, image segmentation, and network reliability.
  • Union-Find (previous chapter) is itself a graph-connectivity engine: it powers Kruskal's MST and answers "are these two nodes connected?" under a stream of edge additions.

String Algorithms

Searching and comparing text is so common that it has its own family of algorithms, all aimed at beating the naive O(n·m) "try the pattern at every position" approach.

  • KMP (Knuth-Morris-Pratt) — O(n + m): Precompute, for the pattern, how far you can shift after a mismatch without rechecking characters you've already matched. The key insight: a partial match already tells you about the text, so you never move the text pointer backward.
  • Rabin-Karp — O(n + m) average: Compare a rolling hash of the pattern against a rolling hash of each text window; only verify character-by-character on a hash collision. A rolling hash updates in O(1) as the window slides — the same idea as a sliding window. It shines for multi-pattern search and plagiarism/diff detection.
  • Z-algorithm / Boyer-Moore: Boyer-Moore scans the pattern right-to-left and can skip ahead by whole pattern lengths, making it sublinear in practice — it's what grep is built on.
  • Aho-Corasick — O(n + total pattern length + matches): Search for thousands of patterns at once by building a trie of the patterns with failure links (KMP generalized to many patterns). This is how intrusion-detection systems and spam filters scan traffic against huge keyword lists in one pass.

The recurring theme: preprocess one input so you never re-examine the other. Combined with suffix arrays/trees from the previous chapter, these cover most of practical text processing.

Bit Manipulation

Operating directly on the bits of an integer turns certain problems into single, branch-free CPU instructions — invaluable in systems code, graphics, cryptography, and performance-critical inner loops.

x & 1            # is x odd?
x << 1           # multiply by 2;   x >> 1 → divide by 2
x & (x - 1)      # clear the lowest set bit
x & -x           # isolate the lowest set bit (used in Fenwick trees)
x ^ y            # differing bits; x ^ x == 0
bin(x).count("1")  # population count (number of set bits)

A few classic tricks worth knowing: XOR finds the single unpaired element in a list where everything else is duplicated (all pairs cancel to 0); a bitmask represents a subset of up to 64 items in one integer, enabling bitmask dynamic programming over subsets (e.g., the Traveling Salesman DP in O(2ⁿ · n)); and x & (x - 1) == 0 tests for a power of two. These are micro-optimizations — reach for them when a profiler points you there or when the problem is inherently about bits (flags, sets, hardware registers).

Number Theory and Math

A handful of mathematical algorithms recur constantly in cryptography, hashing, and competitive programming:

  • Euclid's GCD — O(log n): gcd(a, b) = gcd(b, a mod b). The basis of fraction reduction and the extended version computes modular inverses — the engine inside RSA key math.
  • Sieve of Eratosthenes — O(n log log n): Find all primes up to n by repeatedly crossing out multiples. The standard way to precompute primes for factorization.
  • Fast (binary) exponentiation — O(log n): Compute aⁿ by squaring and combining, halving the exponent each step. Its modular form, pow(a, n, m) in Python, is the workhorse of public-key cryptography, where you exponentiate 2048-bit numbers.
  • Modular arithmetic: "clock arithmetic" that keeps numbers bounded; the foundation of hashing, checksums, and cryptographic schemes.

Why a working engineer should care: these aren't academic curiosities — every TLS handshake that secures this page runs modular exponentiation and GCD-based key math thousands of times. Knowing they're O(log n), not O(n), is why HTTPS is fast enough to be invisible.

Complexity Classes: P, NP, and the Limits of Algorithms

Everything so far has been about finding efficient algorithms. The deepest question in computer science is whether efficient algorithms exist at all for certain problems. That's the domain of complexity classes — and it's what separates "I haven't found a fast algorithm" from "no fast algorithm can exist."

  • P — problems solvable in polynomial time (O(nᵏ)). These are the "tractable" problems: sorting, shortest paths, matching. Everything in the rest of this part lives here.
  • NP — problems whose proposed solutions can be verified in polynomial time, even if finding one might be hard. Given a filled-in Sudoku, you can check it instantly; finding the solution is the hard part. (Every P problem is trivially in NP.)
  • NP-complete — the hardest problems in NP, and all equivalent: a polynomial-time algorithm for any one of them would solve all of NP. Boolean satisfiability (SAT), the Traveling Salesman decision problem, graph coloring, and the knapsack decision problem are all NP-complete.
  • NP-hard — at least as hard as NP-complete problems, but not necessarily in NP themselves (their solutions may not even be checkable in polynomial time). Optimization versions and the halting problem fall here.

The P = NP? question — is every problem whose answer is easy to check also easy to solve? — is the most famous open problem in the field, with a $1M Millennium Prize attached. Almost everyone believes P ≠ NP (the diagram assumes it), but no one has proven it. The stakes are enormous: if P = NP, most cryptography collapses overnight, since its security rests on certain problems being hard to solve but easy to verify.

Why this matters in practice — recognizing intractability is a senior skill. When the problem you're handed is NP-hard (and a surprising number of real scheduling, routing, packing, and layout problems are), the right move is not to search harder for an exact polynomial algorithm that cannot exist. Instead you change the goal:

  • Approximation algorithms that provably get within a factor of optimal (e.g., a 2-approximation for vertex cover).
  • Heuristics and metaheuristics — greedy, local search, simulated annealing, genetic algorithms — that find good-enough solutions without guarantees.
  • Exact methods that prune hard — branch-and-bound, integer-linear-programming solvers, SAT/SMT solvers — which routinely crack large NP-hard instances despite the exponential worst case.
  • Restricting the input — many NP-hard problems become easy on trees, on small parameters (fixed-parameter tractability), or under realistic constraints.

The principal-engineer takeaway: half of applied algorithm design is recognizing which box your problem falls in. Spending a week optimizing an exact solver for an NP-hard problem at scale is a failure of diagnosis; reaching for a clean approximation is the senior move. Conversely, reaching for a heuristic when a polynomial exact algorithm exists is leaving correctness on the table.

Algorithmic Paradigms

Recognizing which of these four strategies applies to a problem is often the hard part.

1. Divide and Conquer

Split the problem into independent subproblems, solve each recursively, combine the results. Works when subproblems don't share work.

Examples: Merge sort, quick sort, binary search, Strassen's matrix multiplication (O(n^2.81) vs the naive O(n³)).

Complexity analysis: The Master Theorem gives closed-form solutions for recurrences of the form T(n) = aT(n/b) + O(n^c):

  • If a > b^c: T(n) = O(n^log_b(a)) — subproblem work dominates
  • If a = b^c: T(n) = O(n^c log n) — work balanced across levels
  • If a < b^c: T(n) = O(n^c) — combining work dominates

Analogy: Eating an elephant one bite at a time. The bites are identical in structure, just smaller.

2. Dynamic Programming

Like divide and conquer, but for problems where subproblems overlap — the same sub-computation would be needed multiple times. Store (memoize) results so each is computed exactly once.

Two implementation styles:

  • Top-down (memoization): Recursive with a cache. Compute only the subproblems actually needed. More natural to write; overhead from recursion.
  • Bottom-up (tabulation): Iterative, fill a table from smallest subproblems up. No recursion overhead; often faster. Requires knowing which subproblems you'll need in advance.

DP works when the problem has optimal substructure (the optimal solution contains optimal solutions to its subproblems) and overlapping subproblems (the same subproblems recur). If subproblems don't overlap, divide-and-conquer is sufficient.

The transformation is mechanical once you see it — Fibonacci goes from exponential to linear by caching:

# Naive recursion: O(2ⁿ) — recomputes the same values exponentially often
def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)

# Top-down memoization: O(n) — each subproblem solved once
from functools import lru_cache
@lru_cache(maxsize=None)
def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)

# Bottom-up tabulation: O(n) time, O(1) space — no recursion at all
def fib(n):
    a, b = 0, 1
    for _ in range(n):
        a, b = b, a + b
    return a

The progression — recursion → memoized recursion → tabulation → space-optimized tabulation — is the same refactoring you apply to nearly every DP problem.

Classic DP problems:

  • Knapsack: maximize value under a weight constraint
  • Longest Common Subsequence: used in DNA alignment, diff tools (git diff)
  • Edit Distance (Levenshtein): minimum edits to transform one string into another — spell checkers, fuzzy search
  • Coin Change: fewest coins to make an amount
  • Matrix Chain Multiplication: optimal order to multiply a sequence of matrices

Analogy: Showing your work in math class. Instead of recalculating 7 × 8 every time you need it, you write it down once and look it up. The cache is your scratch paper.

3. Greedy

Make the locally optimal choice at each step and never backtrack. Greedy is simpler than DP but only provably correct for problems with the greedy choice property — a locally optimal decision always extends to a globally optimal solution.

Where greedy works: Dijkstra's algorithm (always expand the nearest node), Huffman coding (build optimal prefix codes bottom-up), activity selection (maximum non-overlapping intervals), minimum spanning trees (always add the cheapest edge that doesn't create a cycle).

Where greedy fails: The 0/1 Knapsack problem. With items of weight 6, 5, and 3 and capacity 8, a greedy approach (take the heaviest item first) picks weight 6 then can't fit anything else (value: 6). But taking weights 5+3 (value: 8) is optimal.

Analogy: Always taking the biggest bill from a pile. Works for standard currency denominations (greedy change-making is optimal for most currencies). Doesn't work for arbitrary coin systems.

4. Backtracking

Systematically try all possibilities, pruning branches the moment they can't lead to a valid solution.

Examples: N-Queens (place N non-attacking queens on an N×N board), Sudoku solvers, constraint satisfaction problems, generating all valid permutations, subset sum.

Backtracking is essentially DFS through the space of partial solutions. The insight is in the pruning: in practice, most branches are cut very early, making backtracking far faster than brute force even though the worst case is exponential.

Analogy: Solving a maze by trying paths but immediately turning back the moment you hit a wall. You systematically eliminate dead ends without re-exploring them, and you're guaranteed to find a solution if one exists.


Putting it together: Real-world algorithm design rarely fits neatly into one category. Dijkstra's is greedy but uses a heap (a data structure). Efficient DP implementations use hash maps for memoization. The paradigms are lenses for thinking about problems — the goal is recognizing which lens applies and combining them as needed.

Ch. 23

Python in Practice: From Concept to Code

The previous two chapters were about ideas: what a hash table is, why a heap gives you the smallest element in logarithmic time, when divide-and-conquer beats brute force. This chapter is about how those ideas turn into running code. The language we use is Python — not because it is the fastest, but because it gets out of the way. Its standard library is, in effect, the catalog from Chapter 21 already built, tested, and tuned in C. Knowing which batteries are included — and the handful of idioms that quietly change a program's complexity class — is what separates code that merely works from code that works at scale.

Why Python for this. Python is the closest thing the field has to executable pseudocode. A binary search reads almost exactly like its description; a graph is a dictionary of lists. That transparency is the point: the structure of the solution should be visible through the syntax, not buried under boilerplate. Everything here applies conceptually to any language — only the spelling changes.

The Built-in Containers, and What They Really Are

Python's four workhorse containers are direct embodiments of structures from Chapter 21. Recognizing the structure underneath the convenient syntax is what lets you predict performance.

Container Underlying structure Lookup Notes
list Dynamic array O(n) by value, O(1) by index Amortized O(1) append/pop; O(n) insert(0,..)/pop(0)
dict Hash table O(1) average Insertion-ordered since Python 3.7
set Hash table (keys only) O(1) average Membership, dedup, set algebra
tuple Immutable array O(1) by index Hashable → usable as a dict key or set member

The single most consequential thing to internalize is the difference between the two O(1)-vs-O(n) membership tests:

# 'x in a_list' scans every element — O(n)
if user_id in banned_list:        # slow when the list is large
    ...

# 'x in a_set' hashes once and probes — O(1) average
if user_id in banned_set:         # the version that survives scale
    ...

A loop that checks membership against a list is a quadratic algorithm wearing a linear disguise — one of the most common causes of code that is fast in testing and catastrophic in production. Converting the lookup target to a set once, up front, collapses the inner cost to constant time.

Analogy: A list is a numbered row of lockers — to find whether a particular item is inside, you open every locker. A set is the index card system at a library: one hash of the title tells you exactly which drawer to check. Same items, completely different cost to ask "is this here?"

Two container traps deserve naming because they bite nearly everyone once:

# Building a 2D grid. The right way:
grid = [[0] * cols for _ in range(rows)]   # each row is its own list
# The wrong way — every row is the SAME list object:
grid = [[0] * cols] * rows                 # writing grid[0][0] changes all rows

# A mutable default argument is created ONCE, at definition time, and shared:
def add(item, bucket=[]):     # bug: bucket persists across calls
    bucket.append(item); return bucket
def add(item, bucket=None):   # the safe idiom
    if bucket is None: bucket = []
    bucket.append(item); return bucket

The Standard Library Is the Data-Structure Catalog

Most of Chapter 21 ships in the box. Reaching for these instead of hand-rolling is not laziness — the library versions are correct, C-optimized, and instantly recognizable to any reader.

collections — the everyday upgrades. A Counter is a multiset and a frequency map in one; defaultdict removes the "check-then-initialize" dance; a deque is the real queue.

from collections import Counter, defaultdict, deque

Counter("mississippi").most_common(1)   # [('i', 4)] — frequency in one line

graph = defaultdict(list)                # no KeyError on first touch
graph[u].append(v)                       # the node auto-creates an empty list

queue = deque([start])                   # popleft() is O(1); list.pop(0) is O(n)
queue.append(x); queue.popleft()

The deque distinction is not academic. A breadth-first search built on a plain list with pop(0) is O(n²) because every dequeue shifts the entire list left; the same code on a deque is O(n). The queue from Chapter 21 only behaves like a queue when its backing structure supports O(1) removal from the front.

heapq — the priority queue. Python's heap is a min-heap living inside an ordinary list. It is the engine behind Dijkstra's algorithm and every "top-K" problem.

import heapq
heap = [3, 1, 4, 1, 5]
heapq.heapify(heap)          # O(n)
heapq.heappush(heap, 2)      # O(log n)
heapq.heappop(heap)          # O(log n) → 1, the minimum
heap[0]                      # O(1) peek at the minimum — don't pop to look

# Need a max-heap? Negate on the way in and out:
max_heap = [-x for x in data]; heapq.heapify(max_heap)
largest = -heapq.heappop(max_heap)

# Keyed priority: push tuples; ties break on a counter so payloads never compare
heapq.heappush(pq, (priority, next(counter), payload))

Analogy: A heap is a hospital triage desk. It does not keep a fully sorted list of every patient — that would be wasted effort. It guarantees exactly one thing cheaply: the most urgent case is always at the front. Inserting a new patient or removing the current most-urgent one is quick; asking for a full ranking is not what it is for.

bisect — binary search you don't have to re-derive. Maintaining a sorted list and querying it is so common that the binary search of Chapter 22 is packaged directly.

import bisect
a = [1, 3, 4, 7, 10]
bisect.bisect_left(a, 4)     # 2 — first index where a[i] >= 4
bisect.insort(a, 5)          # insert 5, keeping the list sorted
# Count values in [lo, hi]: bisect_right(a, hi) - bisect_left(a, lo)

Idioms That Quietly Change the Complexity

A few Python habits are not stylistic preferences — they decide whether a routine is linear or quadratic.

# Building a string. '+=' in a loop is O(n²): each step copies the whole string.
out = "".join(pieces)            # O(n) — the only correct way to assemble strings

# Comprehensions over manual append loops: clearer and faster
squares = [x * x for x in nums]

# Generators are lazy — they compute one value at a time, holding nothing in memory
total = sum(x * x for x in range(10**8))   # no 100-million-element list is built

The generator point matters beyond memory: laziness lets any() and all() short-circuit, stopping at the first decisive element instead of scanning the whole sequence. The cost of a stream-processing pipeline can hinge entirely on whether its stages are lazy.

Turning the Paradigms into Templates

The four paradigms from Chapter 22 are powerful precisely because each collapses into a near-mechanical skeleton. Once you recognize the shape of a problem, the code almost writes itself — the thinking is in the recognition, not the typing.

Two pointers / sliding window turn many O(n²) scans into a single linear pass by maintaining a window instead of re-examining every pair:

def longest_unique(s):           # longest substring with no repeats — O(n)
    seen, left, best = {}, 0, 0
    for right, ch in enumerate(s):
        if ch in seen and seen[ch] >= left:
            left = seen[ch] + 1  # jump the window past the duplicate
        seen[ch] = right
        best = max(best, right - left + 1)
    return best

Graph traversal (Chapter 22's BFS/DFS) is the same template every time — only the data structure holding the frontier changes. A queue gives breadth-first (and shortest paths on unweighted graphs); a stack or the call stack gives depth-first:

def bfs(graph, start):
    seen, queue = {start}, deque([start])
    while queue:
        node = queue.popleft()           # swap for stack.pop() to get DFS
        for nb in graph[node]:
            if nb not in seen:
                seen.add(nb); queue.append(nb)

Dynamic programming top-down is, in Python, often a one-line decorator away — lru_cache is memoization. The transformation from exponential to linear that Chapter 22 described mechanically becomes:

from functools import lru_cache
@lru_cache(maxsize=None)         # adds the cache; the logic is unchanged
def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)

Backtracking is depth-first search over partial solutions, and its skeleton is always choose → recurse → undo:

def subsets(nums):
    out = []
    def backtrack(start, path):
        out.append(path[:])              # record the current partial solution
        for i in range(start, len(nums)):
            path.append(nums[i])         # choose
            backtrack(i + 1, path)       # recurse
            path.pop()                   # undo — restore state for the next branch
    backtrack(0, [])
    return out

Binary search on the answer generalizes Chapter 22's search: when a problem asks for the smallest (or largest) value satisfying a monotonic condition, binary-search the answer space rather than an array. "What is the slowest eating speed that still finishes in time?" and "what is the smallest ship capacity that delivers in N days?" are the same problem — find the threshold where a yes/no test flips.

Diagnosing a Problem: From Signal to Tool

Most of the difficulty in applying this toolkit is recognition — matching the texture of a problem to the structure that solves it. These pairings come up again and again:

When you see… Reach for…
"Top K", "K largest / smallest / closest" Heap (heapq)
"Subarray / substring sum equals K" Prefix sums + hash map
"Longest / shortest window satisfying…" Sliding window
"All permutations / subsets / combinations" Backtracking
"Overlapping subproblems", "in how many ways" Dynamic programming
"Next greater / smaller element" Monotonic stack
"Connected regions", "islands", "groups" BFS/DFS or Union-Find
"Shortest path", unweighted vs. weighted BFS vs. Dijkstra
"Ordering with prerequisites" Topological sort
"Prefix / autocomplete" Trie
Sorted input + a target Binary search or two pointers

The input size is itself a hint about the target complexity — the upper bound on n quietly tells you which paradigms are even admissible:

Input size Feasible complexity Typical approach
n ≤ 12 O(n!) brute-force permutations, backtracking
n ≤ 25 O(2ⁿ) subset enumeration, bitmask DP
n ≤ 500 O(n³) interval DP, Floyd–Warshall
n ≤ 5,000 O(n²) nested loops, 2D DP
n ≤ 10⁶ O(n log n) sorting, heaps, BFS/DFS
n ≤ 10⁸ O(n) single pass, prefix sums, hashing
n > 10⁸ O(log n) or O(1) binary search, closed-form math

The senior takeaway: the leap from competent to expert here is not memorizing more algorithms — it is faster diagnosis. Reading "find the K most frequent" and immediately seeing a Counter feeding a heap, or "smallest value that works" and reaching for binary search on the answer, is the skill. The library and the templates above are the vocabulary; recognizing which sentence to write is the fluency.

The Complete Reference

The narrative above explains why each tool exists and when to reach for it. What follows is the lab manual: a complete, runnable Python reference covering every structure, algorithm, and idiom in this chapter, organized for scanning. Nothing here is decorative — each block is code you can paste and run.

"""
╔══════════════════════════════════════════════════════════════════════════════╗
║           PYTHON REFERENCE — COMPLETE REFERENCE GUIDE                       ║
║           Data structures, algorithms, and the idioms that tie them        ║
╚══════════════════════════════════════════════════════════════════════════════╝

TABLE OF CONTENTS
─────────────────
  1.  Python Fundamentals & Syntax
  2.  Data Types & Type System
  3.  Strings & String Methods
  4.  Lists, Tuples, Sets, Dicts — Deep Dive
  5.  Comprehensions & Generators
  6.  Functions — Args, Kwargs, Closures, Decorators
  7.  Object-Oriented Programming (OOP)
  8.  Iterators & Itertools
  9.  Error Handling & Context Managers
  10. File I/O
  11. Sorting, Searching, Custom Comparators
  12. Bit Manipulation
  13. Collections Module
  14. Heapq & Priority Queues
  15. Functional Programming (map, filter, reduce, lambda)
  16. Recursion & Memoization
  17. Dynamic Programming Patterns
  18. Graph Algorithms
  19. Tree Algorithms
  20. Linked List Patterns
  21. Sliding Window & Two Pointers
  22. Binary Search Patterns
  23. Stack & Queue Patterns
  24. Backtracking
  25. Greedy Algorithms
  26. Complexity Cheat Sheet
  27. Python-Specific Tricks
  28. Common Patterns Summary
"""

1. Python Fundamentals & Syntax

# --- Variable assignment & multiple assignment ---
x = 10
a, b, c = 1, 2, 3
a, b = b, a                         # Swap without temp variable

# --- Unpacking ---
first, *rest = [1, 2, 3, 4, 5]     # first=1, rest=[2,3,4,5]
*init, last = [1, 2, 3, 4, 5]      # init=[1,2,3,4], last=5
first, *mid, last = [1, 2, 3, 4]   # first=1, mid=[2,3], last=4

# --- Truthiness / Falsiness ---
# Falsy: False, None, 0, 0.0, "", [], {}, set(), ()
# Truthy: everything else

# --- Walrus operator := (Python 3.8+) — assign & evaluate in one expression ---
import re
if m := re.search(r'\d+', "abc123"):
    print(m.group())                # "123"

# Useful in while loops:
# while chunk := file.read(8192):
#     process(chunk)

# --- Ternary expression ---
val = "even" if x % 2 == 0 else "odd"

# --- Chained comparisons ---
result = 1 < x < 100               # Pythonic, evaluates correctly

# --- None checks — always use `is` not `==` ---
if x is None:
    pass
if x is not None:
    pass

# --- f-strings (Python 3.6+) ---
name = "Ada"
print(f"Hello {name}, x={x:.2f}")  # Format floats
print(f"{x!r}")                     # repr
print(f"{1_000_000:,}")             # 1,000,000
print(f"{'left':<10}|{'right':>10}")# Alignment

# --- Pass, continue, break, else on loops ---
for i in range(10):
    if i == 3: continue
    if i == 7: break
else:
    print("Loop completed without break")  # Only runs if no break hit

# --- Global & nonlocal ---
counter = 0
def increment():
    global counter
    counter += 1

def make_counter():
    count = 0
    def inc():
        nonlocal count
        count += 1
        return count
    return inc

2. Data Types & Type System

# --- Numeric types ---
i = 42          # int (arbitrary precision)
f = 3.14        # float (64-bit double)
c = 3 + 4j      # complex
from fractions import Fraction
frac = Fraction(1, 3)               # Exact rational arithmetic

# --- Integer tricks ---
print(10 // 3)          # 3  — floor division
print(-10 // 3)         # -4 — floor (rounds toward -inf, NOT toward zero!)
print(10 % 3)           # 1
print(-10 % 3)          # 2  — Python modulo always non-negative when divisor positive
print(divmod(10, 3))    # (3, 1) — quotient and remainder together
print(2 ** 10)          # 1024
print(abs(-5))          # 5
print(pow(2, 10, 1000)) # 24 — modular exponentiation, O(log n)

# --- Float gotchas ---
print(0.1 + 0.2 == 0.3)    # False!
import math
print(math.isclose(0.1 + 0.2, 0.3))  # True

# --- Type conversion ---
int("42"), int(3.9)         # 42, 3 (truncates, does NOT round)
float("3.14")
str(42)
bool(0), bool(""), bool([]) # False, False, False
list("abc")                 # ['a', 'b', 'c']
tuple([1, 2, 3])
set([1, 1, 2, 3])           # {1, 2, 3}

# --- Type checking ---
isinstance(42, int)             # True
isinstance(42, (int, float))    # True — checks multiple types
type(42) is int                 # True — exact type, no inheritance

# --- Constants ---
import sys
print(sys.maxsize)              # 9223372036854775807 (2^63 - 1 on 64-bit)
print(float('inf'))             # Infinity
print(float('-inf'))            # -Infinity
print(float('nan'))             # NaN
INF = float('inf')

3. Strings & String Methods

s = "Hello, World!"

# --- Basics ---
len(s)                          # 13
s[0], s[-1]                     # 'H', '!'
s[7:12]                         # 'World'
s[::-1]                         # Reverse: '!dlroW ,olleH'
s.lower(), s.upper()
s.strip(), s.lstrip(), s.rstrip()
s.replace("World", "Python")
s.split(",")                    # ['Hello', ' World!']
",".join(["a", "b", "c"])      # 'a,b,c'

# --- Search ---
s.find("World")                 # 7, returns -1 if not found
s.index("World")                # 7, raises ValueError if not found
s.count("l")                    # 3
s.startswith("Hello")           # True
s.endswith("!")                 # True
"World" in s                    # True

# --- Check content ---
"abc".isalpha()     # True
"123".isdigit()     # True
"abc123".isalnum()  # True
"   ".isspace()     # True

# --- Formatting ---
"Hello {}".format("World")
"{0} {1} {0}".format("ha", "ha!")
"{name}".format(name="Ada")

# --- Useful patterns ---
# Count character frequencies
from collections import Counter
Counter("mississippi")          # Counter({'s': 4, 'i': 4, 'p': 2, 'm': 1})

# Anagram check
def is_anagram(s, t):
    return Counter(s) == Counter(t)

# Palindrome check
def is_palindrome(s):
    s = s.lower()
    return s == s[::-1]

# --- String to list of chars and back ---
chars = list("hello")           # ['h', 'e', 'l', 'l', 'o']
"".join(chars)                  # 'hello'

# --- ord and chr ---
ord('a')    # 97
chr(97)     # 'a'
ord('A')    # 65
# Lowercase letter index: ord(c) - ord('a') → 0..25
# Check if lowercase: 'a' <= c <= 'z'

# --- String multiplication ---
"ab" * 3    # 'ababab'
"-" * 20    # '--------------------'

# --- Partition ---
"hello=world".partition("=")    # ('hello', '=', 'world')

# --- Encoding ---
"hello".encode("utf-8")         # b'hello'
b"hello".decode("utf-8")        # 'hello'

4. Lists, Tuples, Sets, Dicts — Deep Dive

# ===== LISTS =====
# Ordered, mutable, allows duplicates. O(1) append/pop, O(n) insert/delete.

lst = [3, 1, 4, 1, 5, 9, 2, 6]

lst.append(7)           # Add to end: O(1)
lst.pop()               # Remove & return last: O(1)
lst.pop(0)              # Remove & return index 0: O(n)
lst.insert(0, 99)       # Insert at index: O(n)
lst.remove(1)           # Remove first occurrence of value: O(n)
lst.index(5)            # Find index of value: O(n)
lst.count(1)            # Count occurrences: O(n)
lst.reverse()           # In-place reverse: O(n)
lst.sort()              # In-place sort: O(n log n)
lst.extend([10, 11])    # Append all from iterable: O(k)
lst.copy()              # Shallow copy
lst.clear()             # Remove all elements

# Slicing: lst[start:stop:step]  — always returns a NEW list
lst = [0, 1, 2, 3, 4, 5]
lst[1:4]                # [1, 2, 3]
lst[::2]                # [0, 2, 4]
lst[::-1]               # [5, 4, 3, 2, 1, 0]
lst[1:4] = [10, 20]     # Replace slice in-place

# --- 2D list initialization (CORRECT way) ---
rows, cols = 3, 4
grid = [[0] * cols for _ in range(rows)]  # Each row is independent
# grid = [[0] * cols] * rows              # WRONG! All rows share same reference

# --- Flatten a nested list ---
nested = [[1, 2], [3, 4], [5]]
flat = [x for sub in nested for x in sub]   # [1, 2, 3, 4, 5]
import itertools
flat2 = list(itertools.chain.from_iterable(nested))

# ===== TUPLES =====
# Ordered, IMMUTABLE, allows duplicates. Hashable (can be dict key / set member).

t = (1, 2, 3)
t = 1, 2, 3             # Parentheses optional
single = (1,)           # Single-element tuple — comma required!
x, y, z = t             # Unpacking

# Named tuples — lightweight struct
from collections import namedtuple
Point = namedtuple('Point', ['x', 'y'])
p = Point(3, 4)
p.x, p.y                # Access by name
p[0], p[1]              # Still indexable

# ===== SETS =====
# Unordered, mutable, NO duplicates. O(1) average add/remove/lookup.

s = {1, 2, 3, 4}
s.add(5)
s.remove(5)             # Raises KeyError if not found
s.discard(99)           # No error if not found
s.pop()                 # Remove & return arbitrary element

# Set operations
a, b = {1, 2, 3}, {2, 3, 4}
a | b                   # Union: {1, 2, 3, 4}
a & b                   # Intersection: {2, 3}
a - b                   # Difference: {1}
a ^ b                   # Symmetric difference: {1, 4}
a.issubset(b)
a.issuperset(b)
a.isdisjoint(b)

# frozenset — immutable set, hashable
fs = frozenset([1, 2, 3])

# ===== DICTS =====
# Key-value pairs, ordered by insertion (Python 3.7+). O(1) average get/set/delete.

d = {"a": 1, "b": 2, "c": 3}
d["a"]                  # 1, raises KeyError if missing
d.get("z")              # None (default)
d.get("z", 0)           # 0 (custom default)
d["d"] = 4              # Insert/update
del d["a"]              # Remove key
d.pop("b")              # Remove & return value
d.pop("z", None)        # Safe pop

d.keys()                # dict_keys view
d.values()              # dict_values view
d.items()               # dict_items view (key, value) pairs
"a" in d                # True — O(1) key lookup
d.update({"e": 5})      # Merge another dict in
{**d, "f": 6}           # Merge with unpacking (Python 3.5+)
d | {"f": 6}            # Merge operator (Python 3.9+)

# setdefault — get value, set it if missing
d.setdefault("new_key", []).append(1)

# defaultdict
from collections import defaultdict
dd = defaultdict(list)
dd["fruits"].append("apple")    # No KeyError — auto-creates empty list

dd2 = defaultdict(int)
dd2["count"] += 1               # Auto-starts at 0

# --- Dict comprehension ---
squares = {x: x**2 for x in range(6)}

# --- Inverting a dict ---
inv = {v: k for k, v in d.items()}

# --- Sorting dict by value ---
sorted(d.items(), key=lambda x: x[1])

5. Comprehensions & Generators

# --- List comprehension ---
squares = [x**2 for x in range(10)]
evens = [x for x in range(20) if x % 2 == 0]
matrix = [[i * j for j in range(5)] for i in range(5)]

# --- Dict comprehension ---
word_len = {w: len(w) for w in ["hello", "world", "python"]}

# --- Set comprehension ---
unique_lens = {len(w) for w in ["hello", "world", "python"]}

# --- Generator expression — lazy, memory-efficient ---
gen = (x**2 for x in range(10))    # Does NOT compute yet
sum(x**2 for x in range(10))       # No brackets needed inside function call

# --- Generator function ---
def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

fib = fibonacci()
[next(fib) for _ in range(8)]      # [0, 1, 1, 2, 3, 5, 8, 13]

# yield from — delegate to sub-generator
def chain(*iterables):
    for it in iterables:
        yield from it

# --- any() / all() with generators ---
any(x > 5 for x in [1, 2, 3, 6])  # True — short-circuits
all(x > 0 for x in [1, 2, 3, 4])  # True

6. Functions — Args, Kwargs, Closures, Decorators

# --- *args and **kwargs ---
def func(a, b, *args, **kwargs):
    print(a, b)             # Positional
    print(args)             # Tuple of extra positionals
    print(kwargs)           # Dict of extra keywords

func(1, 2, 3, 4, x=5, y=6)
# 1 2
# (3, 4)
# {'x': 5, 'y': 6}

# --- Keyword-only args (after *) ---
def func2(a, b, *, keyword_only):
    pass

# --- Positional-only args (before /) ---
def func3(pos_only, /, normal, *, kw_only):
    pass

# --- Default mutable argument gotcha ---
def append_to(elem, lst=[]):    # WRONG! lst is created ONCE and shared
    lst.append(elem)
    return lst

def append_to_safe(elem, lst=None):  # Correct pattern
    if lst is None:
        lst = []
    lst.append(elem)
    return lst

# --- Closures ---
def make_multiplier(n):
    def multiplier(x):
        return x * n   # n is captured from enclosing scope
    return multiplier

double = make_multiplier(2)
double(5)   # 10

# --- Decorators ---
import functools
import time

def timer(func):
    @functools.wraps(func)          # Preserves __name__, __doc__
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        end = time.perf_counter()
        print(f"{func.__name__} took {end - start:.4f}s")
        return result
    return wrapper

@timer
def slow_function():
    time.sleep(0.01)

# Decorator with arguments (factory pattern)
def retry(max_attempts=3):
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            for attempt in range(max_attempts):
                try:
                    return func(*args, **kwargs)
                except Exception as e:
                    if attempt == max_attempts - 1:
                        raise
            return None
        return wrapper
    return decorator

@retry(max_attempts=5)
def flaky_function():
    pass

# --- Memoization decorator ---
from functools import lru_cache, cache

@lru_cache(maxsize=None)    # Cache all results
def fib_cached(n):
    if n < 2: return n
    return fib_cached(n-1) + fib_cached(n-2)

@cache                      # Python 3.9+ shorthand for lru_cache(maxsize=None)
def fib_v2(n):
    if n < 2: return n
    return fib_v2(n-1) + fib_v2(n-2)

7. Object-Oriented Programming (OOP)

class Animal:
    species_count = 0           # Class variable — shared across all instances

    def __init__(self, name: str, age: int):
        self.name = name        # Instance variable
        self.age = age
        Animal.species_count += 1

    def __repr__(self):         # Unambiguous — for debugging, used by repr()
        return f"Animal(name={self.name!r}, age={self.age})"

    def __str__(self):          # Human-readable — used by str() and print()
        return f"{self.name} (age {self.age})"

    def __eq__(self, other):
        if not isinstance(other, Animal): return NotImplemented
        return self.name == other.name and self.age == other.age

    def __hash__(self):         # Required if __eq__ is defined (to use in sets/dicts)
        return hash((self.name, self.age))

    def __lt__(self, other):    # For sorting / comparisons
        return self.age < other.age

    def __len__(self):
        return self.age

    def speak(self):            # Instance method
        raise NotImplementedError

    @classmethod
    def create_puppy(cls, name):  # Class method — receives class, not instance
        return cls(name, 0)

    @staticmethod
    def is_valid_age(age):      # Static method — no cls or self
        return 0 <= age <= 150

    @property
    def description(self):      # Property — accessed like attribute
        return f"{self.name}, {self.age} years old"

    @description.setter
    def description(self, value):
        # Custom setter logic
        pass


class Dog(Animal):
    def __init__(self, name, age, breed):
        super().__init__(name, age)     # Call parent __init__
        self.breed = breed

    def speak(self):
        return f"{self.name} says Woof!"

    def __repr__(self):
        return f"Dog(name={self.name!r}, age={self.age}, breed={self.breed!r})"


# --- Abstract base classes ---
from abc import ABC, abstractmethod

class Shape(ABC):
    @abstractmethod
    def area(self) -> float:
        pass

    @abstractmethod
    def perimeter(self) -> float:
        pass

class Circle(Shape):
    def __init__(self, radius: float):
        self.radius = radius

    def area(self) -> float:
        return math.pi * self.radius ** 2

    def perimeter(self) -> float:
        return 2 * math.pi * self.radius


# --- Dataclasses (Python 3.7+) — auto-generates __init__, __repr__, __eq__ ---
from dataclasses import dataclass, field

@dataclass
class Point:
    x: float
    y: float
    z: float = 0.0                          # Default value

    def distance_to_origin(self):
        return (self.x**2 + self.y**2 + self.z**2) ** 0.5

@dataclass(order=True)                      # Also generates __lt__, __le__, etc.
class Employee:
    sort_index: int = field(init=False, repr=False)  # Not in __init__
    name: str
    salary: float

    def __post_init__(self):
        self.sort_index = self.salary       # Set derived fields here

# --- __slots__ — memory optimization, also prevents adding new attributes ---
class Efficient:
    __slots__ = ['x', 'y']
    def __init__(self, x, y):
        self.x = x
        self.y = y

# --- Dunder methods summary ---
# __init__       Constructor
# __del__        Destructor
# __repr__       repr(obj) — for developers
# __str__        str(obj) — for users
# __len__        len(obj)
# __getitem__    obj[key]
# __setitem__    obj[key] = value
# __delitem__    del obj[key]
# __contains__   key in obj
# __iter__       iter(obj)
# __next__       next(obj)
# __call__       obj()
# __enter__      with obj as x:
# __exit__       end of with block
# __eq__, __lt__ ==, <
# __add__        obj + other
# __mul__        obj * other
# __bool__       bool(obj)
# __hash__       hash(obj)

8. Iterators & Itertools

import itertools

# --- Core itertools ---
# count(start, step)       — infinite counter
# cycle(iterable)          — infinite cycle
# repeat(obj, n)           — repeat n times

list(itertools.accumulate([1, 2, 3, 4, 5]))         # [1, 3, 6, 10, 15] — cumulative sum
list(itertools.accumulate([1, 2, 3, 4, 5], max))    # [1, 2, 3, 4, 5] running max

list(itertools.chain([1, 2], [3, 4], [5]))          # [1, 2, 3, 4, 5]

list(itertools.combinations([1, 2, 3], 2))
# [(1,2), (1,3), (2,3)] — order doesn't matter, no repeats

list(itertools.combinations_with_replacement([1, 2], 2))
# [(1,1), (1,2), (2,2)]

list(itertools.permutations([1, 2, 3], 2))
# All ordered pairs

list(itertools.product([1, 2], [3, 4]))
# [(1,3), (1,4), (2,3), (2,4)] — cartesian product

list(itertools.product([0, 1], repeat=3))
# All 3-bit binary combos

list(itertools.islice(range(100), 5, 15, 2))        # [5, 7, 9, 11, 13]

list(itertools.groupby("AAABBBCCD"))
# [('A', iter), ('B', iter), ('C', iter), ('D', iter)]
# NOTE: Must be sorted first for meaningful grouping

list(itertools.starmap(pow, [(2, 3), (3, 2), (4, 2)]))  # [8, 9, 16]

list(itertools.takewhile(lambda x: x < 5, [1, 2, 6, 2, 1]))  # [1, 2]
list(itertools.dropwhile(lambda x: x < 5, [1, 2, 6, 2, 1]))  # [6, 2, 1]

list(itertools.zip_longest([1, 2, 3], [4, 5], fillvalue=0))
# [(1,4), (2,5), (3,0)]

# --- enumerate and zip ---
for idx, val in enumerate(["a", "b", "c"], start=1):
    print(idx, val)

for av, bv in zip([1, 2, 3], [4, 5, 6]):
    print(av, bv)

# Unzip
pairs = [(1, 'a'), (2, 'b'), (3, 'c')]
nums, letters = zip(*pairs)        # nums=(1,2,3), letters=('a','b','c')

9. Error Handling & Context Managers

# --- Exception hierarchy ---
# BaseException
#   └── Exception
#         ├── ValueError
#         ├── TypeError
#         ├── KeyError
#         ├── IndexError
#         ├── AttributeError
#         ├── NameError
#         ├── ZeroDivisionError
#         ├── OverflowError
#         ├── RuntimeError
#         │     └── RecursionError
#         ├── StopIteration
#         ├── OSError
#         │     ├── FileNotFoundError
#         │     └── PermissionError
#         └── ArithmeticError
#               ├── ZeroDivisionError
#               └── OverflowError

# --- try / except / else / finally ---
try:
    result = 10 / 0
except ZeroDivisionError:
    print("Divided by zero!")
except (TypeError, ValueError) as e:
    print(f"Type or value error: {e}")
except Exception as e:
    print(f"Unexpected: {e}")
    raise                           # Re-raise the exception
else:
    print("No exception occurred")  # Only runs if try succeeded
finally:
    print("Always runs")            # Cleanup — always executes

# --- Custom exceptions ---
class InsufficientFundsError(ValueError):
    def __init__(self, amount, balance):
        super().__init__(f"Cannot withdraw {amount}, balance is {balance}")
        self.amount = amount
        self.balance = balance

# --- Context managers ---
# Using class-based __enter__ / __exit__
class ManagedResource:
    def __enter__(self):
        print("Acquiring resource")
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        print("Releasing resource")
        return False    # False = don't suppress exceptions

# Using contextlib
from contextlib import contextmanager

@contextmanager
def managed_resource():
    print("Acquiring")
    try:
        yield "resource"    # Value bound to `as` variable
    finally:
        print("Releasing")

with managed_resource() as r:
    print(f"Using {r}")

10. File I/O

# --- Reading files ---
with open("file.txt", "r") as f:
    content = f.read()              # Entire file as string
    # OR
    lines = f.readlines()           # List of lines (includes \n)
    # OR
    for line in f:                  # Memory-efficient iteration
        line = line.strip()

# --- Writing files ---
with open("output.txt", "w") as f:  # "w" overwrites, "a" appends
    f.write("Hello\n")
    f.writelines(["line1\n", "line2\n"])

# --- JSON ---
import json
data = {"key": "value", "num": 42}
json_str = json.dumps(data, indent=2)
parsed = json.loads(json_str)

with open("data.json", "w") as f:
    json.dump(data, f)
with open("data.json", "r") as f:
    data = json.load(f)

# --- CSV ---
import csv
with open("data.csv", "r") as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(row)  # dict per row

# --- sys.stdin for competitive programming ---
import sys
input_data = sys.stdin.read().split()
# or: lines = sys.stdin.readlines()

11. Sorting, Searching, Custom Comparators

# --- sorted() vs list.sort() ---
lst = [3, 1, 4, 1, 5, 9, 2, 6]
sorted_lst = sorted(lst)                    # Returns new list
lst.sort()                                  # In-place, returns None
sorted(lst, reverse=True)                   # Descending
sorted(lst, key=lambda x: -x)              # Equivalent

# --- Key function ---
words = ["banana", "apple", "cherry", "date"]
sorted(words, key=len)                      # By length
sorted(words, key=lambda w: (len(w), w))    # Multi-key: length, then alpha

# --- Sort objects ---
people = [("Alice", 30), ("Bob", 25), ("Charlie", 35)]
sorted(people, key=lambda p: p[1])          # By age

# --- functools.cmp_to_key — when you need a comparison function ---
import functools
def compare(a, b):
    # Return negative if a < b, 0 if equal, positive if a > b
    return a - b

sorted([3, 1, 4, 1, 5], key=functools.cmp_to_key(compare))

# Classic use: sort numbers to form the largest number
nums = [10, 9, 2, 45, 98]
def largest_number_cmp(a, b):
    if a + b > b + a: return -1
    elif a + b < b + a: return 1
    return 0

nums_str = [str(n) for n in nums]
sorted(nums_str, key=functools.cmp_to_key(largest_number_cmp))

# --- min / max with key ---
max(words, key=len)         # Longest word
min(people, key=lambda p: p[1])  # Youngest person

# --- bisect — binary search on sorted list ---
import bisect
a = [1, 3, 4, 7, 10]
bisect.bisect_left(a, 4)    # 2 — leftmost position to insert 4
bisect.bisect_right(a, 4)   # 3 — rightmost position to insert 4
bisect.insort_left(a, 5)    # Insert in sorted order

# Count elements < target
def count_less_than(a, target):
    return bisect.bisect_left(a, target)

# Count elements in [lo, hi]
def count_in_range(a, lo, hi):
    return bisect.bisect_right(a, hi) - bisect.bisect_left(a, lo)

12. Bit Manipulation

# --- Operators ---
# &   AND         5 & 3  -> 1     (0101 & 0011 = 0001)
# |   OR          5 | 3  -> 7     (0101 | 0011 = 0111)
# ^   XOR         5 ^ 3  -> 6     (0101 ^ 0011 = 0110)
# ~   NOT         ~5     -> -6    (inverts all bits, ~n = -(n+1))
# <<  Left shift  1 << 3 -> 8     (multiply by 2^3)
# >>  Right shift 8 >> 2 -> 2     (floor divide by 2^2)

# --- Common bit tricks ---
n = 42
n & 1               # Check if odd (last bit)
n | 1               # Set last bit (make odd)
n & ~1              # Clear last bit (make even)
n ^ n               # 0 — XOR with itself

# Get bit i
(n >> i) & 1

# Set bit i
n | (1 << i)

# Clear bit i
n & ~(1 << i)

# Toggle bit i
n ^ (1 << i)

# Check power of 2
n > 0 and (n & (n - 1)) == 0

# Remove lowest set bit (Brian Kernighan's trick)
n & (n - 1)

# Extract lowest set bit
n & (-n)

# Count set bits (popcount)
bin(n).count('1')
n.bit_count()           # Python 3.10+

# Swap without temp
a, b = 5, 3
a ^= b; b ^= a; a ^= b

# XOR trick: find the unique number where all others appear twice
nums = [1, 2, 3, 2, 1]
result = 0
for num in nums:
    result ^= num       # result = 3

# --- int to binary string and back ---
bin(42)                 # '0b101010'
bin(42)[2:]             # '101010'
int('101010', 2)        # 42
format(42, '08b')       # '00101010' (8-bit, zero-padded)

# --- Python has arbitrary precision ints — no overflow! ---
# But: ~n = -(n+1), and right shift of negative fills with 1 (arithmetic shift)
# To simulate 32-bit: use mask = 0xFFFFFFFF

13. Collections Module

from collections import Counter, defaultdict, deque, OrderedDict, namedtuple, ChainMap

# --- Counter ---
c = Counter("abracadabra")          # Counter({'a': 5, 'b': 2, 'r': 2, 'c': 1, 'd': 1})
c.most_common(2)                    # [('a', 5), ('b', 2)]
c["z"]                              # 0 — doesn't raise KeyError!
c + Counter("aaa")                  # Add counts
c - Counter("aaa")                  # Subtract (drops negatives)
c.elements()                        # Iterator of elements with their counts
sum(c.values())                     # Total count

# --- deque (double-ended queue) ---
# O(1) append/pop from both ends; O(n) access by index
dq = deque([1, 2, 3])
dq.append(4)            # Right end
dq.appendleft(0)        # Left end
dq.pop()                # Remove right
dq.popleft()            # Remove left — this is what makes it O(1) vs list!
dq.rotate(2)            # Rotate right by 2
dq.rotate(-2)           # Rotate left by 2
deque([1, 2, 3], maxlen=3)  # Fixed-size sliding window

# --- OrderedDict ---
od = OrderedDict()
od['a'] = 1
od['b'] = 2
od.move_to_end('a')         # Move to last
od.move_to_end('a', last=False)  # Move to first
od.popitem(last=True)       # LIFO pop
od.popitem(last=False)      # FIFO pop

# --- ChainMap — combine dicts, lookups search all ---
defaults = {'color': 'red', 'size': 'M'}
overrides = {'color': 'blue'}
combined = ChainMap(overrides, defaults)
combined['color']   # 'blue' — from overrides
combined['size']    # 'M'    — from defaults

14. Heapq & Priority Queues

import heapq

# Python's heapq is a MIN-heap
# To simulate MAX-heap: negate values

nums = [3, 1, 4, 1, 5, 9, 2, 6]
heapq.heapify(nums)             # In-place: O(n)
heapq.heappush(nums, 0)         # O(log n)
smallest = heapq.heappop(nums)  # O(log n) — removes and returns smallest
nums[0]                         # Peek at smallest: O(1) (don't pop!)

heapq.heappushpop(nums, 7)      # Push then pop (more efficient than separate calls)
heapq.heapreplace(nums, 7)      # Pop then push (heap must be non-empty)

heapq.nsmallest(3, nums)        # 3 smallest: O(n log k)
heapq.nlargest(3, nums)         # 3 largest:  O(n log k)

# --- Max-heap pattern ---
max_heap = [-x for x in [3, 1, 4, 1, 5, 9]]
heapq.heapify(max_heap)
largest = -heapq.heappop(max_heap)  # 9

# --- Heap with custom key (tuples) ---
# Heap compares element by element: (priority, data)
tasks = [(3, "low priority"), (1, "urgent"), (2, "medium")]
heapq.heapify(tasks)
priority, task = heapq.heappop(tasks)   # (1, "urgent")

# When data is not comparable, use (priority, counter, data) to break ties
from itertools import count
unique = count()
pq = []
heapq.heappush(pq, (1, next(unique), "task_a"))
heapq.heappush(pq, (1, next(unique), "task_b"))  # Counter breaks tie

# --- K Largest Elements ---
def k_largest(nums, k):
    return heapq.nlargest(k, nums)

# --- K Smallest using max-heap of size k ---
def k_smallest_heap(nums, k):
    heap = []
    for num in nums:
        heapq.heappush(heap, -num)
        if len(heap) > k:
            heapq.heappop(heap)
    return [-x for x in heap]

# --- Merge K sorted lists ---
def merge_k_sorted(lists):
    result = []
    heap = []
    for i, lst in enumerate(lists):
        if lst:
            heapq.heappush(heap, (lst[0], i, 0))
    while heap:
        val, list_idx, elem_idx = heapq.heappop(heap)
        result.append(val)
        if elem_idx + 1 < len(lists[list_idx]):
            nxt = lists[list_idx][elem_idx + 1]
            heapq.heappush(heap, (nxt, list_idx, elem_idx + 1))
    return result

15. Functional Programming

from functools import reduce

# --- lambda ---
square = lambda x: x ** 2
add = lambda x, y: x + y

# --- map / filter / reduce ---
list(map(str, [1, 2, 3]))               # ['1', '2', '3']
list(map(lambda x: x*2, [1, 2, 3]))    # [2, 4, 6]
list(filter(lambda x: x % 2, [1,2,3,4]))  # [1, 3]  — odd numbers
reduce(lambda acc, x: acc + x, [1,2,3,4,5], 0)  # 15

# Prefer comprehensions over map/filter in most cases (more Pythonic)
# Use map/filter when working with existing functions:
list(map(int, "12345"))                 # [1, 2, 3, 4, 5]
list(map(abs, [-1, -2, 3]))            # [1, 2, 3]

# --- partial functions ---
from functools import partial
def power(base, exp):
    return base ** exp

square = partial(power, exp=2)
cube = partial(power, exp=3)
square(5)   # 25
cube(3)     # 27

# --- operator module (avoid lambdas for simple ops) ---
import operator
operator.add(1, 2)      # 3
operator.itemgetter(1)([1, 2, 3])   # 2 — useful for sorting
operator.attrgetter('age')

sorted(people, key=operator.itemgetter(1))  # Sort by second element

16. Recursion & Memoization

import sys
sys.setrecursionlimit(10**6)    # Default is 1000 — increase for deep recursion

# --- Classic recursion patterns ---

# Factorial
def factorial(n):
    if n <= 1: return 1
    return n * factorial(n - 1)

# Power (fast exponentiation)
def power(base, exp):
    if exp == 0: return 1
    if exp % 2 == 0:
        half = power(base, exp // 2)
        return half * half
    return base * power(base, exp - 1)

# --- Memoization — top-down DP ---
# Option 1: lru_cache decorator
@lru_cache(maxsize=None)
def fib(n):
    if n < 2: return n
    return fib(n-1) + fib(n-2)

# Option 2: Manual memo dict
def fib_memo(n, memo={}):
    if n in memo: return memo[n]
    if n < 2: return n
    memo[n] = fib_memo(n-1, memo) + fib_memo(n-2, memo)
    return memo[n]

# Option 3: Pass memo explicitly (avoids mutable default gotcha)
def fib_clean(n, memo=None):
    if memo is None: memo = {}
    if n in memo: return memo[n]
    if n < 2: return n
    memo[n] = fib_clean(n-1, memo) + fib_clean(n-2, memo)
    return memo[n]

# --- Recursion to iteration (using explicit stack) ---
def tree_traversal_iterative(root):
    stack = [root]
    result = []
    while stack:
        node = stack.pop()
        if node:
            result.append(node.val)
            stack.append(node.right)
            stack.append(node.left)
    return result

17. Dynamic Programming Patterns

# --- 1. Fibonacci / linear DP ---
def fib_dp(n):
    if n < 2: return n
    a, b = 0, 1
    for _ in range(2, n + 1):
        a, b = b, a + b
    return b

# --- 2. 0/1 Knapsack ---
def knapsack(weights, values, capacity):
    n = len(weights)
    dp = [0] * (capacity + 1)
    for i in range(n):
        for w in range(capacity, weights[i] - 1, -1):  # Iterate backwards!
            dp[w] = max(dp[w], dp[w - weights[i]] + values[i])
    return dp[capacity]

# --- 3. Unbounded Knapsack (Coin Change) ---
def coin_change(coins, amount):
    dp = [float('inf')] * (amount + 1)
    dp[0] = 0
    for amt in range(1, amount + 1):
        for coin in coins:
            if coin <= amt:
                dp[amt] = min(dp[amt], dp[amt - coin] + 1)
    return dp[amount] if dp[amount] != float('inf') else -1

# --- 4. Longest Common Subsequence (LCS) ---
def lcs(s1, s2):
    m, n = len(s1), len(s2)
    dp = [[0] * (n + 1) for _ in range(m + 1)]
    for i in range(1, m + 1):
        for j in range(1, n + 1):
            if s1[i-1] == s2[j-1]:
                dp[i][j] = dp[i-1][j-1] + 1
            else:
                dp[i][j] = max(dp[i-1][j], dp[i][j-1])
    return dp[m][n]

# --- 5. Longest Increasing Subsequence (LIS) — O(n log n) ---
def lis(nums):
    tails = []
    for num in nums:
        pos = bisect.bisect_left(tails, num)
        if pos == len(tails):
            tails.append(num)
        else:
            tails[pos] = num
    return len(tails)

# --- 6. Edit Distance ---
def edit_distance(s, t):
    m, n = len(s), len(t)
    dp = list(range(n + 1))
    for i in range(1, m + 1):
        new_dp = [i] + [0] * n
        for j in range(1, n + 1):
            if s[i-1] == t[j-1]:
                new_dp[j] = dp[j-1]
            else:
                new_dp[j] = 1 + min(dp[j], new_dp[j-1], dp[j-1])
        dp = new_dp
    return dp[n]

# --- 7. Max Subarray — Kadane's Algorithm ---
def max_subarray(nums):
    max_sum = curr_sum = nums[0]
    for num in nums[1:]:
        curr_sum = max(num, curr_sum + num)
        max_sum = max(max_sum, curr_sum)
    return max_sum

# --- 8. House Robber pattern ---
def house_robber(nums):
    if not nums: return 0
    prev2, prev1 = 0, 0
    for num in nums:
        prev2, prev1 = prev1, max(prev1, prev2 + num)
    return prev1

# --- 9. 2D DP — Unique Paths ---
def unique_paths(m, n):
    dp = [1] * n
    for _ in range(1, m):
        for j in range(1, n):
            dp[j] += dp[j-1]
    return dp[n-1]

# --- 10. DP on intervals — Burst Balloons ---
def max_coins(nums):
    nums = [1] + nums + [1]
    n = len(nums)
    dp = [[0] * n for _ in range(n)]
    for length in range(2, n):
        for left in range(0, n - length):
            right = left + length
            for k in range(left + 1, right):
                dp[left][right] = max(
                    dp[left][right],
                    nums[left] * nums[k] * nums[right] + dp[left][k] + dp[k][right]
                )
    return dp[0][n-1]

# --- DP Tips ---
# 1. Define dp[i] clearly: "dp[i] = max profit using first i items"
# 2. Base cases: empty input, single element
# 3. Space optimization: often 2D -> 1D (rolling array)
# 4. Top-down (memoization) vs bottom-up (tabulation)
# 5. Common patterns: subsequence, subarray, partition, interval, state machine

18. Graph Algorithms

from collections import deque

# --- Graph representations ---
# Adjacency list (most common)
graph = {
    0: [1, 2],
    1: [0, 3],
    2: [0, 4],
    3: [1],
    4: [2]
}

# Adjacency list with weights
weighted = {
    0: [(1, 4), (2, 1)],    # (neighbor, weight)
    1: [(3, 1)],
    2: [(1, 2), (3, 5)],
    3: []
}

# --- BFS ---
def bfs(graph, start):
    visited = set([start])
    queue = deque([start])
    order = []
    while queue:
        node = queue.popleft()
        order.append(node)
        for neighbor in graph[node]:
            if neighbor not in visited:
                visited.add(neighbor)
                queue.append(neighbor)
    return order

# BFS shortest path
def bfs_shortest(graph, start, end):
    visited = {start: None}     # node -> parent
    queue = deque([start])
    while queue:
        node = queue.popleft()
        if node == end:
            path = []
            while node is not None:
                path.append(node)
                node = visited[node]
            return path[::-1]
        for neighbor in graph[node]:
            if neighbor not in visited:
                visited[neighbor] = node
                queue.append(neighbor)
    return []   # No path

# --- DFS (iterative) ---
def dfs_iterative(graph, start):
    visited = set()
    stack = [start]
    order = []
    while stack:
        node = stack.pop()
        if node not in visited:
            visited.add(node)
            order.append(node)
            for neighbor in graph[node]:
                if neighbor not in visited:
                    stack.append(neighbor)
    return order

# --- DFS (recursive) ---
def dfs_recursive(graph, node, visited=None):
    if visited is None: visited = set()
    visited.add(node)
    for neighbor in graph[node]:
        if neighbor not in visited:
            dfs_recursive(graph, neighbor, visited)
    return visited

# --- Detect cycle in undirected graph (DFS) ---
def has_cycle_undirected(graph):
    visited = set()
    def dfs(node, parent):
        visited.add(node)
        for neighbor in graph[node]:
            if neighbor not in visited:
                if dfs(neighbor, node): return True
            elif neighbor != parent:
                return True     # Back edge found
        return False
    for node in graph:
        if node not in visited:
            if dfs(node, -1): return True
    return False

# --- Topological Sort (Kahn's algorithm — BFS) ---
def topo_sort_kahn(n, edges):
    in_degree = [0] * n
    adj = defaultdict(list)
    for u, v in edges:
        adj[u].append(v)
        in_degree[v] += 1
    queue = deque(i for i in range(n) if in_degree[i] == 0)
    order = []
    while queue:
        node = queue.popleft()
        order.append(node)
        for neighbor in adj[node]:
            in_degree[neighbor] -= 1
            if in_degree[neighbor] == 0:
                queue.append(neighbor)
    return order if len(order) == n else []  # Empty = cycle exists

# --- Topological Sort (DFS) ---
def topo_sort_dfs(graph):
    visited, stack = set(), []
    def dfs(node):
        visited.add(node)
        for neighbor in graph.get(node, []):
            if neighbor not in visited:
                dfs(neighbor)
        stack.append(node)
    for node in graph:
        if node not in visited:
            dfs(node)
    return stack[::-1]

# --- Dijkstra's Algorithm (shortest path, non-negative weights) ---
def dijkstra(graph, start):
    dist = defaultdict(lambda: float('inf'))
    dist[start] = 0
    heap = [(0, start)]    # (distance, node)
    while heap:
        d, node = heapq.heappop(heap)
        if d > dist[node]: continue     # Stale entry
        for neighbor, weight in graph[node]:
            new_dist = dist[node] + weight
            if new_dist < dist[neighbor]:
                dist[neighbor] = new_dist
                heapq.heappush(heap, (new_dist, neighbor))
    return dist

# --- Union-Find (Disjoint Set Union) ---
class UnionFind:
    def __init__(self, n):
        self.parent = list(range(n))
        self.rank = [0] * n
        self.components = n

    def find(self, x):
        if self.parent[x] != x:
            self.parent[x] = self.find(self.parent[x])  # Path compression
        return self.parent[x]

    def union(self, x, y):
        px, py = self.find(x), self.find(y)
        if px == py: return False   # Already connected
        # Union by rank
        if self.rank[px] < self.rank[py]:
            px, py = py, px
        self.parent[py] = px
        if self.rank[px] == self.rank[py]:
            self.rank[px] += 1
        self.components -= 1
        return True

    def connected(self, x, y):
        return self.find(x) == self.find(y)

# --- Number of Islands (BFS on grid) ---
def num_islands(grid):
    if not grid: return 0
    rows, cols = len(grid), len(grid[0])
    count = 0
    def bfs(r, c):
        queue = deque([(r, c)])
        grid[r][c] = '0'
        while queue:
            row, col = queue.popleft()
            for dr, dc in [(-1,0),(1,0),(0,-1),(0,1)]:
                nr, nc = row + dr, col + dc
                if 0 <= nr < rows and 0 <= nc < cols and grid[nr][nc] == '1':
                    grid[nr][nc] = '0'
                    queue.append((nr, nc))
    for r in range(rows):
        for c in range(cols):
            if grid[r][c] == '1':
                bfs(r, c)
                count += 1
    return count

# --- Bellman-Ford (handles negative edges) ---
def bellman_ford(n, edges, src):
    dist = [float('inf')] * n
    dist[src] = 0
    for _ in range(n - 1):
        for u, v, w in edges:
            if dist[u] + w < dist[v]:
                dist[v] = dist[u] + w
    # Check for negative cycles
    for u, v, w in edges:
        if dist[u] + w < dist[v]:
            return None     # Negative cycle exists
    return dist

# --- Directions for 2D grid problems ---
DIRS_4 = [(-1, 0), (1, 0), (0, -1), (0, 1)]          # Up, down, left, right
DIRS_8 = [(-1,-1),(-1,0),(-1,1),(0,-1),(0,1),(1,-1),(1,0),(1,1)]  # All 8

def in_bounds(r, c, rows, cols):
    return 0 <= r < rows and 0 <= c < cols

19. Tree Algorithms

class TreeNode:
    def __init__(self, val=0, left=None, right=None):
        self.val = val
        self.left = left
        self.right = right

    def __repr__(self):
        return f"TreeNode({self.val})"

# --- Build tree from level-order list ---
def build_tree(values):
    if not values or values[0] is None: return None
    root = TreeNode(values[0])
    queue = deque([root])
    i = 1
    while queue and i < len(values):
        node = queue.popleft()
        if i < len(values) and values[i] is not None:
            node.left = TreeNode(values[i])
            queue.append(node.left)
        i += 1
        if i < len(values) and values[i] is not None:
            node.right = TreeNode(values[i])
            queue.append(node.right)
        i += 1
    return root

# --- Traversals ---
def inorder(root):          # Left -> Root -> Right (gives sorted BST)
    if not root: return []
    return inorder(root.left) + [root.val] + inorder(root.right)

def preorder(root):         # Root -> Left -> Right
    if not root: return []
    return [root.val] + preorder(root.left) + preorder(root.right)

def postorder(root):        # Left -> Right -> Root
    if not root: return []
    return postorder(root.left) + postorder(root.right) + [root.val]

# Iterative inorder (important!)
def inorder_iterative(root):
    stack, result = [], []
    curr = root
    while curr or stack:
        while curr:
            stack.append(curr)
            curr = curr.left
        curr = stack.pop()
        result.append(curr.val)
        curr = curr.right
    return result

# Level-order (BFS)
def level_order(root):
    if not root: return []
    queue = deque([root])
    result = []
    while queue:
        level = []
        for _ in range(len(queue)):
            node = queue.popleft()
            level.append(node.val)
            if node.left: queue.append(node.left)
            if node.right: queue.append(node.right)
        result.append(level)
    return result

# --- Tree properties ---
def height(root):
    if not root: return 0
    return 1 + max(height(root.left), height(root.right))

def is_balanced(root):
    def check(node):
        if not node: return 0
        left = check(node.left)
        if left == -1: return -1
        right = check(node.right)
        if right == -1: return -1
        if abs(left - right) > 1: return -1
        return 1 + max(left, right)
    return check(root) != -1

def diameter(root):
    result = [0]
    def depth(node):
        if not node: return 0
        left, right = depth(node.left), depth(node.right)
        result[0] = max(result[0], left + right)
        return 1 + max(left, right)
    depth(root)
    return result[0]

# --- BST operations ---
def bst_search(root, val):
    if not root or root.val == val: return root
    if val < root.val: return bst_search(root.left, val)
    return bst_search(root.right, val)

def bst_insert(root, val):
    if not root: return TreeNode(val)
    if val < root.val: root.left = bst_insert(root.left, val)
    elif val > root.val: root.right = bst_insert(root.right, val)
    return root

def bst_inorder_successor(root, target):
    successor = None
    while root:
        if target.val < root.val:
            successor = root
            root = root.left
        else:
            root = root.right
    return successor

# --- LCA (Lowest Common Ancestor) ---
def lca(root, p, q):
    if not root or root == p or root == q: return root
    left = lca(root.left, p, q)
    right = lca(root.right, p, q)
    if left and right: return root  # p and q on different sides
    return left or right

# LCA for BST (more efficient):
def lca_bst(root, p, q):
    while root:
        if p.val < root.val and q.val < root.val:
            root = root.left
        elif p.val > root.val and q.val > root.val:
            root = root.right
        else:
            return root

20. Linked List Patterns

class ListNode:
    def __init__(self, val=0, next=None):
        self.val = val
        self.next = next

def make_list(values):
    dummy = ListNode(0)
    curr = dummy
    for v in values:
        curr.next = ListNode(v)
        curr = curr.next
    return dummy.next

def to_list(head):
    result = []
    while head:
        result.append(head.val)
        head = head.next
    return result

# --- Reverse linked list (iterative) ---
def reverse_list(head):
    prev, curr = None, head
    while curr:
        nxt = curr.next
        curr.next = prev
        prev = curr
        curr = nxt
    return prev

# --- Reverse linked list (recursive) ---
def reverse_recursive(head):
    if not head or not head.next: return head
    new_head = reverse_recursive(head.next)
    head.next.next = head
    head.next = None
    return new_head

# --- Detect cycle (Floyd's algorithm) ---
def has_cycle(head):
    slow = fast = head
    while fast and fast.next:
        slow = slow.next
        fast = fast.next.next
        if slow == fast: return True
    return False

def find_cycle_start(head):
    slow = fast = head
    while fast and fast.next:
        slow = slow.next
        fast = fast.next.next
        if slow == fast:
            slow = head
            while slow != fast:
                slow = slow.next
                fast = fast.next
            return slow
    return None

# --- Middle of linked list ---
def find_middle(head):
    slow = fast = head
    while fast and fast.next:
        slow = slow.next
        fast = fast.next.next
    return slow

# --- Merge two sorted lists ---
def merge_sorted(l1, l2):
    dummy = curr = ListNode(0)
    while l1 and l2:
        if l1.val <= l2.val:
            curr.next = l1; l1 = l1.next
        else:
            curr.next = l2; l2 = l2.next
        curr = curr.next
    curr.next = l1 or l2
    return dummy.next

# --- Remove Nth node from end ---
def remove_nth_from_end(head, n):
    dummy = ListNode(0, head)
    fast = slow = dummy
    for _ in range(n + 1):
        fast = fast.next
    while fast:
        slow = slow.next
        fast = fast.next
    slow.next = slow.next.next
    return dummy.next

# --- Palindrome linked list ---
def is_palindrome_list(head):
    slow = fast = head
    while fast and fast.next:
        slow = slow.next
        fast = fast.next.next
    # Reverse second half
    prev, curr = None, slow
    while curr:
        nxt = curr.next; curr.next = prev; prev = curr; curr = nxt
    # Compare
    left, right = head, prev
    while right:
        if left.val != right.val: return False
        left = left.next; right = right.next
    return True

21. Sliding Window & Two Pointers

# --- Fixed-size sliding window ---
def max_sum_subarray(nums, k):
    window_sum = sum(nums[:k])
    max_sum = window_sum
    for i in range(k, len(nums)):
        window_sum += nums[i] - nums[i - k]
        max_sum = max(max_sum, window_sum)
    return max_sum

# --- Variable-size sliding window ---
# Template:
def sliding_window_variable(nums, target):
    left = 0
    window_state = 0    # Could be sum, count, dict, etc.
    result = 0
    for right in range(len(nums)):
        window_state += nums[right]     # Expand window
        while window_state > target:    # Shrink condition
            window_state -= nums[left]
            left += 1
        result = max(result, right - left + 1)
    return result

# Longest substring without repeating characters
def length_of_longest_substring(s):
    char_count = {}
    left = 0
    max_len = 0
    for right, ch in enumerate(s):
        char_count[ch] = char_count.get(ch, 0) + 1
        while char_count[ch] > 1:
            char_count[s[left]] -= 1
            if char_count[s[left]] == 0:
                del char_count[s[left]]
            left += 1
        max_len = max(max_len, right - left + 1)
    return max_len

# Minimum window substring
def min_window(s, t):
    need = Counter(t)
    missing = len(t)
    best = ""
    left = 0
    for right, ch in enumerate(s):
        if need[ch] > 0: missing -= 1
        need[ch] -= 1
        if missing == 0:
            while need[s[left]] < 0:
                need[s[left]] += 1
                left += 1
            if not best or right - left + 1 < len(best):
                best = s[left:right+1]
            need[s[left]] += 1
            missing += 1
            left += 1
    return best

# --- Two pointers ---
# Two sum (sorted array)
def two_sum_sorted(nums, target):
    left, right = 0, len(nums) - 1
    while left < right:
        s = nums[left] + nums[right]
        if s == target: return [left, right]
        elif s < target: left += 1
        else: right -= 1
    return []

# Container with most water
def max_water(height):
    left, right = 0, len(height) - 1
    max_area = 0
    while left < right:
        area = min(height[left], height[right]) * (right - left)
        max_area = max(max_area, area)
        if height[left] < height[right]: left += 1
        else: right -= 1
    return max_area

# 3Sum
def three_sum(nums):
    nums.sort()
    result = []
    for i, a in enumerate(nums):
        if i > 0 and nums[i] == nums[i-1]: continue
        left, right = i + 1, len(nums) - 1
        while left < right:
            s = a + nums[left] + nums[right]
            if s == 0:
                result.append([a, nums[left], nums[right]])
                while left < right and nums[left] == nums[left+1]: left += 1
                while left < right and nums[right] == nums[right-1]: right -= 1
                left += 1; right -= 1
            elif s < 0: left += 1
            else: right -= 1
    return result

22. Binary Search Patterns

# --- Classic binary search ---
def binary_search(nums, target):
    left, right = 0, len(nums) - 1
    while left <= right:
        mid = left + (right - left) // 2    # Avoids overflow (critical in other languages)
        if nums[mid] == target: return mid
        elif nums[mid] < target: left = mid + 1
        else: right = mid - 1
    return -1

# --- Find leftmost position (lower bound) ---
def lower_bound(nums, target):
    left, right = 0, len(nums)
    while left < right:
        mid = (left + right) // 2
        if nums[mid] < target: left = mid + 1
        else: right = mid
    return left     # First index where nums[i] >= target

# --- Find rightmost position (upper bound) ---
def upper_bound(nums, target):
    left, right = 0, len(nums)
    while left < right:
        mid = (left + right) // 2
        if nums[mid] <= target: left = mid + 1
        else: right = mid
    return left - 1     # Last index where nums[i] <= target

# --- Binary search on answer (search space) ---
# Template: "find minimum X such that condition(X) is True"
def binary_search_answer(lo, hi, condition):
    while lo < hi:
        mid = (lo + hi) // 2
        if condition(mid):
            hi = mid
        else:
            lo = mid + 1
    return lo

# Example: Koko eating bananas
def min_eating_speed(piles, h):
    def can_finish(speed):
        return sum(math.ceil(p / speed) for p in piles) <= h
    return binary_search_answer(1, max(piles), can_finish)

# Example: Minimum capacity to ship packages
def ship_capacity(weights, days):
    def can_ship(cap):
        d, curr = 1, 0
        for w in weights:
            if curr + w > cap: d += 1; curr = 0
            curr += w
        return d <= days
    return binary_search_answer(max(weights), sum(weights), can_ship)

# --- Search in rotated sorted array ---
def search_rotated(nums, target):
    left, right = 0, len(nums) - 1
    while left <= right:
        mid = (left + right) // 2
        if nums[mid] == target: return mid
        if nums[left] <= nums[mid]:     # Left half is sorted
            if nums[left] <= target < nums[mid]: right = mid - 1
            else: left = mid + 1
        else:                           # Right half is sorted
            if nums[mid] < target <= nums[right]: left = mid + 1
            else: right = mid - 1
    return -1

# --- Find peak element ---
def find_peak(nums):
    left, right = 0, len(nums) - 1
    while left < right:
        mid = (left + right) // 2
        if nums[mid] > nums[mid + 1]: right = mid
        else: left = mid + 1
    return left

23. Stack & Queue Patterns

# --- Monotonic stack ---
# Useful for: next greater/smaller element, histogram problems

# Next Greater Element
def next_greater_element(nums):
    result = [-1] * len(nums)
    stack = []  # Stores indices
    for i, n in enumerate(nums):
        while stack and nums[stack[-1]] < n:
            result[stack.pop()] = n
        stack.append(i)
    return result

# Largest Rectangle in Histogram
def largest_rectangle(heights):
    stack = []
    max_area = 0
    heights = heights + [0]     # Sentinel to flush stack
    for i, h in enumerate(heights):
        start = i
        while stack and stack[-1][1] > h:
            idx, height = stack.pop()
            max_area = max(max_area, height * (i - idx))
            start = idx
        stack.append((start, h))
    return max_area

# Trapping Rain Water (monotonic stack OR two pointers)
def trap_rain(height):
    left, right = 0, len(height) - 1
    left_max = right_max = water = 0
    while left < right:
        if height[left] < height[right]:
            if height[left] >= left_max: left_max = height[left]
            else: water += left_max - height[left]
            left += 1
        else:
            if height[right] >= right_max: right_max = height[right]
            else: water += right_max - height[right]
            right -= 1
    return water

# Daily Temperatures (next greater using stack)
def daily_temps(temps):
    result = [0] * len(temps)
    stack = []
    for i, t in enumerate(temps):
        while stack and temps[stack[-1]] < t:
            j = stack.pop()
            result[j] = i - j
        stack.append(i)
    return result

# --- Valid parentheses ---
def is_valid_parens(s):
    stack = []
    pairs = {')': '(', '}': '{', ']': '['}
    for ch in s:
        if ch in '({[': stack.append(ch)
        elif not stack or stack[-1] != pairs[ch]: return False
        else: stack.pop()
    return len(stack) == 0

# --- Queue using two stacks ---
class MyQueue:
    def __init__(self):
        self.in_stack, self.out_stack = [], []

    def push(self, x):
        self.in_stack.append(x)

    def pop(self):
        self._transfer()
        return self.out_stack.pop()

    def peek(self):
        self._transfer()
        return self.out_stack[-1]

    def _transfer(self):
        if not self.out_stack:
            while self.in_stack:
                self.out_stack.append(self.in_stack.pop())

# --- Min Stack ---
class MinStack:
    def __init__(self):
        self.stack = []
        self.min_stack = []

    def push(self, val):
        self.stack.append(val)
        min_val = min(val, self.min_stack[-1] if self.min_stack else val)
        self.min_stack.append(min_val)

    def pop(self):
        self.stack.pop()
        self.min_stack.pop()

    def get_min(self):
        return self.min_stack[-1]

24. Backtracking

# Template:
# def backtrack(state, choices):
#     if base_case(state):
#         result.append(state.copy())
#         return
#     for choice in choices:
#         if is_valid(choice, state):
#             make_choice(state, choice)
#             backtrack(state, updated_choices)
#             undo_choice(state, choice)      <- KEY: restore state

# --- Permutations ---
def permutations(nums):
    result = []
    def backtrack(path, remaining):
        if not remaining:
            result.append(path[:])
            return
        for i, n in enumerate(remaining):
            path.append(n)
            backtrack(path, remaining[:i] + remaining[i+1:])
            path.pop()
    backtrack([], nums)
    return result

# --- Subsets ---
def subsets(nums):
    result = []
    def backtrack(start, path):
        result.append(path[:])
        for i in range(start, len(nums)):
            path.append(nums[i])
            backtrack(i + 1, path)
            path.pop()
    backtrack(0, [])
    return result

# --- Combination Sum (unlimited use) ---
def combination_sum(candidates, target):
    result = []
    def backtrack(start, path, remaining):
        if remaining == 0:
            result.append(path[:])
            return
        for i in range(start, len(candidates)):
            if candidates[i] > remaining: break
            path.append(candidates[i])
            backtrack(i, path, remaining - candidates[i])   # i (not i+1) for reuse
            path.pop()
    candidates.sort()
    backtrack(0, [], target)
    return result

# --- N-Queens ---
def solve_n_queens(n):
    result = []
    cols = set()
    diag1 = set()   # row - col
    diag2 = set()   # row + col

    def backtrack(row, board):
        if row == n:
            result.append(["".join(r) for r in board])
            return
        for col in range(n):
            if col in cols or (row - col) in diag1 or (row + col) in diag2:
                continue
            cols.add(col); diag1.add(row - col); diag2.add(row + col)
            board[row][col] = 'Q'
            backtrack(row + 1, board)
            cols.remove(col); diag1.remove(row - col); diag2.remove(row + col)
            board[row][col] = '.'

    board = [['.' for _ in range(n)] for _ in range(n)]
    backtrack(0, board)
    return result

# --- Word Search ---
def word_search(board, word):
    rows, cols = len(board), len(board[0])
    def dfs(r, c, idx):
        if idx == len(word): return True
        if not (0 <= r < rows) or not (0 <= c < cols): return False
        if board[r][c] != word[idx]: return False
        tmp, board[r][c] = board[r][c], '#'
        found = any(dfs(r+dr, c+dc, idx+1) for dr, dc in DIRS_4)
        board[r][c] = tmp
        return found
    return any(dfs(r, c, 0) for r in range(rows) for c in range(cols))

25. Greedy Algorithms

# Greedy works when: local optimal choices lead to global optimum.
# Key: Think about what invariant you're maintaining.

# --- Activity Selection / Interval Scheduling ---
def max_non_overlapping(intervals):
    intervals.sort(key=lambda x: x[1])     # Sort by end time
    count = 0
    last_end = float('-inf')
    for start, end in intervals:
        if start >= last_end:
            count += 1
            last_end = end
    return count

# --- Merge Intervals ---
def merge_intervals(intervals):
    intervals.sort(key=lambda x: x[0])
    merged = [intervals[0]]
    for start, end in intervals[1:]:
        if start <= merged[-1][1]:
            merged[-1][1] = max(merged[-1][1], end)
        else:
            merged.append([start, end])
    return merged

# --- Jump Game ---
def can_jump(nums):
    max_reach = 0
    for i, jump in enumerate(nums):
        if i > max_reach: return False
        max_reach = max(max_reach, i + jump)
    return True

def min_jumps(nums):
    jumps = farthest = curr_end = 0
    for i in range(len(nums) - 1):
        farthest = max(farthest, i + nums[i])
        if i == curr_end:
            jumps += 1
            curr_end = farthest
    return jumps

# --- Gas Station ---
def can_complete_circuit(gas, cost):
    if sum(gas) < sum(cost): return -1
    tank = start = 0
    for i, (g, c) in enumerate(zip(gas, cost)):
        tank += g - c
        if tank < 0:
            start = i + 1
            tank = 0
    return start

# --- Task Scheduler ---
def task_scheduler(tasks, n):
    counts = Counter(tasks)
    max_freq = max(counts.values())
    max_count = sum(1 for v in counts.values() if v == max_freq)
    return max(len(tasks), (max_freq - 1) * (n + 1) + max_count)

26. Complexity Cheat Sheet

"""
DATA STRUCTURE COMPLEXITIES
─────────────────────────────────────────────────────────────────────────────
                     Access    Search    Insert    Delete    Space
─────────────────────────────────────────────────────────────────────────────
Array (list)         O(1)      O(n)      O(n)      O(n)      O(n)
list.append/pop()    O(1)*     -         O(1)*     O(1)*     -
dict / set           O(1)*     O(1)*     O(1)*     O(1)*     O(n)
Sorted dict/set      -         O(log n)  O(log n)  O(log n)  O(n)
Heap (heapq)         O(1)peek  O(n)      O(log n)  O(log n)  O(n)
deque append/pop     O(1)      O(n)      O(1)      O(1)      O(n)
Linked list          O(n)      O(n)      O(1)head  O(1)head  O(n)
BST (balanced)       O(log n)  O(log n)  O(log n)  O(log n)  O(n)
Trie                 O(m)      O(m)      O(m)      O(m)      O(ALPHABET*n)
  * = amortized

ALGORITHM COMPLEXITIES
─────────────────────────────────────────────────────────────────────────────
Binary Search            O(log n)
Sorting (comparison)     O(n log n)  — lower bound for comparison sort
Counting Sort            O(n + k)    — k is range of values
BFS / DFS                O(V + E)
Dijkstra (heap)          O((V+E) log V)
Bellman-Ford             O(VE)
Floyd-Warshall           O(V^3)
Topological Sort         O(V+E)
Union-Find               O(α(n)) ≈ O(1)  amortized with path compression
Knapsack (0/1)           O(n * W)
LCS                      O(m * n)
Matrix Multiply          O(n^3) naive, O(n^2.37) Strassen

PYTHON-SPECIFIC
─────────────────────────────────────────────────────────────────────────────
len(list/dict/str)       O(1)
x in list                O(n)    <- Use set for O(1) lookup!
x in set/dict            O(1)
list slicing [i:j]       O(j-i)
list concatenation       O(n)
"".join(list)            O(n)    <- Always use join, not +=
sorted()                 O(n log n)
list.sort()              O(n log n)  Timsort (stable, adaptive)
heapq.heapify            O(n)
Counter(iterable)        O(n)
"""

27. Python-Specific Tricks

# --- String building: always join, never += in a loop ---
# Slow — creates a new string each iteration: O(n^2)
result = ""
for c in "hello":
    result += c

# Fast — O(n)
result = "".join(["h", "e", "l", "l", "o"])

# --- Check if all chars are unique ---
def all_unique(s):
    return len(s) == len(set(s))

# --- Most frequent element ---
def most_frequent(nums):
    return Counter(nums).most_common(1)[0][0]

# --- Rotate list ---
def rotate(nums, k):
    k %= len(nums)
    nums[:] = nums[-k:] + nums[:-k]  # In-place with slice assignment

# --- Group anagrams ---
def group_anagrams(strs):
    d = defaultdict(list)
    for s in strs:
        d[tuple(sorted(s))].append(s)
    return list(d.values())

# --- Flatten nested structure (iterative) ---
def flatten(lst):
    stack = lst[::-1]
    result = []
    while stack:
        item = stack.pop()
        if isinstance(item, list):
            stack.extend(item[::-1])
        else:
            result.append(item)
    return result

# --- Prefix sums for range queries ---
def range_sum_setup(nums):
    prefix = [0] * (len(nums) + 1)
    for i, n in enumerate(nums):
        prefix[i+1] = prefix[i] + n
    def query(l, r):                # Sum of nums[l..r] inclusive
        return prefix[r+1] - prefix[l]
    return query

# --- Difference array for range updates ---
def range_add(nums, updates):
    diff = [0] * (len(nums) + 1)
    for l, r, val in updates:
        diff[l] += val
        diff[r+1] -= val
    curr = 0
    for i in range(len(nums)):
        curr += diff[i]
        nums[i] += curr
    return nums

# --- Two-pointer partition (Dutch National Flag) ---
def dutch_flag(nums):
    lo, mid, hi = 0, 0, len(nums) - 1
    while mid <= hi:
        if nums[mid] == 0:
            nums[lo], nums[mid] = nums[mid], nums[lo]
            lo += 1; mid += 1
        elif nums[mid] == 1:
            mid += 1
        else:
            nums[mid], nums[hi] = nums[hi], nums[mid]
            hi -= 1
    return nums

# --- Fast I/O for competitive programming ---
# input = sys.stdin.readline     # Faster than built-in input()
# print = sys.stdout.write       # Use sys.stdout.write(str + "\n")

# --- Integer sqrt without import ---
def isqrt(n):
    if n < 0: raise ValueError
    x = int(n ** 0.5)
    while x * x > n: x -= 1
    while (x + 1) * (x + 1) <= n: x += 1
    return x
# Python 3.8+: math.isqrt(n)

# --- GCD and LCM ---
from math import gcd
def lcm(a, b):
    return a * b // gcd(a, b)

# Python 3.9+: math.lcm(a, b)

# --- Useful math ---
import math
math.ceil(7/2)          # 4
math.floor(7/2)         # 3
round(2.5)              # 2 (banker's rounding!) round(3.5)=4
math.log2(8)            # 3.0
math.log(100, 10)       # 2.0

# --- All prime numbers up to n (Sieve of Eratosthenes) ---
def sieve(n):
    is_prime = [True] * (n + 1)
    is_prime[0] = is_prime[1] = False
    for i in range(2, int(n**0.5) + 1):
        if is_prime[i]:
            for j in range(i*i, n+1, i):
                is_prime[j] = False
    return [i for i in range(2, n+1) if is_prime[i]]

# --- Check prime ---
def is_prime(n):
    if n < 2: return False
    if n == 2: return True
    if n % 2 == 0: return False
    for i in range(3, int(n**0.5) + 1, 2):
        if n % i == 0: return False
    return True

# --- Trie data structure ---
class TrieNode:
    def __init__(self):
        self.children = {}
        self.is_end = False

class Trie:
    def __init__(self):
        self.root = TrieNode()

    def insert(self, word):
        node = self.root
        for ch in word:
            if ch not in node.children:
                node.children[ch] = TrieNode()
            node = node.children[ch]
        node.is_end = True

    def search(self, word):
        node = self.root
        for ch in word:
            if ch not in node.children: return False
            node = node.children[ch]
        return node.is_end

    def starts_with(self, prefix):
        node = self.root
        for ch in prefix:
            if ch not in node.children: return False
            node = node.children[ch]
        return True

# --- Segment Tree (range query + point update) ---
class SegmentTree:
    def __init__(self, nums):
        n = len(nums)
        self.n = n
        self.tree = [0] * (2 * n)
        self.tree[n:] = nums
        for i in range(n - 1, 0, -1):
            self.tree[i] = self.tree[2*i] + self.tree[2*i+1]

    def update(self, pos, val):
        pos += self.n
        self.tree[pos] = val
        while pos > 1:
            pos //= 2
            self.tree[pos] = self.tree[2*pos] + self.tree[2*pos+1]

    def query(self, l, r):      # Sum of [l, r)
        l += self.n; r += self.n
        result = 0
        while l < r:
            if l & 1: result += self.tree[l]; l += 1
            if r & 1: r -= 1; result += self.tree[r]
            l >>= 1; r >>= 1
        return result

# --- Binary Indexed Tree (Fenwick Tree) — simpler range sum ---
class BIT:
    def __init__(self, n):
        self.n = n
        self.tree = [0] * (n + 1)

    def update(self, i, delta):     # 1-indexed
        while i <= self.n:
            self.tree[i] += delta
            i += i & (-i)

    def query(self, i):             # Prefix sum [1..i]
        total = 0
        while i > 0:
            total += self.tree[i]
            i -= i & (-i)
        return total

    def range_query(self, l, r):    # Sum [l..r]
        return self.query(r) - self.query(l - 1)

# --------
# Testing in Python
# --------
# 1. The function you want to test
def add(a, b):
    return a + b

# 2. The test block
if __name__ == "__main__":
    # Test 1: Positive numbers
    assert add(2, 3) == 5, "Test 1 failed: Expected 5"

    # Test 2: Negative numbers
    assert add(-1, -1) == -2, "Test 2 failed: Expected -2"

    print("All tests passed!")

28. Common Patterns Summary

"""
PATTERN RECOGNITION GUIDE
─────────────────────────────────────────────────────────────────────────────

SIGNAL                                  PATTERN TO CONSIDER
─────────────────────────────────────────────────────────────────────────────
"Top K" / "K largest/smallest"          Heap (min or max)
"K closest"                             Heap with distance key
Sorted array + target                   Binary Search / Two Pointers
"Subarray sum = k"                      Prefix Sum + HashMap
"Longest subarray / substring"          Sliding Window
"All permutations / subsets"            Backtracking
"Optimal choices at each step"          Greedy or DP
"Overlapping subproblems"               DP (top-down memo or bottom-up)
"Next greater / smaller element"        Monotonic Stack
"Islands / connected components"        BFS/DFS on grid
"Shortest path (unweighted)"            BFS
"Shortest path (weighted)"              Dijkstra / Bellman-Ford
"Course prerequisites / ordering"       Topological Sort
"Union / Find groups"                   Union-Find (DSU)
"Word prefix / autocomplete"            Trie
"Range sum queries"                     Prefix Sum / Segment Tree / BIT
"Linked list cycle / middle"            Floyd's Slow/Fast Pointer
"Tree path / ancestor"                  DFS / LCA
"Bracket matching"                      Stack
"Rotated array"                         Binary Search (modified)
"Matrix search"                         Binary Search / BFS

DECISION TREE FOR DP
─────────────────────────────────────────────────────────────────────────────
1. Can I define subproblem as a suffix/prefix of input?  -> 1D DP (array)
2. Does it involve two sequences?                        -> 2D DP (LCS, Edit Distance)
3. Is it about choosing items with a budget/capacity?    -> Knapsack
4. Does the problem involve a range [i..j]?              -> Interval DP
5. Does it involve a state machine / transitions?        -> State DP
6. Does the recursion tree have repeated subproblems?    -> Memoize it

COMMON COMPLEXITY TARGETS
─────────────────────────────────────────────────────────────────────────────
n ≤ 10:          O(n!) — backtracking / all perms OK
n ≤ 20:          O(2^n) — bitmask DP / subset enumeration
n ≤ 500:         O(n^3) — Floyd-Warshall, interval DP
n ≤ 5000:        O(n^2) — 2D DP, brute force nested loops
n ≤ 10^6:        O(n log n) — sorting, heap, BFS/DFS
n ≤ 10^8:        O(n) — linear pass, prefix sum, hash
n > 10^8:        O(log n) or O(1) — binary search, math

CODING APPROACH
─────────────────────────────────────────────────────────────────────────────
1. Clarify: input size, edge cases, return type, constraints
2. Examples: walk through 2-3 examples including edge cases
3. Brute force: state it first, then optimize
4. Pattern: identify which paradigm applies
5. Complexity: analyze time and space before coding
6. Code: clean, readable, use helper functions
7. Test: trace through your examples, check edge cases

EDGE CASES TO ALWAYS CONSIDER
─────────────────────────────────────────────────────────────────────────────
[ ] Empty input ([], "", None, 0)
[ ] Single element
[ ] All same elements
[ ] Negative numbers / zero
[ ] Already sorted / reverse sorted
[ ] Overflow (Python ints are safe, but simulate 32-bit if asked)
[ ] Disconnected graph / forest
[ ] Null tree root
[ ] Cycle in linked list / graph
"""