Memory Management — Stack, Heap, Virtual Memory

Why Memory Management Matters

Memory is the most critical resource in any computer system. Poor memory management leads to crashes, security vulnerabilities, and terrible performance. Understanding how the OS manages memory helps you write efficient code, debug memory issues, and design scalable systems.

Why this matters for your career:

Memory leaks and buffer overflows are among the most common production bugs
Memory management knowledge is essential for systems programming (C, C++, Rust)
Even high-level languages (Python, JavaScript) benefit from understanding memory
Interview questions about stack vs. heap, garbage collection, and virtual memory are very common

Stack vs. Heap

Stack Memory

The stack is a region of memory that grows and shrinks automatically as functions are called and return. It is a LIFO (Last In, First Out) structure.

Stack characteristics:

Very fast allocation (just move the stack pointer)
Automatic deallocation when function returns
Limited size (typically 1-8 MB per thread)
Stores local variables, function parameters, return addresses
No fragmentation — allocation and deallocation are perfectly nested

Heap Memory

The heap is a region of memory used for dynamic allocation. The programmer (or garbage collector) controls when memory is allocated and freed.

Heap characteristics:

Slower allocation (must find a free block)
Manual or GC-based deallocation
Large size (limited by available RAM)
Stores objects, data structures, dynamically sized data
Can suffer from fragmentation

Comparison Table

| Aspect | Stack | Heap | |--------|-------|------| | Speed | Fast (pointer increment) | Slow (search for free block) | | Size | Limited (MB) | Large (GB) | | Lifetime | Function scope | Programmer controlled | | Allocation | Automatic | Manual (malloc/new) or GC | | Deallocation | Automatic on return | Manual (free/delete) or GC | | Fragmentation | None | External fragmentation possible | | Thread safety | Each thread has its own stack | Shared across threads | | Data type | Local variables, primitives | Objects, arrays, structs |

Code Example: Stack vs. Heap

import sys

def stack_example():
    # These variables are on the stack
    x = 10
    y = 20
    result = x + y
    return result

def heap_example():
    # Lists are allocated on the heap
    data = [1, 2, 3, 4, 5]
    # The reference 'data' is on the stack, but the list itself is on the heap
    return data

# In Python, integers are objects on the heap too
# But simple integers may be interned (cached)
print(f"Size of int: {sys.getsizeof(42)} bytes")      # 28 bytes
print(f"Size of list: {sys.getsizeof([1,2,3])} bytes")  # 88 bytes + 3*28 for ints

Virtual Memory

Virtual memory is an abstraction layer that gives each process the illusion of having its own private, contiguous address space. The OS maps virtual addresses to physical addresses using page tables.

Benefits of Virtual Memory

| Benefit | Description | |---------|-------------| | Isolation | Each process has its own address space, cannot access other processes' memory | | Security | A process cannot read or write kernel memory | | Simplified programming | Each program sees a contiguous address space starting at 0 | | Efficient RAM usage | Only active pages need to be in RAM; inactive pages can be paged to disk | | Shared memory | Multiple processes can map the same physical page (shared libraries) | | Overcommit | OS can allocate more virtual memory than physical RAM exists |

Paging

Memory is divided into fixed-size blocks called pages (typically 4 KB). The OS maintains a page table that maps virtual pages to physical frames.

Virtual Address → Page Number + Offset
                       ↓
                 Page Table (maps to physical frame)
                       ↓
            Physical Address = Frame Number + Offset

When a process accesses a virtual page that is not in RAM, a page fault occurs. The OS loads the page from disk (swap) into RAM, potentially evicting another page.

Page Replacement Algorithms

| Algorithm | Strategy | Pros | Cons | |-----------|----------|------|------| | FIFO | Evict oldest page | Simple | May evict frequently used pages | | LRU (Least Recently Used) | Evict page unused for longest | Good performance | Hard to implement exactly | | Clock (Second Chance) | Approximate LRU with a circular buffer | Good performance, efficient | Slightly worse than true LRU | | Optimal (Belady's) | Evict page used farthest in future | Best possible | Impossible (needs future knowledge) |

Memory Fragmentation

External Fragmentation

Occurs when free memory is broken into small blocks between allocated blocks. The total free memory may be large, but no single block is large enough to satisfy a request.

Internal Fragmentation

Occurs when allocated memory is slightly larger than requested. For example, allocating 30 bytes from a 32-byte block wastes 2 bytes.

Comparison

| Type | Cause | Solution | |------|-------|----------| | External | Variable-size allocations and deallocations | Memory compaction, or use slab allocator | | Internal | Fixed-size allocation blocks | Use variable-size allocation or buddy system |

Garbage Collection (GC)

Languages like Java, Python, Go, and JavaScript use automatic garbage collection to manage heap memory.

Common GC Algorithms

| Algorithm | How It Works | Pros | Cons | |-----------|-------------|------|------| | Reference Counting | Track number of references to each object | Simple, immediate reclamation | Can't handle circular references | | Mark-and-Sweep | Mark reachable objects, sweep unmarked ones | Handles cycles | Pauses program execution (STW) | | Generational | Objects are young or old; scan young more often | Efficient (most objects die young) | More complex | | Concurrent (Go, Java ZGC) | Run GC concurrently with program | Minimal pauses | CPU overhead |

Python's GC Example

import gc

# Enable debugging
# gc.set_debug(gc.DEBUG_LEAK)

# Create a circular reference (reference counting cannot handle this)
class Node:
    def __init__(self, name):
        self.name = name
        self.ref = None

a = Node("A")
b = Node("B")
a.ref = b
b.ref = a  # circular reference!

del a
del b

# Python's generational GC will collect the cycle
print(f"Unreachable objects collected: {gc.collect()}")

Practical Tips

| Problem | Likely Cause | Fix | |---------|-------------|-----| | Memory leak in long-running process | Objects held by global references | Review static/global containers | | OutOfMemoryError | Heap too small or leak | Increase heap size (-Xmx) or fix leak | | StackOverflowError | Deep recursion | Convert to iterative or increase stack size | | High GC overhead | Too many allocations | Object pooling, reduce allocation rate | | Slow memory allocation | External fragmentation | Use slab allocator or jemalloc |

Summary

Memory management is one of the most important responsibilities of an operating system and programming language runtime. Understanding the stack vs. heap distinction, virtual memory, paging, and garbage collection helps you write efficient, stable, and secure code.

Key takeaways:

Stack: fast, limited size, automatic allocation/deallocation, per-thread
Heap: flexible, large, manual or GC-managed, shared across threads
Virtual memory gives each process its own address space via page tables
Page faults occur when accessing data not in RAM — expensive!
External fragmentation: many small free blocks; internal fragmentation: wasted bytes within blocks
Garbage collection automates heap management but adds overhead and pauses
Reference counting can't handle cycles; generational GC is most efficient
Monitor memory usage in production to detect leaks early

What's Next: Networking & HTTP

The next chapter covers networking fundamentals — TCP/IP, DNS, HTTP protocols, and how the internet works under the hood.

Memory Hierarchy

Modern computers have multiple levels of memory, each with different speed and size:

| Level | Type | Size | Latency | Managed By | |-------|------|------|---------|-----------| | L1 Cache | CPU cache | 32-64 KB | ~1 ns | Hardware | | L2 Cache | CPU cache | 256-512 KB | ~4 ns | Hardware | | L3 Cache | Shared CPU cache | 2-32 MB | ~12 ns | Hardware | | RAM | Main memory | 8-64 GB | ~80 ns | OS + Hardware | | SSD | Persistent storage | 256 GB - 2 TB | ~100 μs | OS | | HDD | Persistent storage | 1-10 TB | ~10 ms | OS |

The gap between CPU speed and memory speed is the main performance bottleneck in modern computing.