Memory Hierarchy
Why does your computer have different types of memory? The answer lies in the fundamental trade-off between speed, capacity, and cost. Memory hierarchy organizes different types of memory to give us the best of all worlds.
The Memory Pyramid
Memory Hierarchy Pyramid
+---------------------------------------------+
| |
| /\ |
| / \ |
| /Registers\ |
| / (Fastest, \ |
| / Smallest) \ |
| /--------------\ |
| / L1 Cache \ |
| / (Very Fast, \ |
| / Small) \ |
| /----------------------\ |
| / L2 Cache \ |
| / (Fast, Medium) \ |
| /----------------------------\ |
| / L3 Cache (Shared, Larger) \ |
|/----------------------------------\ |
| Main Memory (RAM) \ |
| (Moderate Speed, Large) \ |
|----------------------------------------\ |
| Secondary Storage (SSD/HDD) \ |
| (Slow, Very Large) \ |
|------------------------------------------\ |
| Tertiary Storage (Tape, Cloud) \ |
| (Slowest, Largest) \|
+---------------------------------------------+
Why Hierarchy Exists
The reason for this hierarchy is simple economics and physics:
- Speed: Faster memory is more expensive per byte
- Capacity: Larger memory is slower
- Cost: We can't afford to make all memory fast
By organizing memory in layers, we get:
- Fast access to frequently used data (in small, fast memory)
- Large capacity for all data (in slow, large memory)
- Reasonable cost (by using small amounts of expensive memory)
Memory Characteristics
Memory Type Comparison
+---------------------------------------------+
| Type | Speed | Size | Cost |
|-----------|------------|---------|----------|
| Registers | 1 cycle | KB | $$$$$ |
| L1 Cache | 2-4 cycles | 32-64KB | $$$$ |
| L2 Cache | 10 cycles | 256KB-1MB| $$$ |
| L3 Cache | 20-30 cycles| 2-8MB | $$ |
| Main Memory| 100+ cycles| GB | $ |
| SSD | 10K+ cycles| TB | $ |
| HDD | 10M+ cycles| TB+ | $ |
+---------------------------------------------+
The Principle of Locality
Memory hierarchy works because of locality - programs tend to access the same data repeatedly (temporal locality) or access data near recently accessed data (spatial locality). This means we can keep recently used data in fast memory and it's likely to be needed again soon.