Cache Memory
The memory hierarchy
| Level | Memory | Speed | Size |
|---|---|---|---|
| Fastest | CPU registers | Instant | Bytes |
| L1 cache | Very fast | KB | |
| L2 cache | Fast | MB | |
| L3 cache | Decent | Larger MB | |
| RAM | Slower | GB | |
| Slowest | Hard disk | Very slow | TB |
Higher up means faster, smaller and costlier; lower down means slower, bigger and cheaper.
What cache does
Cache is a small, very fast memory placed between the CPU and RAM. The CPU is far quicker than RAM, so cache keeps frequently used data close to the CPU and avoids waiting. Picture a chef who keeps salt and oil beside the stove (cache) and fetches rare items from the pantry (RAM).
Why it works: locality
- Temporal locality: data used just now is likely to be used again soon (a loop counter).
- Spatial locality: data next to what was just used is likely to be needed (reading an array in order).
Hit and miss
- Cache hit: the data is in cache. It arrives in a few clock cycles.
- Cache miss: the data is not in cache, so the CPU goes to RAM, which takes many times longer.
Hit rate = (cache hits ÷ total memory accesses) × 100
For example, 950 hits in 1,000 accesses is a 95% hit rate, so the miss rate is 5%. Modern CPUs reach about 90 to 99%.
L1, L2 and L3
| Level | Features |
|---|---|
| L1 | Smallest and fastest. Inside each CPU core, one set per core. |
| L2 | Larger and a little slower. Still very close to the core. |
| L3 | Largest. Shared by all cores; the last safety net before RAM. |
Search order: L1 → L2 → L3 → RAM. On a miss, the data fetched from RAM is also copied into cache for next time. Think: L1 is your pocket, L2 your bag, L3 the cupboard, RAM the shop down the road.
Ways of accessing cache
| Method | How it works | Best for |
|---|---|---|
| Hierarchical | Cache first; RAM only on a miss | Most modern CPUs |
| Simultaneous | Cache and RAM are asked together; the first answer wins | Low-latency systems |
| Look-aside | A form of simultaneous access; on a miss the RAM data is already on its way | High-performance designs |
Write policies
| Feature | Write through | Write back |
|---|---|---|
| When RAM is updated | Every write, immediately | Only when the cache block is replaced |
| Speed | Slower | Faster |
| Safety | Very safe: RAM always has the latest copy | Needs a dirty bit |
| Used in | Simple and embedded systems | Modern CPU caches (usually L1 and L2) |
A dirty bit is a flag on a cache block meaning "modified, RAM does not have this yet". Analogy: write through is editing a document and saving to a USB drive at the same moment; write back is saving to the USB only when you finish (faster, but you lose work if the computer crashes first).
Keep practising
Notes are only half the work. Test yourself with questions and discuss doubts with other aspirants.
Prepared for exam revision from standard computer textbooks, Microsoft documentation and official sources. Features differ between Excel and Word versions (2016, 2019, 2021, 365) so check which version your exam follows. Found a mistake? Tell us.