A register read costs about one cycle while a trip out to disk costs on the order of a hundred thousand, five orders of magnitude apart, and the whole cache tower exists only because of that gap: L1 around 4 cycles, L2 around 12, L3 around 40, RAM around 200. A cache never fetches a lone byte but pulls in a whole 64-byte line, so walking an array in order lets one miss pay in advance for many hits right behind it. A hot loop working over a small set hits almost every time, whereas addresses scattered randomly across memory miss almost every time and the average latency creeps up toward RAM speed.
A register read costs about one cycle while a trip out to disk costs on the order of a hundred thousand, five orders of magnitude apart, and the whole cache tower exists only because of that gap: L1 around 4 cycles, L2 around 12, L3 around 40, RAM around 200. A cache never fetches a lone byte but pulls in a whole 64-byte line, so walking an array in order lets one miss pay in advance for many hits right behind it. A hot loop working over a small set hits almost every time, whereas addresses scattered randomly across memory miss almost every time and the average latency creeps up toward RAM speed.
Related topics in Mind & Machine