series6 notebooks · 39 plates · about 139 minutes of reading
How chips compute
I’m building Miri, a GPU-first programming language, which means I spend a lot of time thinking about what the hardware does with the code it’s given. These notebooks are that thinking, drawn out with instruments you can poke.
There are six of them. Start with the CPU even if you came for the TPU: the GPU and the TPU are each a different bet on how to spend the same silicon, and the bets only make sense once you’ve seen the one they bet against. The last two leave the hardware for the software: the different ways to write a GPU program, and everything that happens to it on the way to the chip.
The reading path
-
What a CPU is, and how it works
A CPU is a chip of billions of transistor switches, driven by a clock. Over and over it fetches an instruction from memory, decodes it and executes it: add, compare, load, store, jump. Modern cores go fast by overlapping instructions in a pipeline, keeping hot data in caches and predicting which way branches will go. Everything from NANDCounting with wiresFetch, decode, executeThe assembly lineFar from homeThe cacheBetting on the future -
What a GPU is, and how it works
A GPU runs thousands of simple threads at once instead of a few fast ones. Threads move in groups of 32, called warps, that execute the same instruction together. While one warp waits for memory, the GPU switches to another, so the arithmetic units stay busy. That suits graphics, matrix maths and AI. A thousand slow brushesThirty-two in lockstepNever idleOne trip, not thirty-twoBring it in onceTensor coresGrid, block, warp, thread -
What a TPU is, and how it works
A TPU is Google’s chip for machine learning. Its core is a systolic array: a grid of multiply-add units that pass numbers to their neighbours on every clock tick, so each number is read from memory once and reused across the whole grid. It works in low-precision bfloat16, and the XLA compiler plans the work ahead of time. A chip for one equationThe systolic arrayGood-enough numbersFeeding the beastCompiled, not scheduledA pod is one computer -
CPU vs GPU vs TPU
A CPU is built for fast, branchy, one-thing-at-a-time work. A GPU handles thousands of identical operations in parallel, and a TPU handles the large matrix multiplications in neural networks. In a real system they work together: the CPU runs the program and feeds the data, while the accelerator does the heavy maths. Three bets on siliconSame job, three runnersThe rooflineOne step, three chipsThe cost of movingWhich one, when -
Four ways to program a GPU
GPU programming models differ in the unit you write code for. SIMD intrinsics describe one vector of lanes. SIMT kernels, in CUDA, HLSL or Metal, describe one thread. Tile languages like Triton describe one block of data. Array libraries like PyTorch and JAX describe the whole array. Each level up trades control for less code. Who counts the lanes?Who owns the widthPassing notes in the warpWrite for the tileWrite for the whole arrayPicking a level -
How GPU code reaches the chip
GPU code is usually compiled twice. Ahead of time, a compiler turns CUDA, HLSL, GLSL or Metal source into a portable intermediate form: PTX, SPIR-V, DXIL or Apple’s AIR. At run time, the GPU driver compiles that into machine code for the exact chip installed, because GPU instruction sets change with every hardware generation. Machine code that won’t sit stillOne kernel, three languagesRegisters decide the crowdWords all the way downApple’s roadCompiled while you playMany roads, a few hubs
Questions people ask
- What is the difference between a CPU, a GPU and a TPU?
A CPU spends its silicon on making one stream of instructions fast, with caches, branch prediction and out-of-order execution. A GPU spends it on thousands of arithmetic lanes that run in lockstep. A TPU goes further and lays out a fixed grid of multiply-add units for matrix maths.
- Why are GPUs and TPUs used for AI?
Neural networks are mostly matrix multiplication: the same multiply and add, repeated billions of times on independent numbers. Hardware that does thousands of those at once wins, and a TPU is built for little else.
- How do you program a GPU?
You pick a level. In CUDA, HLSL or Metal you write a kernel for one thread and launch thousands of them. In Triton you write for a tile of the data and the compiler handles the threads. In PyTorch or JAX you write whole-array maths and a compiler builds the kernels. The driver then compiles the result for the specific GPU.