Slavik Shynkarenko

series3 notebooks · 18 plates · about 45 minutes of reading

How LLMs and agents work

I build AI agents, and the most useful thing I can tell someone starting out is that the loop holds no magic. It’s a model, a prompt, a handful of tools and careful bookkeeping of what goes into the context window.

Three notebooks, in order: the model, then the agent wrapped around it, then the memory that agent pretends to have. Each one takes about a quarter of an hour, longer if you play with the instruments.

start with № 06: How a large language model works →

The reading path

  1. № 066 plates · 15 min

    How a large language model works

    A large language model is a neural network trained to predict the next token, a word or piece of a word, from the text before it. It turns text into tokens and tokens into vectors, uses attention to decide which earlier words matter, then picks the next token from a probability distribution. Repeating that one step writes whole answers. Words into numbersOne job: the next wordA model you can trainMeaning as directionPaying attentionThe context window
  2. № 076 plates · 15 min

    Anatomy of an AI agent

    An AI agent is a language model running in a loop with tools. On each turn the model reads its context, decides on an action, calls a tool such as search, code execution or a file edit, and reads the result back into its context. It repeats until the task is done, within limits set by guardrails. A mind without handsThe loopTools are a contractA desk that fills upShapes of an agentKnowing when to stop
  3. № 086 plates · 15 min

    How agents remember

    An AI agent remembers by putting text back into the model’s context window. Short-term memory is the conversation itself. Long-term memory is stored outside the model, often as embeddings in a vector database, and retrieval-augmented generation (RAG) searches it by meaning and pastes the best matches into the prompt before the model answers. A brilliant goldfishCutting the book into cardsMeaning as distanceFinding the right cardThe open-book examWhat’s worth remembering

Questions people ask

What is the difference between an LLM and an AI agent?

An LLM takes text in and predicts text out, once. An agent runs an LLM in a loop with tools: the model picks an action, the program carries it out, and the result goes into the next prompt until the task is done.

How do AI agents remember things?

Only through the context window. Anything an agent should remember has to be stored somewhere, then retrieved and put back into the prompt, either as a summary or as chunks found by vector search. That second route is what retrieval-augmented generation means.

What is a context window?

The most tokens a model can read at once: the system prompt, the tool definitions, the conversation and any retrieved documents together. Whatever falls outside it is invisible to the model on that turn.

Other series

6 notebooks How chips compute One CPU core, then thousands of GPU lanes, then a TPU’s grid of multipliers, how the three split the work, and how you program a GPU and compile for it. 2 notebooks How code becomes a program What a compiler does between your source file and the machine, and the different ways languages clean up after themselves. 7 notebooks What the cloud costs FinOps from the start: the framework, the bill, who pays for what, how to spend less and how to see it coming. Roughly the ground of the FinOps Certified Practitioner exam. 6 notebooks What AI costs The same discipline for tokens, GPUs and seats: how AI is billed, split, forecast, cut and judged. Roughly the ground of FinOps Certified: AI Value.

← all notebooks