Lesson 3 · Part 1. How language models work · 6 min

What a language model is

Goal. Explain a language model in one sentence, and say why every app feature built on one is a loop.

You build apps. You know functions: input in, output out. A language model is a function too. Once you hold that picture, the rest of the course is easier.

The one-sentence model

A language model takes a list of tokens. It returns a probability for every possible next token.

That is all. It does not "understand" your question. It scores which small piece of text is most likely to come next.

Take the text The capital of France is. The model might return something like this.

All the probabilities add up to 1. A program then picks one token. That pick is called sampling.

Writing text is a loop

One pick gives one token. To write a sentence, the program repeats the same steps.

1. Give the model the tokens so far.
2. Get probabilities for the next token.
3. Pick one token.
4. Add it to the text. Go back to step 1.

The loop stops when the model picks a special "end" token, or when you hit a size limit.

Note Think of a while loop that appends to a StringBuilder. The model is the function you call inside the loop. Your app owns the loop, so your app decides when to stop.

Why this matters for an app

Three things follow from "a function in a loop".

• Speed is per token. Ten tokens means ten trips through the model. A long answer takes longer.
• The text can change between runs. Step 3 can pick differently each time. Lesson 5 shows how you control that.
• The model has a size limit. It can only look at a fixed number of tokens at once.

Check yourself

Recap

• A language model is a function: tokens in, next-token probabilities out.
• Text comes from a loop: score, pick, append, repeat.
• Next lesson: what a token is, and why token count decides speed and memory.