Lesson 7 · Part 1. How language models work · 8 min
Weights and training
Goal. Describe what is inside a model in one paragraph, and explain why training changes weights but running a model does not.
You do not need to write any of the maths in this lesson. You will load a file and call it. But a short picture of the inside explains why a model is a file, and why changing its behaviour means training.
The transformer in one paragraph
Inside the function there is a stack of identical layers. This design is called a transformer. In each layer, every token "looks at" the earlier tokens and decides which ones matter for it. This step is called attention. Then the layer transforms the result with large matrix multiplications. A matrix multiplication is a lot of multiply-and-add operations on tables of numbers. The numbers inside these tables are the model's parameters, also called weights. A model called "1.5B" has 1.5 billion of them. After the last layer, the model turns the result into one probability per token.
That is all you need for now.
Note Attention is like a lookup over the text so far. Each new token asks "which earlier tokens help me?" and weighs them.
Training changes weights, running does not
training is the process that sets the weights. The model sees a huge amount of text. For each piece it tries to predict the next token. When it is wrong, a small adjustment is made to the weights. After billions of such steps the model predicts well.
Running a model, called inference, never changes the weights. The weights are read-only. This has two consequences for you.
- A model file is like a database file or an image asset. It is data, not code. You can copy it, ship it, and delete it.
- If you want a different behaviour, you must train again. The result is a new file, or a small add-on file next to the old one. That is what fine-tuning does, and you will do it in Part 2.
| Training | Inference | |
|---|---|---|
| What it does | Changes the weights | Reads the weights |
| Where it happens in this course | On the Mac | On the phone |
| Needs | A lot of memory and time | Much less |
| Result | A new file or adapter | One answer |
Training itself can be short for a small job. One of our 1.5B fine-tuning runs took about 23 minutes on a Mac. You will see it in lesson 17.
Why this matters for an app
Think of the model file as a bundled resource. Your app does not edit it at runtime. The phone only runs inference. All the learning happened before the file was built. That is why a model on a phone does not get smarter as the user types, and why it cannot learn a fact you tell it in one chat once the chat is over.
Check yourself
What do the numbers in "1.5B" count?
A 1.5B model has about 1.5 billion weights.
Does running a model on a phone change its weights?
Only training changes weights. Inference reads them.
You want the model to behave differently. What must you do?
Behaviour lives in the weights, and only training changes them.
Recap
- A transformer is a stack of layers. Attention lets each token look at earlier ones. The tables of numbers inside are the weights.
- Training sets the weights. Inference only reads them.
- A model file is data, like a database file, so you can copy and ship it.
- Next lesson: quantization, which explains why a phone has room for these files.