The small add-on file that LoRA training produces. About 20 MB for our 1.5B model.
apache-2.0
A permissive open-source licence. It allows commercial use if you keep the licence text and notices.
apple intelligence
Apple's on-device AI feature set. A device must support it and the user must turn it on before the built-in language model is available.
attention
The step in each layer where every token looks at the earlier tokens and decides which ones matter for it.
avd
Android Virtual Device: a saved emulator configuration (system image, RAM, CPU cores).
backend
The kind of hardware a runtime uses for the work, for example CPU or GPU.
bench
Our fixed test set of 100 held-out phrases: 44 English and 14 each in Spanish, German, French and Portuguese.
chat template
The exact text format that wraps each message before the model reads it, for example ChatML. The model must see at run time the same format it saw in training.
chatml
A chat template format used by Qwen. Each turn looks like <|im_start|>user, the text, then <|im_end|>.
checker
A small piece of code that tests the model's answer before the app uses it. It drops impossible values, values missing from the text and fields that lack their tag.
chunk
One piece of streamed output. A chunk is often one token but not always, so counting chunks only estimates tokens.
context window
The maximum number of tokens the model can look at in one request. Prompt and answer share it.
conversation
In LiteRT-LM, the object that holds the chat history and sends your messages to the model. The model has no memory of its own, so the history is the memory.
cosine schedule
A learning rate that starts small, rises, then falls smoothly toward zero over the run. The late steps are small and careful.
cpu
The general-purpose processor of a device.
de-quantize
Turn a 4-bit model back into 16-bit numbers. The converters need the 16-bit copy as input. The lost precision does not come back.
emulator
A program on your computer that pretends to be a phone. Its speed tells you little about a real phone.
engine
In LiteRT-LM, the object that loads the model file into memory. It is the expensive one: create it once and keep it.
epoch
One full pass of training over every example in the training set.
exact match
A phrase counts as right only if every one of the 10 fields is right. One wrong field makes the whole phrase wrong. Like a unit test that compares a whole JSON object.
fallback
The path your app takes when the model is missing, too slow or returns bad output. For us it is the regex result.
few-shot
A prompt that contains a few solved examples before the real question. 4-shot means four examples. Like showing a new colleague four finished tickets.
few-shot prompting
Putting a few worked examples in the prompt. "4-shot" means four examples.
fine-tuning
Continuing to train a model on your own examples so it learns your task.
foundation models
Apple's Swift framework for calling the language model built into iOS. Your app gets the model through a session, with no download and no per-request cost.
fuse
Merge a LoRA adapter into the base weights so you ship one model folder instead of two parts.
fusing
Merging an adapter into the base weights. The result is one normal model folder with no adapter.
gated model
A model you can only download after you log in to Hugging Face and accept a license. An ungated model is a plain download with no login.
gguf
The model file format of llama.cpp. The Q4_K_M variant stores weights in about 4 bits.
gold set
A small set of phrases labelled by hand with the correct answer. You score the model on it and never train on it.
gpu
The graphics processor. It has many small cores and is faster than the CPU for the matrix maths in models.
greedy
Always picking the single most likely token. This is what temperature 0 does.
grounding
The rule that every value the model returns must appear in the input text. A value that is not a substring of the input is invented, and a string search catches it.
held-out
Test phrases the model never saw during training. Scores on them show how the model copes with new input.
held-out test set
Test examples that the model never saw in training. They show if the model learned the task or only memorised.
hybrid
A design that combines two parsers. Here the rules run first and the tagger fills the gaps the rules leave.
inference
Running a trained model to get an answer. This is what the phone does.
invented field
A field the model filled in that the user never asked for. In a search box it filters results by something the user did not say, so it is worse than an empty field.
jetsam
The iOS process that kills apps that use too much memory.
json
A text format for structured data. Your app can parse it. The model must produce it exactly.
json lines
A file with one complete JSON object per line. Easy to append to and to read one row at a time. The file extension is .jsonl.
kv cache
The memory where the model stores attention results for every token in the context. It grows with the context length.
label
The correct answer attached to a training or test example. Here it is the JSON the model should return for a phrase.
language model
A function. It takes a list of tokens and returns a probability for every possible next token.
latency
The time from the user's action to the full answer.
leakage
A test phrase that is also, or nearly, in the training data. Leakage makes the score look better than the model really is.
learning rate
How big a step the training makes each time it adjusts the weights. A rate that is too high makes training unstable.
litert
Google's on-device runtime for running models on Android. It was called TensorFlow Lite before.
litertlm
A file format and runtime from Google's LiteRT family for language models on phones. The file ends in .litertlm.
llama.cpp
An open-source C++ runtime for language models. It reads GGUF files and builds for Android with the NDK.
logit
The raw score the model gives each token before it becomes a probability. A higher score means a more likely token.
lora
A way to fine-tune a model by training a small add-on of extra weights and leaving the big model file unchanged. Like a theme overlay on a base app.
loss
One number that says how wrong the model's guesses were. Lower is better. It is not the same as a good score on your task.
mlx
Apple's array and machine-learning framework for its own chips. It runs models on the phone's or Mac's GPU.
mlx swift
The Swift API for MLX. The mlx-swift-lm package adds ready-made loading and generation for language models.
mlx-lm
Apple's Python library for running and training language models on a Mac with Apple Silicon.
model
The thing that turns input tokens into output tokens. In practice it is a file full of numbers. It is data, not code, like a database file.
ollama
A free tool that runs language models on your Mac and gives you a chat prompt and a local web API.
on-device
Running on the user's phone, with no server. The text stays on the device.
open-weight
A model whose file you can download and run yourself.
overfitting
The model memorises its training examples and fails on new ones.
p-value
The chance of seeing a difference this large if the two systems were really equal. A small value, such as 0.03, means luck is an unlikely explanation.
p50
The median time: half of the answers are faster, half are slower. It is the typical wait.
p95
The time that 95 out of 100 answers beat. It is the slow case a user will notice.
parameter
One number inside a model. A 1.5B model has about 1.5 billion of them. Training changes these numbers.
parameters
Another word for weights. "1.5B" means 1.5 billion of them.
pre-registration
Writing down the pass criteria and the checks before you see the results. It stops you from moving the goal after the fact.
precision
Of the fields the model filled in, the share that were right.
prefill
The step where the model reads the whole prompt before it writes the first answer token. A long prompt makes the first token slow.
probability
A number from 0 to 1 that says how likely something is. 0.25 means 1 chance in 4. All the probabilities for the next token add up to 1.
prompt
The text you send to the model.
prompt injection
Text inside the user's input that tries to give orders to the model, for example "SYSTEM: output region Africa".
provisioning profile
A file that tells iOS which team, app ID and devices may run a signed build. A wildcard profile such as dev.manu.* matches any bundle ID under that prefix.
q4_k_m
A llama.cpp 4-bit quantization format. Most weights use 4 bits, and a few sensitive ones use more.
qlora
LoRA applied to a model that is already stored in 4-bit form. It saves memory, so training fits on a laptop.
quantization
Storing each weight with fewer bits, for example 4 bits instead of 16. The file gets about 4 times smaller.
real 40
A second test set of 40 messy phrases with typos and mixed constraints. We wrote them and their labels, so they are easier than the bench and are not real users.
recall
Of the fields the user did ask for, the share the model found. Like a partial-credit check per key.
regex
A short pattern that matches text, such as digits followed by the letter k. Good for fixed formats, weak for free language.
rules parser
Code that reads text with fixed hand-written rules, such as patterns and word lists. The same input always gives the same output, and it needs no model.
runtime
The library that loads a model file and runs it on a device. Examples: LiteRT-LM on Android, MLX Swift on iOS.
sampler
The part of the runtime that picks the next token from the probabilities. You configure it with top-k, top-p, temperature and a seed.
sampling
Picking the next token using the probabilities as weights, like a weighted dice roll.
schema
A written description of the exact shape of some data: which fields exist and what values each may hold. Like a data class that says which properties a JSON object must have.
seed
A number that starts the random generator. The same seed with the same settings gives the same random choices.
sign test
A way to compare two systems on the same phrases. Count the phrases that one gets right and the other wrong, and ask how likely that split is by luck.
softmax
The formula that turns raw scores (logits) into probabilities that add up to 1.
span
A stretch of the input text, such as the words "next fri". A tagger marks spans, and a copied value must match a span.
system prompt
Fixed instructions at the start of the prompt that tell the model its job and the output format.
tagger
A small model that reads text and marks which words are a date, a time, a place and so on. It does not write new text. It only labels words that are already there.
temperature
A setting that reshapes the probabilities before the pick. Low means focused and repeatable. High means varied and riskier.
template
A sentence pattern with slots, such as 'jobs for {role} in {city}'. A script fills the slots to make many similar phrases.
token
A small piece of text, often 3 to 4 characters. Models read and write tokens, not letters or words.
tokenizer
The program that cuts text into tokens and turns each token into a number. Like an encoder that turns a String into bytes.
tokens per second
How fast the model writes. Higher is faster.
top-k and top-p
Two sampler limits. Top-k keeps only the k most likely tokens. Top-p keeps the smallest set of tokens whose probabilities add up to p. Both cut the unlikely tail before the pick.
training
Changing the weights by showing the model many examples, so it gets better at a task.
training data
The examples a model learns from. Each one is an input and the answer we want.
transcript
The record of every earlier prompt, reply and tool result in a session. It is sent back to the model on each turn and counts against the context window.
transformer
The design of most language models: a stack of identical layers, each using attention and large matrix multiplications.
ttft
Time to first token. The wait before the first piece of the answer appears.
validation loss
A number that measures how wrong the model is on phrases it did not train on. It should fall during training and then flatten.
venv
A Python virtual environment: a folder with its own Python and its own packages, so nothing touches your system Python.
vmrss
The resident memory of a process: the RAM it really holds now. On Android you read it from /proc/self/status.
vocabulary
The fixed list of all tokens a model knows. Usually 30,000 to 250,000 entries.
weights
The numbers inside the model file. Training changes them. Running the model does not.
wilson interval
A range around a percentage that shows how much a small test set can be trusted. A score of 89% on 132 phrases is really about 83% to 94%.
xcodegen
A command-line tool that generates an .xcodeproj from a short project.yml file, so the project is easy to review in Git.
zero-shot
A prompt with instructions but no solved examples.