Notes from building AI systems.
What we write here comes from delivery work: how a problem was scoped, what the evidence turned out to say, and what we would do differently next time.
The sample you did not get
A survey with 2.4 million responses called the 1936 US election wrong, and a survey with a few thousand called it right. Sample size was never the problem. Here is the arithmetic of what actually goes wrong when data selects itself.
What a p-value actually promises
A p-value of 0.03 does not mean there is a 3 percent chance you are wrong. Worked through with real numbers, the probability of being wrong can be above 30 percent while the p-value reads 0.03, and the gap between those two figures explains a great deal of published nonsense.
Correlation, causation and the third variable
Everyone can recite that correlation is not causation. Far fewer can say what to do about it. Here is the arithmetic of a confounded dataset, worked twice, where the same numbers support opposite conclusions depending on one column you were not shown.
Diffusion models and generation by denoising
Start with an image that is pure noise. Train a network to predict and remove the noise, one step at a time. That is how diffusion models generate images, and why they are slow but scalable.
Chain of thought and what it does not prove
Asking a language model to show its working gives it more tokens to use, and more tokens buy more computation. The steps it shows you are outputs, not evidence of how it computed the answer.
LoRA and fine-tuning without the bill
Fine-tuning a 175-billion-parameter model costs thousands of dollars in GPU memory and compute. LoRA reduces the trainable parameters by a factor of 10,000 by learning a low-rank update to the weights instead of updating them directly, and can merge the result back into the...
RLHF and how InstructGPT was taught to follow
A language model trained to predict text will refuse harmful requests and ignore explicit instructions indifferently. InstructGPT learns to follow instructions by training a reward model on human rankings of outputs, then using that as a signal to steer the model away from...
Scaling laws and the Chinchilla correction
For about two years the field spent its compute budget in the wrong shape, building models too large for the data they were given. The correction came from training over 400 models rather than from arguing, and the arithmetic that settles it fits on one line.
GPT-3 and learning from the prompt
A model can complete a task on a new dataset after seeing a few examples in its context, with no weight updates. This is what happened inside GPT-3, and it turned out to be mechanical.
BERT and the case for reading both ways
BERT broke from the left-to-right prediction chain by letting each token see its entire context. The trick costs everything that generating text requires, and that trade built the encoder-only branch.
Attention is all you need, what the paper actually says
The 2017 paper is eleven pages about machine translation. Its headline number was a BLEU score that almost nobody quotes now, and the thing it actually changed is visible in a table of complexity classes near the middle.
Governance that engineers can live with
An 84-question review arrives for a model that has been live for a year, and the team writes the answers from memory. Storing the evidence as it happens costs about seven gigabytes a year. Reconstructing it later costs three weeks and produces a worse document.
The cost model nobody writes down
The same feature costs Rs 5.1 lakh a month or Rs 77 lakh a month depending on one decision, and that decision is not which model you picked. It is how long you promised a user would wait.
Monitoring for silent failure
The drift alarm fired in week 8. The queue it was supposed to protect had gone over capacity in week 5, and by week 12 the team was 41,000 tickets behind. The statistic was watched carefully. It was the wrong statistic.
Deploying a model is not the finish line
The model shipped on a Thursday. The feature transformer it was trained with did not. Scores stayed plausible for nineteen days because a category encoding had shifted by one position, and nothing in the release was wrong enough to fail.
Experiment tracking for people who forget
Two engineers ran the same configuration and got 0.871 and 0.884. Neither was wrong. Across ten thousand random seeds on one small benchmark, the gap between the luckiest and the unluckiest run was 1.82 points, which is larger than most of the improvements teams celebrate.
Data pipelines that do not rot
An upstream team changed an amount column from rupees to paise. Every job stayed green, every type check passed, and the model's average order value moved by a factor of a hundred. The pipeline did not break. It kept running and started lying.
The first week of an AI project
A fraud team reviews 200 cases a day and cannot review more. A better model does not raise that ceiling; it changes which 200 cases sit under it. Finding the real constraint in week one is worth more than the next six weeks of modelling.
Putting a model behind an interface
A spinner for ninety seconds, then a red toast saying something went wrong. Forty support tickets in an hour. The model was fine. The interface had no way to say slow but working, no way to say partly done, and no way to stop.
Agents, tools, and the limits of autonomy
An agent that succeeds on 60 percent of tasks succeeds on the same task eight times running far less often than that. Measured retail reliability was under 25 percent at pass to the eighth. That gap, not the average, is what decides where a human belongs.
Hallucination is a design problem
Three citation chips under a paragraph. Two check out. The third links to a page that does not contain the claim, and the interface drew all three identically. Audited generative search engines support only 51.5 percent of their sentences with citations.
Evaluating a system that can say anything
The suite went from 71 to 76 percent on Friday and support tickets doubled on Monday. Sixty questions, written by the person who wrote the prompt, graded by exact match. Here is what to measure instead, and what a benchmark score is actually evidence of.
Fine-tuning against prompting, a real comparison
Four hundred labelled tickets, a 900-token prompt, and a meeting that cannot decide. The choice is not about capability. It is about what changes, what it costs per request, and who maintains it in a year.
Retrieval-augmented generation, honestly
The demo answered every question in the first meeting. Six weeks later it confidently cited a policy that was superseded in 2023. Retrieval fixes one specific problem, and the failures that remain are mostly failures of ranking, chunking, and position.
Embeddings are coordinates for meaning
Cosine similarity of 0.83 between two support tickets that share no words, and 0.79 between two that mean opposite things. The vector space is real and useful. The metaphor that explains it is wrong in three specific places.
Language models predict the next token
A support ticket goes in, an order number comes out, and the order number does not exist. The objective explains it. A model scores continuations, and a plausible continuation is not a true one.
Training a network without wasting a month
The progress bar says twenty six days. Before accepting that, work out how many steps the run actually needs, what the batch size is buying, and what evidence would let you stop on day three instead.
The transformer block piece by piece
Six stages, 3.15 million parameters at a width of 512, and two thirds of them in the part nobody talks about. The cost of attention only overtakes the rest past about three thousand tokens, and the arithmetic says where.
Attention is a lookup you can learn
Four words, two dimensions, sixteen dot products and one softmax. The word "bank" starts out equally financial and aquatic and comes out decisively aquatic, and every step of that is arithmetic you can check.
Sequences, recurrence and its limits
A factor of 0.9 repeated fifty times is 0.005 and a factor of 1.1 repeated fifty times is 117. Recurrent networks live between those two numbers, and everything built to rescue them is an attempt to get the factor to one.
Convolutions and what a filter sees
Nine weights and one bias replace four hundred and eighty three billion. The arithmetic of one kernel over one small patch shows where the saving comes from and what it quietly assumes about your data.
Why activation functions matter
A sigmoid at a moderate input learns about a hundred times slower than one at zero, and ten of them in a row cut the gradient by a factor of a million. The arithmetic explains most of what happened to activation functions since.
Backpropagation worked by hand
One forward pass, one backward pass, nine weight gradients and a numerical check, on a network small enough to compute on paper. The step that follows overshoots and kills a unit, which is also worth watching.
A neuron is a weighted sum and a decision
A single unit is a linear model with a switch on the end. Stacking them buys nothing until the switch is there, which a four-row truth table proves in about six lines of arithmetic.
Dimensionality reduction without the hand waving
PCA is variance and nothing else, worked through on five points where the arithmetic closes exactly. Then what t-SNE and UMAP actually preserve, and the three things a reader of those pictures is entitled to conclude.
Clustering when nobody labelled anything
k-means run by hand on eight numbers, why its score always improves when you ask for more clusters, and two shapes of data where it returns a confident answer that is the wrong grouping.
Support vector machines and the margin
Among the many boundaries that separate two classes, one leaves the most clear space. Computing that margin by hand, watching a kernel turn an impossible problem into an easy one, and the costs that keep the method off large datasets.
Gradient boosting explained by its mistakes
Each round fits a small tree to what is still wrong and adds a fraction of it. Three rounds worked through on six houses, what the learning rate is really buying, and why the method chases a mislabelled row until it believes it.
Random forests and why averaging works
Averaging unstable trees removes variance, and the arithmetic says exactly how much: the correlation between the trees sets a floor that no number of extra trees goes below. What sits under that floor is everything a forest cannot fix.
Decision trees and how they split
Entropy and information gain worked by hand on twelve rows, why the algorithm picks one question over another, and why a tree left to grow ends up writing rules from two examples.
Logistic regression and decision boundaries
Classification is geometry. A straight line drawn through a feature space, coefficients that multiply odds rather than add percentages, and a threshold that slides the line without ever turning it.
Linear regression is still the baseline
The simplest model is the one every other model has to beat. A line fitted by hand on six shops, what its two numbers do and do not license you to say, and how to read the margin when something bigger wins.
Reading a model you did not build
Inheriting a system in production, from a vendor or a colleague who left. What to ask for, what to test when nobody answers, and how to decide whether to keep running it.
The cost of a wrong answer
Precision against recall is not a technical preference. It is a budget decision made in units of money and harm, and the threshold is where the business states its position.
What probability buys you
A score of 0.9 is a claim about the world that can be checked. Calibration, why modern models are confidently wrong, and how to measure the gap before it costs something.
Bias and variance in plain terms
Not a formula to memorise but a diagnostic to run. How to tell a model that cannot see the pattern from one that is following the sample, and what each one costs to fix.
Train, test, and the lie of a single score
One accuracy number is an estimate with an error bar nobody printed. How splits work, what cross-validation buys, and why the gap you are celebrating may be a rounding difference.
How a dataset becomes features
A model never sees your data. It sees a matrix of numbers somebody chose. Encoding, scaling, and why the representation decides more than the algorithm.
The data comes first
Why data quality decides more projects than model choice: how leakage gets in, how labels go wrong, and what a dataset has to look like to be usable.
What a model actually learns
Fitting parameters to reduce a loss is not understanding. A worked example, the three shapes a fit can take, and what generalisation really costs.
Read a series in order
Each series builds on the one before it, and each article on the one before that. Start at part 1 of whichever is closest to what you are working on.
Foundations
- Part 1. What a model actually learns
- Part 2. The data comes first
- Part 3. How a dataset becomes features
- Part 4. Train, test, and the lie of a single score
- Part 5. Bias and variance in plain terms
- Part 6. What probability buys you
- Part 7. The cost of a wrong answer
- Part 8. Reading a model you did not build
Core machine learning
- Part 1. Linear regression is still the baseline
- Part 2. Logistic regression and decision boundaries
- Part 3. Decision trees and how they split
- Part 4. Random forests and why averaging works
- Part 5. Gradient boosting explained by its mistakes
- Part 6. Support vector machines and the margin
- Part 7. Clustering when nobody labelled anything
- Part 8. Dimensionality reduction without the hand waving
Deep learning
- Part 1. A neuron is a weighted sum and a decision
- Part 2. Backpropagation worked by hand
- Part 3. Why activation functions matter
- Part 4. Convolutions and what a filter sees
- Part 5. Sequences, recurrence and its limits
- Part 6. Attention is a lookup you can learn
- Part 7. The transformer block piece by piece
- Part 8. Training a network without wasting a month
Applied AI
- Part 1. Language models predict the next token
- Part 2. Embeddings are coordinates for meaning
- Part 3. Retrieval-augmented generation, honestly
- Part 4. Fine-tuning against prompting, a real comparison
- Part 5. Evaluating a system that can say anything
- Part 6. Hallucination is a design problem
- Part 7. Agents, tools, and the limits of autonomy
- Part 8. Putting a model behind an interface
Practice
Generative AI papers
- Part 1. Attention is all you need, what the paper actually says
- Part 2. BERT and the case for reading both ways
- Part 3. GPT-3 and learning from the prompt
- Part 4. Scaling laws and the Chinchilla correction
- Part 5. RLHF and how InstructGPT was taught to follow
- Part 6. LoRA and fine-tuning without the bill
- Part 7. Chain of thought and what it does not prove
- Part 8. Diffusion models and generation by denoising
Statistics for data science
Prefer to talk it through?
If you have a problem you would like thought about carefully rather than pitched at, bring it to a call.