Notes on AI engineering
Mechanisms, patterns, references, a glossary, and reproducible labs, from the fundamentals through whatever building real AI systems requires.
Premises
- A deployed system is a protocol: tools, context, evaluation, and recorded failure modes.
- Vendor documentation describes mechanisms. The work is to implement them so that each hop can be inspected.
- If a step cannot be traced, it cannot be measured or corrected.
Foundation
Seven building blocks to start from
01
Vector representation
Meaning is represented as geometry in a vector space.
Read
02
Function approximation
A model is a parameterized function fitted by optimization.
Read
03
Attention and transformers
Attention is the data structure of context; its cost scales with window length.
Read
04
Probability and uncertainty
The model outputs a probability distribution over tokens.
Read
05
Context as the interface
The context window is the interface: its contents determine the next step.
Read
06
Evaluation
Iteration requires a measurement procedure.
Read
07
The agentic loop
An agent is a loop: perceive, decide, act, observe, under a resource bound.
Read
Experiments
In-browser labs
Three implementations of the corresponding primitives: the agent loop, the context budget, and an evaluation harness.