Understanding AI Under the Hood

Most software is deterministic. Same input, same output. AI models are probabilistic. Same input, different output. They learn patterns from data instead of following explicit rules.

Here’s the stripped-down version of how they work.

Neural networks

Layers of interconnected neurons that process information:

• Input layer — text becomes numbers (tokens)
• Hidden layers — apply weights, pass through activation functions, learn patterns
• Output layer — produces predictions (e.g. next word)

It’s math. Linear algebra, calculus, statistics. Frameworks like PyTorch abstract it, but that’s what’s happening.

Tokenization

Text must become numbers before a model can process it. Tokenization breaks text into units (tokens) and maps each to a number. Models use subword methods like Byte Pair Encoding — “unbelievable” might become ["un", "believ", "able"]. That lets them handle words they’ve never seen.

Pretraining

Models are trained on huge amounts of text. The process:

1. Forward pass — input flows through the network, produces a prediction
2. Loss — measure how wrong the prediction is
3. Backpropagation — compute gradients (how to adjust each weight)
4. Gradient descent — update weights to reduce loss

Repeat over many epochs. The network gets better at predicting the next token.

Transformers and attention

Transformers use self-attention: they consider the entire input sequence at once, not word-by-word. Words become vectors (embeddings); similar words have similar vectors. Attention lets the model connect “it” to “cat” in “The cat sat on the mat because it was tired.”

Fine-tuning and RLHF

After pretraining, models get fine-tuned on smaller, high-quality datasets for specific tasks. RLHF (Reinforcement Learning from Human Feedback) aligns them with human preferences: humans rank outputs, a reward model learns from that, the main model is tuned to maximize the reward.

Inference

At runtime, the model predicts from new inputs. Temperature controls randomness: low = deterministic, high = creative. Chain of thought — generating reasoning steps before the answer — improves complex tasks.

Distillation and evals

Distillation trains a smaller “student” model to mimic a larger “teacher.” You trade some quality for speed and cost. Evals are tests for AI: measure what matters before you ship.

The takeaway

LLMs are pattern-matching systems built from math. They’re trained on data, fine-tuned for tasks, and evaluated like any other system. Understanding the basics helps you use them better — and know when to reach for RAG, fine-tuning, or something simpler.

---

Need help applying this? Talk to us.

Inspired by Understanding AI by Lee Robinson — condensed for our style.