Understanding LLMs: What's Actually Happening Under the Hood

Large language models feel like magic until you see the shape of what they're doing: predicting the next token, over and over, guided by everything that came before it.

Tokens, not words

Models don't read words — they read tokens, small chunks of text that might be a whole word, part of one, or a single punctuation mark. Understanding this explains a lot of odd LLM behavior, from weird spelling mistakes to why longer prompts cost more.

Context windows

Every model has a context window — the amount of text it can "see" at once. In our cohorts, we spend real time on how to work within that limit: what to include, what to summarize, and when to reach for retrieval instead of stuffing everything into the prompt.

Why this matters for builders

You don't need to train a model to build with one well. Knowing how tokenization, context, and sampling work is what separates prompts that work by luck from systems that work by design — which is exactly what we build toward in the LLMs module.