What is a Token (in LLMs)?
A token is the chunk of text a language model actually processes: a word, part of a word, or punctuation. As a rule of thumb, 1,000 tokens is about 750 English words. Model context windows, API pricing, and answer length limits are all counted in tokens.
Why a marketing team should care
Retrieval-backed assistants pull page excerpts into a token budget before answering. Long, meandering pages waste that budget; tight sections with the answer up front fit more usable evidence into the window. Writing for token economy is the same discipline as writing a liftable answer.
Related measures
- Context window: the maximum tokens a model can consider at once
- Chunking: splitting content into retrieval-sized token blocks
- Output cap: why AI answers cut off and cite only a few sources