Token

You read words. AI reads tokens. A token is roughly a piece of a word — sometimes a whole word, sometimes just a chunk. The word "hamburger" might be two tokens: "ham" and "burger." The word "the" is one token. A long word like "unbelievable" might be three or four tokens.

Why does this matter? Because AI has a limit on how many tokens it can handle at once. It's like a conveyor belt with a maximum capacity. If you give it a really long document, it might not be able to read the whole thing because it runs out of token space. That limit is called the "context window."

Also, tokens are how AI companies charge you. When you use the API (the programmer version of ChatGPT), you pay per token. More tokens in your question = more money. More tokens in the answer = more money. It's like being charged per word at the world's most expensive telegraph office.

Because tokens explain two things that will confuse you otherwise: why AI sometimes "forgets" what you told it earlier in a long conversation (it ran out of token space), and why AI services cost what they cost (you're paying per token).

Tokens are produced by a tokenizer (like BPE — Byte Pair Encoding) that breaks text into subword units. GPT-4 uses roughly 1 token per 4 characters of English text. Context windows range from 4K to 128K+ tokens depending on the model. Token costs for GPT-4 are roughly $0.03/1K input tokens and $0.06/1K output tokens.