Token (AI)

A token is the smallest unit a language model breaks text into, usually a word fragment, a short whole word, or a punctuation mark. AI models process language token by token. The number of tokens partly determines how much an AI request costs and how much text a model can process at once.

What is a token?

Before a Large Language Model processes text, it breaks it into tokens. A token is a language building block, often a word fragment. As a rule of thumb, one token in English corresponds to roughly four characters or about three quarters of a word; longer and compound words tend to be split into more tokens. A long compound word can therefore consist of several tokens, depending on the model.

The AI reads the tokens of the input and produces its answer token by token as well. Tokens are thus the actual currency that language models reckon in.

Why tokens matter

Tokens have three practical consequences:

• Cost. Most AI providers bill per processed token. Long inputs and long answers cost more.
• Speed. More tokens mean more computation and therefore slightly longer response times.
• Limit. Every model has a context window, that is, a maximum number of tokens it can take into account at once.

An example

If a customer sends a short question of 20 words, that's roughly 30 to 40 tokens. If the system additionally attaches a long conversation history and several documents from the knowledge base, that can quickly turn into a few thousand tokens. So good systems deliberately select only the relevant passages instead of sending everything along.

Tokens and SendSeven

SendSeven is a unified messaging platform with a built-in AI assistant. To keep answers fast and on point, the RAG technique doesn't send your entire knowledge base to the model, only the passages that match the question. That keeps token usage low and the answers relevant. The AI support bot guide shows how to set up the AI assistant. In line with GDPR, Made in Germany. Try it free for 14 days.