Context Window

The context window is the amount of text an AI model can take into account at once for a request, measured in tokens. Everything inside the window feeds into the answer, anything beyond it is cut off. In long conversations, the context window decides how much history the AI can include.

What is the context window?

A Large Language Model can't process an unlimited amount of text at once. The context window is the upper limit for that, measured in tokens. Into this window fit the current question, the conversation history so far, instructions, and any attached documents. Whatever no longer fits, the model can't take into account in its answer.

Modern models have very large context windows, yet it is never unlimited, and more context also means more computing cost.

Why the context window matters

In customer service, the context window shows up in three places:

• Conversation memory. Only what's in the window is present to the AI. Very long conversations can be “forgotten” at the start.
• Documents. You can't attach an unlimited amount of content at once.
• Cost and speed. A fuller window costs more tokens and therefore a bit more time and money.

The way out: targeted rather than everything

Good systems don't fill the window at random, they select the relevant content. That is exactly what RAG does: instead of sending along the whole knowledge base, it picks out only the passages that match the question and keeps the context window lean and on point.

Context at SendSeven

SendSeven is a unified messaging platform with a built-in AI assistant. The assistant takes the course of a conversation into account and uses RAG to pull only the matching content from your knowledge base, instead of filling the context window with the unnecessary. That keeps answers relevant and costs in check. The knowledge base guide shows how to prepare your knowledge base for it. In line with GDPR, Made in Germany. Try it free for 14 days.