A context window is the amount of text an AI model can hold in mind at any one time, measured in tokens. It covers your messages, the model’s replies, and anything else you have shared, and once that space fills up, the oldest material starts to drop away.
Key Takeaways
- A context window is an AI model’s working memory: everything it can “see” while it answers.
- It is measured in tokens, not words, and a token is roughly three-quarters of an English word.
- Your messages, the model’s replies, uploaded files, and hidden system instructions all share the same space.
- When the window fills up, older parts of the conversation are trimmed or summarised, so the model may forget earlier details.
How does a context window work?
Think of it like a desk. A language model does its thinking on that desk, and the context window is how big the desk is. Every time you send a message, the model reads everything on it — your new message, the earlier turns, and any files you attached — then writes its reply and adds that to the desk too.
The catch is that the desk has a fixed size. If you paste in a fifty-page report, the model can only work with as much of it as fits, and because its replies also land on the desk, a long conversation slowly eats into the space left for new input. That is why a chatbot can feel sharp at the start of a chat and a little forgetful an hour later. The limit comes from the transformer design, which weighs the relationships between every piece of text in the input, so a bigger window needs far more computing power.
What counts toward the context window?
More than most people expect. It is not just the message you typed — almost everything the model is given for a single reply shares the same budget:
- Your messages. Every question and follow-up you send.
- The model’s replies. Each answer becomes part of the conversation the model re-reads next time.
- Uploaded files. A document or image you attach is converted into tokens and takes up room too.
- Hidden instructions. Many chatbots carry a system prompt you never see, telling the model how to behave.
- Retrieved information. When a tool pulls facts from a search or a database before answering, that extra text also lands in the window — see our guide to retrieval-augmented generation.
How big is a context window, in plain terms?
Windows are measured in tokens, the small chunks a model splits text into. A token might be a whole short word, part of a longer word, or a punctuation mark, and one token is roughly three-quarters of an English word, so 1,000 tokens is about 750 words. Tokenization explains why some words cost more space than others.
The size varies a lot. Early chatbots worked with just a few thousand tokens, enough for a short conversation or a page or two. Many of today’s models handle hundreds of thousands, and the largest can take in over a million — roughly a whole bookshelf of text in one go. Even so, a million tokens will hold a long report or a day of chat, not a whole company’s archive. The window also covers input and output together, so a reply can run out of room mid-sentence.
What happens when the context window fills up?
It depends on the tool, but the broad pattern is the same: something has to give. In a chat app, the usual answer is trimming — the oldest messages are quietly dropped, or the early part of the conversation is summarised into a short recap. Either way, the model loses detail: a name from the start, a rule you set, or the reason you began the chat.
Accuracy can also dip before the window is full. Research shows models make best use of information near the start or the end of a long input, and are more likely to miss things buried in the middle. IBM’s explainer on context windows describes this “lost in the middle” effect — so stuffing a window to the brim is not always the smartest move.
How is a context window different from AI memory?
People often mix the two up. A context window is short-term working memory: it only lasts for the current conversation. Some chatbots also offer AI memory, which saves selected facts about you and recalls them in later sessions. If you tell a chatbot your dog is called Biscuit, the window lets it use that name in the next few messages; AI memory would still know it next week. One is a desk, the other is a filing cabinet.
How can you work within a context window?
You cannot change the size of a model’s window, but you can use the space well:
Start a fresh chat for a fresh task. When you switch topics, open a new conversation. The model gets a clean desk, and none of your budget is spent on yesterday’s subject.
Paste only what matters. Before sharing a long document, cut it to the sections the model actually needs. A focused excerpt usually gets a better answer than a whole report.
Summarise long chats yourself. If a conversation has run for a while, ask the model to summarise what you have agreed so far, then start a new chat with that summary. You keep the decisions and drop the dead ends.
Put the important bits last. Models pay most attention to the beginning and the end of an input, so place your key instruction near the end of a long message, not the top.
Check the token count when it matters. Some tools show how many tokens your input uses, so you can check whether a long file will fit before you send it — Anthropic’s context window documentation explains how its models count tokens.
None of this is about gaming the system — it is about giving the model a clean input so it can spend its attention on your question instead of wading through clutter. Our guide to large language models covers the basics of how these systems read text.
FAQs
What is a context window in simple terms?
It is the amount of text an AI model can consider at once while it answers you. Think of it as the model’s working memory or desk space: everything it needs — your messages, its replies, and any files — has to fit on it.
How many words fit in a context window?
It depends on the model, because windows are measured in tokens. Roughly, one token equals about three-quarters of an English word, so a 100,000-token window holds around 75,000 words — about a novel’s length.
Does a bigger context window mean a smarter model?
Not necessarily. A bigger window lets a model handle longer documents, but it does not make its reasoning better — accuracy can even fall when the window is crammed full, because details in the middle get overlooked.
Can I make a context window bigger?
You cannot change a model’s built-in limit, but you can pick a model with a larger window if your tool offers one, and use the space better by keeping chats focused and trimming long documents.
Conclusion
A context window is simply the size of an AI model’s working memory. Knowing it is limited — and that your messages, its replies, and your files all share the same space — explains a lot of everyday chatbot behaviour, from forgotten details to answers that stop halfway through.
The practical takeaway is straightforward: keep chats focused, start fresh when the topic changes, and paste only the material the model needs. Try it next time a chat starts to drift — open a new one, paste in a two-sentence summary of where you got to, and see how much sharper the answers become.
