AI Tokens Explained: What They Are and Why They Cost Money

A person at a kitchen table watching glowing puzzle-piece tiles stream from a laptop into an AI orb and drop into a brass coin meter

Updated November 2026 · our own token test across four tokenizers, current prices, and what a whole book costs to send

The short version: AI models read and write text in small pieces called tokens. In English, one token is about three-quarters of a word. Companies charge for their models by the million tokens, with replies usually costing about five times as much as what you send. In our test, a whole novel came to about 37,000 tokens, and the same question in Hindi took up to nearly five times as many tokens as in English.

Tokens are the pieces of text an AI model reads and writes, and the unit you pay in. We counted them ourselves, across four tokenizers, five languages and one whole novel.

Use ChatGPT, Claude or Gemini for long enough and you will run into the word “token.” It shows up in usage limits, in API price lists (what developers pay to build AI into their own apps) and in error messages about chats that got too long. If you are new to AI, our plain-English guide to what AI is is a good first stop, and the AI glossary covers the other terms you will meet.

Words you will meet

Token
A small piece of text, often a whole short word or part of a longer one, that an AI model reads and writes.
Tokenizer
The tool that cuts your text into tokens before the model sees it. Each AI company uses its own.
Input and output tokens
Input is everything you send, including the chat so far. Output is what the AI writes back.
Context window
The most tokens a model can take in at once, counting your input and its reply.

What is an AI token?

A token is a chunk of text, usually a short word, part of a longer word, a number or a punctuation mark. AI language models don’t read letters or whole words. Before your message reaches the model, a tool called a tokenizer cuts it into tokens and turns each one into a number. The model works with those numbers and writes its answer one token at a time. You can watch it happen as a reply appears in little bursts.

OpenAI’s rule of thumb is that one token is about 4 characters of English, or about three-quarters of a word, so 100 tokens is about 75 words. Our test agreed: the full text of Alice’s Adventures in Wonderland came to 26,525 words and 36,813 tokens on OpenAI’s tokenizer, or 0.72 words per token.

A question about dinner split into 20 colored token boxes, with long words, a year, a price and a large number split into smaller pieces
How OpenAI’s tokenizer cuts up text. We ran each example through o200k, the tokenizer behind OpenAI’s current models, on November 7, 2026. The dots mark spaces, which ride along with the next word.

Common words like “dinner” and “chicken” are one token each, with the space in front of them included. Longer or rarer words get split: “tokenization” becomes “token” and “ization,” and a made-up word like Supercalifragilisticexpialidocious takes 10 tokens. Numbers split too, so the year 2026 is two tokens and a price like $19.99 is four.

How we tested it

On November 7, 2026 we took one everyday request, “Can you suggest an easy dinner I can make tonight with chicken, rice and whatever vegetables I have?”, and wrote it in English, Spanish, Arabic, Japanese and Hindi. We added a two-line piece of Python code, a short message with emoji, and the whole of Alice in Wonderland from Project Gutenberg. Then we counted the tokens four ways:

  • OpenAI’s o200k tokenizer, using tiktoken, OpenAI’s free token-counting tool
  • Gemini 4 Argon, through Google’s token-counting tool in the Gemini API
  • DeepSeek-V3 and Qwen3, two popular open models (free to download and run yourself), using the tokenizer files their makers publish (DeepSeek, Qwen)

We couldn’t include Claude. Anthropic offers a free token-counting tool, but only through its developer platform, which needs an API account.

The same request in…OpenAI (o200k)Gemini 4 ArgonDeepSeek-V3Qwen3
English20202020
Spanish22222929
Arabic31313733
Japanese33192628
Hindi33306096
Python code22272322
English with emoji14111614

Token counts from our test on November 7, 2026. The request was “Can you suggest an easy dinner I can make tonight with chicken, rice and whatever vegetables I have?”, translated. Gemini figures leave out the 31 tokens Gemini 4 Argon adds to every request.

Does the same sentence cost the same in every language?

No, and the gap can be large. In English the dinner question was 20 tokens on all four tokenizers. Spanish was close on OpenAI and Gemini (22) but 29 on DeepSeek and Qwen. Hindi took far more:

Tokens for the same 22-word Hindi question

OpenAI (o200k tokenizer)

Gemini 4 Argon

DeepSeek-V3

Qwen3

Our test, November 7, 2026. The English version of the same question was 20 tokens on all four.

Qwen3 needed 96 tokens for the Hindi question, almost five times the English count, while OpenAI’s tokenizer needed 33. Japanese went the other way on Gemini 4 Argon, which counted 19 tokens (leaving out its 31 extra), fewer than the English version. Tokenizers learn their pieces from the text they were built on, so languages that were well represented get bigger, more efficient pieces and others get chopped finer.

If you pay by the token, the same request can cost several times more in one language than another. And because a model’s working memory is measured in tokens, a long document in Hindi fills it faster than the same document in English.

Something we noticedGemini 4 Argon added 31 tokens to every request we sent. A one-letter message counted as 32 tokens. The two Gemini Flash models we also tried, 3.8 Flash and 2.5 Flash, counted it as 1. On a long document you won’t notice it. On thousands of short requests, those 31 tokens each start to count.

Input tokens and output tokens

Every request has two sides. Input tokens are everything sent to the model: your question, any file you attach, the instructions the app adds behind the scenes, and in a chat, the whole conversation so far. Output tokens are the reply.

Output costs more. Most companies charge about five times as much for output as for input: Claude Sonnet 5.5, for example, is $2 per million input tokens and $10 per million output tokens (Anthropic). The model can read your whole input at once, but it writes its answer one token at a time, and each new token means running the whole model again.

Long chats cost more than you might expect. Each time you send a new message, the app sends the earlier messages again so the model has the context. Message 20 in a long chat carries the tokens of the first 19 with it.

What is a context window?

A context window is the most tokens a model can handle at once, counting what goes in and what comes out. Think of it as the model’s desk: everything it is working with has to fit on it.

Windows have grown quickly. GPT-4 launched in March 2023 with 8,000 tokens, or 32,000 in a pricier version (OpenAI). GPT-4 Turbo reached 128,000 that November (OpenAI), Claude 3 offered 200,000 in March 2024 (Anthropic), and Google had already let early testers try 1 million with Gemini 1.5 Pro (Google). Today 1 million tokens is common among top models, and every model in Anthropic’s current lineup has a window that size (Anthropic). By our count, a million tokens would hold Alice in Wonderland about 27 times over.

When a conversation outgrows the window, the oldest parts drop out or get summarized. So a very long chat can seem to forget what you said at the start.

What do tokens cost in November 2026?

Standard API prices for the main models, in US dollars per million tokens. Prices change often, so check the linked pages before you budget around them.

ModelInput (per 1M tokens)Output (per 1M tokens)
Claude Haiku 5.5 (Anthropic)$0.10$0.50
Claude Sonnet 5.5 (Anthropic)$2$10
Claude Opus 5.5 (Anthropic)$4$20
Claude Fable 5.1 (Anthropic)$10$50
GPT-6 Luna (OpenAI)$0.10$0.50
GPT-6.1 Sol (OpenAI)$2$10
GPT-6 Astra (OpenAI)$10$50
Gemini 4 Argon (Google)$2 ($4 from Dec 1, 2026)$10 ($20 from Dec 1, 2026)
Grok 4.7 (SpaceXSI)$2$6

Standard prices from the Anthropic, OpenAI, Google and SpaceXSI pricing pages, checked November 7, 2026. Some prices rise for very long prompts: above 100,000 tokens for Haiku 5.5, 200,000 for Grok 4.7 and 272,000 for the GPT-6 models.

What it costs to send the whole of Alice in Wonderland to each model once, using our token counts:

ModelTokens for the bookCost to send it once
GPT-6 Luna (OpenAI)36,813$0.004
Claude Haiku 5.5 (Anthropic)about 48,000*$0.005
GPT-6.1 Sol (OpenAI)36,813$0.074
Gemini 4 Argon (Google)38,358$0.077, then $0.15 from Dec 1
Claude Sonnet 5.5 (Anthropic)about 48,000*$0.096
Claude Opus 5.5 (Anthropic)about 48,000*$0.19
GPT-6 Astra (OpenAI)36,813$0.37
Claude Fable 5.1 (Anthropic)about 48,000*$0.48

Token counts for OpenAI and Gemini are from our test. *We had no Claude count. Anthropic says 1 million tokens is about 555,000 words on its current models (Anthropic), and we used that ratio, so the real figure could be a little higher or lower. The Gemini count includes the 31 extra tokens Argon adds to each request. Prices are the standard rates on each company’s pricing page, checked November 7, 2026.

Even on the most expensive models, sending a whole novel costs well under a dollar. Asking for a 500-word summary back adds only about 700 output tokens. Costs climb when a task runs over and over, or when an AI agent works on its own for hours. When a hobbyist let GPT-6 Astra play the video game Portal by itself, reading screenshots and choosing moves, the run took 3,336 steps and ran up a $571.18 bill, as we covered in the newsletter.

Beginners in AI free community on Skool: learn AI in fourteen days, master it in 30, one small win a day, zero experience needed

Why do AI companies charge by the token?

Every token costs them computing time. Each token you send or receive passes through a model with billions of settings, running on expensive graphics chips that use a lot of electricity. Charging by the token ties the price to the work done: a one-line question costs a fraction of a cent, a 300-page report costs more.

Token prices have dropped a long way since 2023. GPT-4 cost $30 per million input tokens and $60 per million output tokens when it launched in 2023 (OpenAI’s 2023 price list). Today OpenAI’s smallest current model, GPT-6 Luna, costs $0.10 and $0.50 (OpenAI). That is one three-hundredth of the 2023 input price, though Luna is a small model, not a direct match for GPT-4. The most capable models still cost more, and Google is raising Gemini 4 Argon’s price on December 1.

Images, audio and video use tokens too

Models that can see and hear turn those inputs into tokens as well. Anthropic cuts each image into squares 28 pixels across and counts one token per square. A 1,000 by 1,000 pixel photo comes to 1,296 tokens (Anthropic). OpenAI uses a similar patch-based method for GPT-6 Astra. A large screenshot can use more of your allowance than a long written question. Video uses the most, because it is many images in a row, often with sound.

Do you need to think about tokens if you just use ChatGPT or Claude?

Mostly not. Free and paid chat plans charge a flat price (or nothing) and set usage limits instead of billing per token. Our guide to what Claude’s free plan includes shows how those limits work in practice. Tokens still explain why you hit a limit sooner on some days than others:

  • A big PDF or a stack of screenshots uses many more tokens than a short question.
  • A long chat re-sends everything earlier in the conversation with each new message.
  • Long, detailed answers use more output tokens than short ones.
  • The strongest models and “thinking” modes use up your allowance faster.

If you keep running out, start a new chat when you change topic, attach only the pages you need, and ask for shorter answers. This prompt does the last part for you:

Prompt for shorter answers
Answer in no more than 150 words. Use short bullet points, skip the introduction and the summary at the end, and don’t repeat my question back to me. If you need more detail from me, ask one question first.

Clear prompts help here as well, because a vague request often leads to a long answer you didn’t need. Our guide to writing better AI prompts has more on that.

Ways to spend fewer tokens if you use an API

  • Pick the smallest model that does the job. Sorting, short summaries and pulling details out of text rarely need a top model, and small models cost 20 to 100 times less.
  • Set a limit on output. APIs let you cap the length of a reply, and asking for brevity in the prompt helps too.
  • Trim the history. Summarize old turns instead of sending the whole conversation every time.
  • Count before you send. OpenAI and Anthropic offer free token counters and Google’s Gemini API has one too, so you can check a big document before paying to process it.
  • Run a model yourself. Open models such as Qwen, DeepSeek, Gemma and gpt-oss can run on your own computer, where the only cost per token is electricity. Our guide to running open-source AI locally shows how, and our open-source AI guide explains the options.

If you are choosing between the big chat apps rather than building with them, our ChatGPT vs Claude vs Gemini comparison and our list of the best AI tools for beginners are more useful places to start.

Made for regular people, by a real person

A free daily newsletter that explains AI terms like this one without the jargon. Curated and edited by a human, in plain English.

Free forever. Unsubscribe anytime.

Frequently asked questions

How many tokens is 1,000 words?

About 1,300 to 1,400 tokens of ordinary English. In our test, Alice in Wonderland worked out at 1,388 tokens per 1,000 words on OpenAI’s tokenizer. Other languages, code and text full of numbers usually take more.

Do free AI tools use tokens?

Yes. Every language model works in tokens, whether or not the app shows them. On a free plan the company pays for those tokens, and that cost is the reason free plans have daily or weekly limits.

Why are output tokens more expensive than input tokens?

The model can read your whole input at once, but it has to write the reply one token at a time, running the whole model again for each token. That makes each output token more work.

What happens when I go over the context window?

The app either drops or summarizes the oldest part of the conversation, or refuses the request with an error. Either way the model can no longer see what fell out and may seem to forget things from earlier.

Are tokens the same across AI models?

No. Each company uses its own tokenizer, so the same text gives different counts. English came out almost the same on all four tokenizers in our test, but Hindi ranged from 30 to 96 tokens. Anthropic says the tokenizer in its newer Claude models produces about 30% more tokens for the same text than older ones did.

Sources

You may also like