In-context learning (ICL) is the ability of large language models to adapt to new tasks using only information provided in the current prompt — without any updates to the model’s weights or additional training. When you show a model a few examples and it immediately applies the pattern to a new case, that’s in-context learning. It’s the core mechanism behind few-shot learning, zero-shot learning, and chain-of-thought prompting — and it’s one of the most surprising properties of modern AI.
Learn Our Proven AI Frameworks
Beginners in AI created 6 branded frameworks to help you master AI: STACK for prompting, BUILD for business, ADAPT for learning, THINK for decisions, CRAFT for content, and CRON for automation.
How In-Context Learning Works
Traditional machine learning requires a clear separation: first you train the model, then you use it. In-context learning blurs this line. The model “learns” from the examples in its current context window — the text it can see right now — and uses that learning to respond appropriately.
Crucially, this learning is temporary. It only lasts for the duration of the current context. Once the session ends or the context window is cleared, the model returns to its default behavior. No weights changed. No permanent adaptation. This is both a feature (safe, reversible, fast) and a limitation (doesn’t accumulate across sessions without external memory).
Researchers believe ICL works because LLMs develop implicit meta-learning capabilities during pre-training on massive text corpora. The model learns patterns about how patterns work — which is why it can recognize a new classification task just from seeing a few labeled examples, even if that exact classification was never in its training data.
What In-Context Learning Enables
In-context learning is what makes LLMs feel like general-purpose tools rather than narrow specialists. Key capabilities it enables:
- Task switching without retraining: Give the model a few examples of sentiment analysis, and it becomes a sentiment analyzer. Switch to translation examples, and it becomes a translator.
- Style adaptation: Show the model examples of your writing style, and it will mimic that style for new pieces.
- Domain adaptation: Inject domain-specific terminology and examples from a specialized field, and the model performs significantly better in that domain.
- Format learning: Demonstrate an unusual output format (JSON schema, custom report structure), and the model applies it consistently.
The growing size of context windows — from 4K tokens in early GPT-3 to 200K tokens in Claude and 1M+ in Gemini — dramatically expands what’s possible with in-context learning. With enough context, you can inject entire codebases, books, or datasets for the model to reason over.
Limitations and Challenges
In-context learning is powerful but not magic:
- “Lost in the middle” problem: Research shows models are better at using information at the beginning and end of their context than in the middle. Long contexts can degrade ICL performance.
- Example quality matters: Noisy or inconsistent examples actively hurt performance.
- Context window limits: Every example and document you inject reduces the space available for the model’s output.
- Cost scales with context: Longer prompts cost more with token-based pricing.
For tasks requiring sustained, high-quality adaptation, fine-tuning still outperforms in-context learning. ICL shines for flexibility, speed, and prototyping — especially when combined with system prompts in agentic AI workflows.
Key Takeaways
- In-context learning lets LLMs adapt to tasks using only information in the current prompt.
- No model training or weight updates are required — learning happens entirely within the context window.
- It’s the mechanism behind few-shot, zero-shot, and chain-of-thought prompting.
- Larger context windows dramatically expand what’s possible with ICL.
- ICL is temporary; for permanent behavioral change, fine-tuning is needed.
Frequently Asked Questions
Is in-context learning the same as fine-tuning?
No. Fine-tuning permanently updates model weights through additional training. In-context learning is temporary, prompt-based adaptation that vanishes when the session ends — no weights are changed.
Does in-context learning work for all AI models?
It’s primarily a property of large language models trained at scale. Smaller models and earlier architectures show limited ICL capability. The phenomenon became prominent with GPT-3 (175B parameters) in 2020 and has become more powerful with each new generation.
What’s the relationship between in-context learning and RAG?
Retrieval-Augmented Generation (RAG) uses ICL as its foundation — it retrieves relevant documents and injects them into the context window, allowing the model to use that information via in-context learning to answer questions more accurately.
How many examples can I fit in the context for in-context learning?
It depends on the model’s context window size and the length of each example. With Claude’s 200K token context, you could potentially fit hundreds of short examples. Longer examples mean fewer fit, and very long contexts can reduce ICL quality due to the “lost in the middle” problem.
Does the order of examples in the context matter?
Yes, somewhat. Research shows recency effects — examples closer to the query tend to have more influence. For few-shot prompting, putting the most representative or carefully crafted examples last (nearest to the question) can improve results.
Want to go deeper? Browse more terms in the AI Glossary or subscribe to our newsletter for daily AI concepts explained in plain English.
Free download: Get the Beginners in AI Report — free daily briefings on the latest in AI research and tools.
Sources
You May Also Like
Get free AI tips daily → Subscribe to Beginners in AI
Get Smarter About AI Every Morning
Free daily newsletter — one story, one tool, one tip. Plain English, no jargon.
Free forever. Unsubscribe anytime.