Updated November 2026 · Rewritten on November 13 for the new Skills features in Claude, ChatGPT and Gemini.
A feedback loop turns your corrections into written rules, so an AI skill stops repeating the same mistakes. Below is a simple version that works in any app, plus the built-in tools that now handle part of it.
The short versionKeep a short “lessons learned” file next to your skill. Tell the AI to read it before each task and to suggest new rules after each one. You approve the rules. In Claude you can now edit a skill in place, and ChatGPT and Gemini now use the same skill format.
Why Do AI Skills Stop Getting Better?
An AI skill is a saved set of instructions for one job, such as writing a newsletter section or summarizing a meeting. You write it once and reuse it. The trouble is that it stays exactly as you wrote it on day one.
So you correct the same things again and again. You swap “purchase” for “buy”, you ask for the summary to come first, you remind it your readers are beginners. The AI forgets those fixes unless you write them down where it will read them next time.
That is the whole idea of a feedback loop: each correction becomes a rule, and the rules travel with the skill. Anthropic’s advice for writing skills puts it in one line.

What Does the Research Say?
Feedback loops, by the numbers
91%
coding test score for an AI that writes itself lessons from feedback and tries again
HumanEval coding test, first-try pass rate
GPT-4 alone
Reflexion agent with written lessons
The Reflexion result is the closest research match to a lessons file. The agent wrote a short note about what went wrong, kept it, and read it before the next try.
One warning: a 2023 study found that models asked to check their own reasoning, with nothing from outside to go on, struggled to fix their answers and sometimes made them worse. The loop works because you supply the correction. The AI writes the rule down; you decide whether it is right.
Build the Loop in Four Steps
Step 1: Start a lessons file
Make a plain text file called lessons.md. It can start almost empty. This template gives it sections to grow into:
# Lessons Learned: [skill name]
Last updated: [date]
## Always do
- [Example: Define every technical term the first time it appears]
- [Example: Put the main answer in the first two sentences]
## Never do
- [Example: Don't write "purchase"; write "buy"]
- [Example: Don't open with "In today's world"]
## Format rules
- [Example: Numbered lists for steps, bullets for everything else]
- [Example: A table whenever you compare three or more things]
## About the audience
- [Example: Readers are beginners; give a concrete example before the theory]
## Mistakes to check for
- [Example: The FAQ section is often missing from first drafts]
## Change log
- [date]: started this file
Step 2: Tell the skill to use it
Add these lines to the skill’s instructions (or a project’s instructions). They make the AI read the file first and propose new rules last. Our CLEAR prompting guide covers writing the rest of the instructions.
Before you start any task, read the Lessons Learned file and follow every rule in it.
When you finish, suggest 2 or 3 specific additions to the Lessons Learned file, based on what went well, what went badly, and any corrections I made. Quote the correction, then write the rule.
Step 3: Review the session
At the end of any session where you corrected the AI, paste this. It turns your corrections into rules you can copy straight into the file:
Look back over this chat. List every correction I made to your work. For each one, write a one-line rule I could add to my Lessons Learned file so you never need that correction again. Skip anything that only applied to this one task.
Read the suggestions before you accept them. Keep the ones that match what you want, and add anything it missed. A rule like “define technical words on first use” is worth keeping; a rule about one odd task isn’t.
Step 4: Keep the file short
More rules aren’t always better. Anthropic suggests keeping a skill’s main file under 500 lines, and the docs for Claude Code, its coding tool, say instruction files longer than about 200 lines “reduce adherence”. Every few weeks, merge duplicate rules and delete ones you no longer need.
How Do You Update a Skill in Claude Now?
Claude changed most here since April. A Claude skill is a folder with a SKILL.md file inside, and custom skills now work on every plan, including Free, as long as code execution (Claude running small programs for you) is turned on. You manage them under Customize > Skills.
You no longer need to download, edit and re-upload a file to change a skill. While you chat, the skill file opens beside the conversation. Highlight the text you want changed, click Edit with Claude, and type the change. You can also ask Claude in the chat to update the skill.

That means your lessons file can live inside the skill folder. Here is a small skill that points to one:
---
name: beginner-explainers
description: Writes short plain-English explainers of AI terms for beginners. Use when asked to explain an AI word or idea.
---
# Beginner explainers
Read lessons.md in this skill before writing, and follow every rule in it.
## Steps
1. Answer in the first two sentences.
2. Give one everyday example.
3. Define any other technical word you use.
4. Keep it under 150 words.
The name must use lowercase letters, numbers and hyphens, and match the folder name. Keep the description under 200 characters (Claude’s upload limit) and say when the skill should be used; that is the part Claude reads to decide whether to load it. The full format rules are in the Agent Skills specification.
To make one without writing files, describe the job in a chat and ask Claude to build a skill. It uses a built-in skill-creator to write the SKILL.md for you. Since March, skill-creator can also write tests for a skill and compare two versions side by side, judging the results without knowing which is which. It’s the lessons-file loop with test scores attached.
Memory, Projects and Skills: Which Should Hold Your Lessons?

Claude now has three places that can remember things, and they suit different kinds of lessons.
| Feature | What it holds | Best for | How you edit it |
|---|---|---|---|
| Skill | Instructions for one kind of task, loaded only when needed | Rules for a repeat job | Edit with Claude, or ask Claude to update it |
| Project | Instructions and files that are always loaded in that project | Background for ongoing work | Set project instructions; add or swap files in project knowledge |
| Memory | Facts about you, saved as topics while you chat | Your preferences across every chat | Settings > Your Claude > Manage |
| Claude Code (CLAUDE.md + auto memory) | Your written rules, plus notes Claude saves from your corrections | Coding and file work | Edit the file, or type /memory |
Memory is on by default on Free, Pro, Max and Ultra, and you can read, edit or delete each topic it saves. It is good for general preferences, but it doesn’t replace a lessons file for a specific task, because you can’t see exactly when it applies. In Claude Code, auto memory has a type called feedback for “corrections you give Claude”, which is a lessons file Claude keeps for you. See our Claude memory settings guide, Projects guide and CLAUDE.md guide for the details.

What We Learned Running Our Own Loop
We run this site this way. When our editor corrects the AI that helps produce these pages, the correction becomes a short written rule that every later session reads first. On November 13, 2026 the file held 81 rules, written between April and November.
| Month (2026) | New rules | Example rule from that month |
|---|---|---|
| April | 11 | Never bulk-edit many published pages in one run |
| May | 21 | Run every new post through a check for AI writing habits |
| June to September | 18 | Check a partner’s link in their page code, not their word |
| October | 14 | Put every copyable prompt in a box with a Copy button |
| November (to the 13th) | 17 | Take screenshots at double resolution so they stay sharp |
From our fileThe rules came in bursts, when we started something new (May was a busy launch month). The best rules say why, so the AI can apply them to cases we never listed. And some rules replaced older ones, so pruning matters as much as adding.
It also shows the limit of a loop, which only learns what you correct. Our October 10 newsletter edition covered Anthropic’s test agents doing jobs nobody asked for, and the fix there was another written rule saying where the job ends.
Feedback Loops in ChatGPT and Gemini
Both now offer skills in the open SKILL.md format (in ChatGPT, on Business, Enterprise, Healthcare and Edu plans), so a lessons file works there too. Custom GPTs and Gems, the older tools, are being retired.
| ChatGPT | Gemini | |
|---|---|---|
| Skills | Skills for Business, Enterprise, Healthcare and Edu, under Plugins > Skills | Gemini Skills for personal accounts 18+, under Customize > Skills |
| Old feature | Custom GPTs retire on December 11, 2026; their instructions become a skill in a plugin | Gems become read-only on November 17, 2026 and are removed on December 17 |
| Where to keep lessons | A Project’s instructions and files, or inside a skill | A file inside the skill (download, add the file, then Replace skill) |
If you built your loop inside a Custom GPT or a Gem, move it now. Copy the instructions and your lessons file somewhere safe before the dates above. For the general idea beyond any one app, see our AI feedback loop guide, and for loops that run on their own, the Ralph loop.
Making It a Habit
- End every real session with the review prompt. It takes a couple of minutes.
- Write the reason with the rule. “Define jargon, because readers are beginners” beats “define jargon”.
- Retest after a model update. Anthropic warns a skill that worked last month can behave differently today, and some rules stop being needed.
- Prune once a month. Merge duplicates and delete rules you no longer need.
New to building skills? Start with our 10 AI skills every beginner should build, then add a lessons file to the one you use most. If you’re new to AI altogether, how to use AI and what is artificial intelligence cover the basics, and our free AI courses list has more.
Frequently Asked Questions
Do I need to code to build a feedback loop?
No. A lessons file is plain text, and the instructions are two sentences. Claude can write the skill file for you when you describe the job in a chat.
Does Claude learn from my corrections on its own?
Partly. Claude’s memory saves preferences as topics you can edit, and Claude Code’s auto memory saves corrections. A lessons file is still the clearest way to control exactly which rules a specific skill follows.
How long should the lessons file be?
Short enough that every rule still matters. Anthropic suggests keeping a skill’s main file under 500 lines; most lessons files work best far shorter. Prune duplicates every few weeks.
What happens to my Custom GPT or Gemini Gem?
Custom GPTs retire on December 11, 2026 and can be migrated to a plugin, where the instructions become a skill. Gems turn read-only on November 17 and are converted to Gemini Skills, then removed on December 17. Save your instructions and lessons file first.
Can the AI grade its own work instead of me?
Not well. Research shows models struggle to correct themselves with no outside feedback, and sometimes get worse. Let the AI suggest rules, but you decide which ones go in.
Made for regular people, by a real person
One AI tool, tip or update a day in plain English, including tricks like this that make the tools you already use work better. Curated and edited by a human.
Free forever. Unsubscribe anytime.
Sources
- Anthropic: Skill authoring best practices
- Anthropic: Agent Skills overview
- Anthropic Help Center: What are skills?
- Anthropic Help Center: Creating custom skills
- Anthropic Help Center: Chat search and memory
- Anthropic: Improving skill-creator (March 3, 2026)
- Claude Code docs: Memory
- Agent Skills specification
- OpenAI Help Center: Custom GPT retirement and migration FAQ
- OpenAI Help Center: Skills in ChatGPT
- Google Gemini Help: Skills
- Madaan et al., Self-Refine (arXiv, 2023)
- Shinn et al., Reflexion (arXiv, 2023)
- Huang et al., Large Language Models Cannot Self-Correct Reasoning Yet (arXiv, 2023; ICLR 2024)
You May Also Like
- Claude Skills Explained: Build Custom AI Workflows
- 10 AI Skills Every Beginner Should Build (With Templates)
- The Feedback Loop: How to Make AI Get Better Every Time You Use It
- The CLEAR Prompting Framework: How to Get Consistently Great AI Output
- The Ralph Loop Explained: AI That Keeps Working Until the Job Is Done
- Best AI Tools for Beginners
- AI Tools Directory

