Introducing self-improving skills: reusable AI capabilities that learn from usage

Fabien Celier

-July 7, 2026

Skills are one of the best ways to scale expertise across digital teams of humans and agents. They contain sets of instructions that encode various team processes and know-how: how to query revenue data, handle support escalations, review contracts, or follow internal policies.

The challenge is that a first version of a skill is rarely the final one, and they take several rounds of iteration to get right. That’s because the improvements that need to be made only become evident through repeat usage: users ask ambiguous questions, tools return unexpected results, policies change, and teams discover that “obvious” context was never actually documented anywhere.

Until now, improving a skill meant doing that synthesis manually. Builders had to collect and understand feedback, inspect conversations, identify recurring patterns, and decide how to update the skill.

Self-improving skills automate and close that loop.

Dust now analyzes how a skill is used, turns conversation and feedback signals into concrete improvement suggestions, and surfaces them as minimal diffs for builders to review. Nothing happens to the model's weights, and the skill doesn't rewrite itself. Rather, the learnable layer is the skill instructions itself, which stays explicit, inspectable, and in the hands of the people who own it.

Skills are where reusable AI know-how lives

A good skill can combine instructions, tools, knowledge sources, examples, escalation rules, formatting preferences, and workflow conventions. It tells Dust not only what to do, but how work should be done in a specific context.

For example:

A support skill might encode how to classify a customer issue, which sources to search first, when to escalate, and how to draft a response.
A data skill might encode which warehouse tables to use, how business metrics are defined, and when to ask a clarifying question before writing SQL.
A writing skill might encode company voice, structure, banned phrases, review rules, and examples of good output.
A sales skill might encode how to prepare for a meeting, which CRM fields matter, and how to distinguish a champion from an economic buyer.

Skills capture institutional knowledge in a form AI systems can use directly. But like any documentation, they're most useful as living artifacts, not static prompts: assumptions go stale, small gaps compound, and the edge cases only show up through real usage. A skill that isn't maintained will stop being useful.

Every skill usage is a test case

Every time a skill is used, it produces evidence. Some of that evidence is implicit: a user corrects the output or asks to use another source. They might ask the same thing again with different wording, or clarify something the skill should have done differently. The result is the same: the skill didn’t produce the result the user expects, resulting in more prompting and less efficiency.

Individually, these signals are noisy. For example, single correction might reflect an unclear query or an exceptional use case rather than a flaw in the skill’s instructions. The solution isn’t to add more instructions, because overfitting a skill to every one-off failure creates contradictory directives that actually degrade outcomes.

Across many conversations, though, patterns start to show up: the same ambiguity trips people up repeatedly, a tool keeps getting skipped, a policy gets misread the same way by different users. That repetition is the real learning signal, evidence that the skill definition is actually missing something, not that one conversation went sideways.

The hard part is synthesis. Nobody wants to read every conversation, cluster the failures by hand, decide which ones matter, and write the patch themselves. This is work AI is genuinely good at.

From usage signals to reviewed suggestions

Self-improving skills turns skill usage into a structured improvement loop.

At a high level, the pipeline looks like this:

The first step is collecting signals from conversations where the skill was used. Dust can look at explicit feedback, conversation content, user corrections, and patterns in how the skill interacted with tools or knowledge sources.

The second step is generating synthetic feedback. Instead of immediately editing the skill, the system first asks: did something meaningful happen here? Was there a failure mode? Was the user correcting a missing instruction? Was the skill missing context, using the wrong tool, or failing to ask a clarifying question?

That synthetic feedback is then aggregated across the current skill version. This matters because the goal is not to chase every individual complaint. The goal is to detect recurring, meaningful opportunities to improve the skill.

Finally, when there is enough signal, Dust proposes a change. The output is not a hidden update. It is a suggestion: a focused diff to the Skill definition, with enough context for a builder to review it.

The important part is the boundary: Dust automates analysis and synthesis, but the builder remains responsible for accepting the change.

A diff is the unit of learning

The self-improvement loop does not try to rewrite the whole skill every time something goes wrong. That would make skills longer, harder to reason about, and more fragile.

The goal is a minimal useful diff.

A good suggestion should be small enough to review, specific enough to address the observed issue, and general enough to help future usage without overfitting to one conversation.

For example, imagine a support skill repeatedly escalates billing questions too quickly. Users ask things like “Why was I charged?” or “This invoice looks wrong,” and the skill treats all of them as the same support path.

A useful suggestion might look like this:

+ When a user asks about billing, first determine whether they mean
+ subscription billing, usage-based billing, or invoice collection.
+ If the request is ambiguous, ask one clarifying question before escalating.

That is not a model update. It is an explicit lesson extracted from usage and written back into the Skill.

Or imagine a data skill that repeatedly answers “revenue” questions using the wrong business definition. The issue is not that the model cannot reason about revenue. The issue is that the Skill needs to encode the organization’s metric taxonomy.

A suggestion might be:

+ Before answering revenue questions, clarify whether the user means
+ booked ARR, recognized revenue, or cash collected when the metric is ambiguous.

Or a research skill might learn better tool ordering:

- For engineering documentation, search Slack and the documentation source.
+ For engineering documentation questions, search the documentation source first.
+ Use Slack only to verify recent decisions, open questions, or unresolved discussions.

These changes are understandable. A builder can inspect them like code. They can ask: Is this backed by real evidence? Is the wording too broad? Does this belong in this skill? Could it create regressions? Should the rule be narrower?

That review step is what makes the loop safe. Self-improving skills improve by making lessons explicit, not by hiding them.

Improving behavior without changing model weights

It is tempting to describe any AI improvement loop as “training.” But self-improving skills do not use fine-tuning.

Fine-tuning changes model weights. That can be powerful, but it is often the wrong primitive for company-specific workflows. Many failures in production AI systems are not general intelligence failures. They are configuration failures.

The skill did not know which internal source was canonical. It did not know which metric definition mattered. It did not know that the team always wants one clarifying question before escalation. It did not know that Slack is noisy for one class of question but useful for another. It did not know that a policy changed last week.

Those are not things you necessarily want buried inside model weights. You want them explicit. You want the builder to review them. You want the organization to understand why behavior changed.

Self-improving skills operate at that explicit layer: instructions, tool guidance, knowledge usage, and workflow conventions. That makes the improvement loop more inspectable, more data-efficient, and easier to govern.

It also makes iteration faster. You don’t need thousands of labeled examples to improve a skill that keeps confusing two business metrics. You need enough evidence to propose the right instruction, and a builder who can approve it.

System prompt learning for reusable AI workflows

One way to think about self-improving skills is as system prompt learning. Most useful learning doesn't look like changing weights, it looks like writing down a better procedure after you hit an edge case. Humans do this all the time: we add a step to the checklist, clarify a definition in the runbook, leave a note that says "next time, do this first."

Self-improving skills apply the same idea to reusable AI workflows. The skill watches how it's actually used, notices lessons that keep recurring, and proposes edits to the instructions that shape its future behavior.

This is powerful because skills are already the place where reusable behavior is encoded. If a skill contains the operating procedure for support triage, then improving the skill is equivalent to improving the reusable procedure. If a skill contains the logic for safe data analysis, then improving the skill makes future analysis more reliable.

The result is not a black-box system that “learns” in an undefined way; it’s a transparent improvement loop where the output of learning is a reviewable instruction change.

Self-improving does not mean self-mutating

Any system that improves itself needs a strong control model, and self-improving skills are built around that constraint. Suggestions aren't applied silently; rather, builders review every change before it becomes part of the Skill. The system proposes, and the human decides.

That review step isn't a formality. A skill might receive many signals, but not every signal should become an instruction: some feedback is simply wrong, some requests are one-offs, and some fixes would solve today's issue while quietly creating tomorrow's regression. Human review keeps the skill intentional rather than accumulating unverified rules.

The same principle governs privacy and data boundaries. Improvements stay in the workspace where the usage happened, just like customer conversations shouldn’t be used to fine-tune foundation models, and learnings shouldn’t silently leak across workspaces. The whole loop is explicit and reviewable.

A skill should get better with use without becoming a black box to the people who rely on it.

Reusable capabilities should improve as they are used

The value of a skill compounds because it is reusable.

A one-off prompt improvement helps one answer. A skill improvement strengthens a capability that can be used again and again. Each approved diff makes the Skill a better representation of how the organization wants work to be done.

Over time, this creates a loop:

That loop is the core idea behind self-improving Skills.

Skills are how teams package reusable AI know-how. Self-improving skills make that know-how easier to maintain. Every conversation can reveal something: an ambiguous instruction, a missing policy, a tool that should have been preferred, or a workflow convention that was assumed but never written down.

Self-improving skills turns those signals into reviewed improvements, so skills can become more useful without becoming less transparent.

The important distinction is that the system improves by making its lessons explicit. Not by hiding them in model weights, and not by applying changes without review, but by proposing small, auditable updates to the reusable instructions, tools, and knowledge that define the Skill.

Reusable capabilities should improve as they are used. Self-improving skills are how that loop becomes part of Dust.