Why your Agent Skills aren't firing

Part 4 of a series on the Agent Skills SDK. Part 3 covered converting content you already have into a real library.

You've got skills now. A dozen of them, converted or written, validating cleanly, wired into an agent.

And then somebody asks the agent a question that one of those skills answers perfectly, and the agent just... answers it. Generically. From its own knowledge. As though the skill you spent an hour on isn't there.

This is the point where most people conclude the approach doesn't work.

It's also, in my experience, almost always the same problem, and it isn't the one people go looking for. The instinct is to go back to the body — add detail, make it more explicit, make it more obviously useful. That's the wrong file. The agent never read the body. It never got that far.

The description is doing all the work

Everything about whether a skill gets used happens before the body is loaded.

The agent sees a catalog: names and descriptions. Nothing else. It chooses from that, and only then does the body get fetched. So the body can hold two thousand tokens of excellent, hard-won knowledge about how your company works, and none of it affects whether the agent ever reads it.

That's an uncomfortable imbalance. The least-reviewed part of the document, the sentence someone wrote last and in a hurry to get past validation, is making the decision that determines whether any of the rest matters.

If your skill isn't firing, the fix is in the description almost every time.

Look at what the agent actually sees

Before changing anything, it's worth checking your assumption. inspect renders a skill exactly as an agent receives it:

agentskills inspect ./skills

It prints the metadata, the catalog entry the agent gets on every turn, and the body it would load if chosen — with a token estimate on the two you actually pay for: the catalog entry, charged every turn, and the body, charged on load.

skills/incident-response  (incident-response)

metadata
  name: incident-response
  description: Triage and mitigate production incidents affecting customer-facing services. Use when alerting has already fired or a customer-visible degradation is confirmed. Not for local development failures, test environment issues, or routine scheduled maintenance.
  version: 1.0.0
  when_to_use: ['Production outage or confirmed customer-facing degradation', 'Severe incidents requiring escalation to on-call']
  when_not_to_use: ['Local development or test environment failures', 'Routine scheduled maintenance']

resources
  none

catalog entry  (~189 tokens, always in context)
<available_skills>
  <skill>
    <name>incident-response</name>
    <description>Triage and mitigate production incidents affecting customer-facing services. Use when alerting has already fired or a customer-visible degradation is confirmed. Not for local development failures, test environment issues, or routine scheduled maintenance.</description>
    <version>1.0.0</version>
    <when_to_use>
      <case>Production outage or confirmed customer-facing degradation</case>
      <case>Severe incidents requiring escalation to on-call</case>
    </when_to_use>
    <when_not_to_use>
      <case>Local development or test environment failures</case>
      <case>Routine scheduled maintenance</case>
    </when_not_to_use>
  </skill>
</available_skills>

body  (~231 tokens, loaded on demand)
# Incident Response

## Declaring an incident

Declare an incident when a production service is degraded or unavailable for
users, when data integrity may be compromised, or when a security breach is
suspected...

The split is the whole point. Those ~189 tokens are paid on every single turn, whether or not incidents come up. The ~231-token body is paid only when the agent, reading that catalog entry, decides the skill is worth opening. If it never opens it, the description is why.

I'd recommend doing this at least once, because the catalog entry is rarely what people picture. It's assembled from your frontmatter rather than being the description as written. Reading it in the form the model receives has a way of exposing a weak description that staring at the YAML does not.

It also answers the first question for free. If the skill isn't in the output at all, you have a registration problem, not a description problem, and everything below is the wrong fix.

Two failures, not one

The obvious failure is the skill that never fires. Easy to notice once you're looking for it, easy to care about.

The one people miss is the opposite: a skill that fires when it shouldn't.

A description that's too broad gets the skill loaded on requests it has nothing useful to say about. You pay the body cost every time. Worse, the model now has instructions in front of it that don't fit the task, and that pushes it toward a narrower, more procedural answer than it would otherwise have given. Confident, well-structured, wrong.

It's more common than I expected and completely invisible if you're only reading outputs, because the answers with a badly-matched skill attached tend to look better. They're structured, they cite process, they read like something an expert wrote. They're just less correct.

So a description has two jobs: get chosen when it should be, and not get chosen when it shouldn't. Most description advice only addresses the first, which is exactly how you end up with descriptions that are broad, keyword-heavy, and quietly harmful.

Write the boundary into the description

The specification says description is where you set those boundaries, and it documents a loop for tuning it against the queries your agent got wrong.

I want to be clear that this works. I spent a while convinced it needed extra machinery, and I was wrong. Most of my activation problems were fixed by writing a better description.

Concretely, here's a description that misfires:

Helps with production incidents, outages, debugging, and service issues.

Keywords, no boundary. That fires on "my dev container won't start," on "why is this test flaky," on anything containing the word "debug." Every one of those loads a runbook written for customer-facing outages and gently steers the model toward paging an on-call engineer about a Docker problem.

Here's the same skill, rewritten:

Triage and mitigate production incidents affecting customer-facing services. Use when alerting has already fired or a customer-visible degradation is confirmed. Not for local development failures, test environment issues, or routine scheduled maintenance.

Three sentences: what it does, when it applies, when it doesn't. That third sentence is the one most people leave out, and it's the one that does the most work.

Note that this is longer, and lint from part 2 will start warning past 500 characters. That tension is real, and the warning is right to exist, because you pay for the description on every turn. But a longer description that gets the decision right beats a short one that doesn't. I'd take the warning over the misfire.

The method that actually works

It isn't clever, but it works: keep the near-misses.

Every time the agent picks wrong, in either direction, write down the query word for word. Not your paraphrase of it. The actual words.

After two weeks you'll have a list, and the list tells you exactly what your description is missing. In my case it was almost always vocabulary. I'd written "production incident." Users typed "the site is down," "customers are complaining," "checkout is broken." Not one of them used my words. I would never have guessed that from my desk, because I wrote the description and every word in it felt obviously correct to me.

Edit against the list, not against your intuition. Your intuition wrote the version that's currently failing.

The optional fields, and why I'm hedging

The SDK supports structured selection metadata:

---
name: incident-response
description: Triage and mitigate production incidents affecting customer-facing services.
when_to_use:
  - Production outage or confirmed customer-facing degradation
  - Severe incidents requiring escalation to on-call
when_not_to_use:
  - Local development or test environment failures
  - Routine scheduled maintenance
---

Five entries per field, 200 characters each, enforced by validate.

Now the honest part. These fields are an addition of my own, not part of the specification, and I have no evidence that they improve activation over a well-written description.

So, there is one concrete reason to use these fields today, and it isn't activation. The retrieval selector uses them for scoring: when_to_use counts in favour, when_not_to_use counts against at half weight. That matters once your library is too big to show the agent every skill at once.

Until then: write the description properly. If it's still misfiring after you've worked through your near-miss list, then reach for these.

Splitting instead of tuning

One more option that's often better than either.

If you find yourself needing four or five separate conditions to describe when a skill applies, the skill is probably two skills. That's why validate caps the selection fields at five entries and says so in the error text. The limit is there to make you ask the question.

Two skills with sharp, narrow descriptions get chosen correctly far more often than one skill trying to cover both. Grouping related guidance into a single document is a good habit when humans read a wiki. It's a bad habit here.

What none of this tells you

Suppose you do all of it, and now the skill fires exactly when it should.

You still don't know whether it helps.

It's entirely possible to have a well-described, correctly activating skill that makes your agent no better than it was. The model already knew how to do the task, and your skill is 800 tokens restating something it could do anyway.

Activation and usefulness are two different things. Everything in this post buys you the first one, and it's worth being honest that it buys you none of the second.

Everything here is in agentskills-sdk v0.5.0, on GitHub.