The saving grace of Claude Code skills is that when writing them yourself, you can give them frontmatter like "use when mentioning X" that makes them become relevant for very specific "shibboleths" - which you can then use when prompting.

Are we at an ideal balance where Claude Code is pulling things in proactively enough... without bringing in irrelevant skills just because the "vibes" might match in frontmatter? Arguably not. But it's still a powerful system.

> idea that agent harnesses should primarily have their functionality dictated by plaintext commands feels like a copout

I think it's more along the lines of acknowledging the fast-paced changes in the field, and refusing to cast into code something that's likely to rapidly evolve in the near future.

Once things settle down into tested practices, we'll see more "permanent" instrumentation arise.

I view them as more idiosyncratic docs, but focused on how to write code (there is so much huggingface code floating around the internet, the models do quite well with it already).

I have not had much success with skills that have tree based logic (if a do x, else do y), they just tend to do everything in the skill (so will do both x and y).

But just as "hey follow this outline of steps a,b,c" it works quite well in my experience.

I think unless you're doing simple tasks, skills are unreliable. For better reliability, I have the agent trigger APIs that handles the complex logic (and its own LLM calls) internally. Has anyone found a solid strategy for making complex 'skills' more dependable?

You can publish scripts with skills you author, right? With carefully constructed markdown that should allow the agent to call tools the right way.

Are you using either CLAUDE.md or .claude/INSTRUCTIONS.md to direct Claude about the different agents?

Also, be aware that when you add new instructions if you don't tell claude to reread these files, it will NOT have it in its context window until you tell it to read them OR you make a new CC session. This was a bit frustrating for me because it was not immediately obvious.

https://scottspence.com/posts/measuring-claude-code-skill-ac... works very well

> sometimes it just continues to go into the next step

Use a structured workflow that loops on every task and includes a pause for user confirmation at the end. Enforce it with a hook. I'm not sure if you can toggle auto-accept this way, but I think the end result is what you're asking for.

I use this with great success, sometimes toggling auto-accept on when confidence is high that Claude can complete a step without guidance, and toggling off when confidence is low and you want to slow down and steer, with Claude stopping between the steps. Now that prompt suggestions are a thing, you can just hit enter to continue on the suggested prompt to continue.

Behavior trees. They are precisely what we need. Somebody just needs to go build the damn thing.

You can write skills that have an associated js/python/whatever script.

> Skills in CC have been a bit frustrating for me. They don't trigger reliably

Referencing them in AGENTS/CLAUDE.md has increased their usage for me.

I think the paper is saying specifically that it's redundant to include information about your coding repository when that information is otherwise available to the agent in higher fidelity forms (e.g. package.json). This makes sense - but not sure it's about Skills directly.

For the former I'd be interested in learning more about that. From a harness perspective the difference would be the inclusion of the description in the system prompt, and an additional tool call to return the skill. While that's certainly less efficient than adding the context directly I'd be surprised if it degraded task performance significantly.

I tend to be quite focussed with my Skill/Tool usage in general though, inviting them in to context when needed rather than increasing the potential for model confusion.

Most LLM "harnessing" seems very lazy and bolted on. You can build much more robustly by leveraging a more complex application layer where you can manage state, but I guess people struggle building that

> So far my experience with skills is that they slow down or confuse agents unless you as the user understand what the skill actually contains and how it works. In general I would rather install a CLI tool and explain to the agent how I want it used vs. trying to get the agent to use a folder of instructions that I don't really understand what's inside.

For Claude Code I add the tooling into either CLAUDE.md or .claude/INSTRUCTIONS.md which Claude reads when you start a new instance. If you update it, you MUST ask Claude to reread the file so it knows the full instructions.

I mean, yes. You should do exactly that: instruct an agent on how to do something you understand in terms you can explain.

Putting that in a `.md` file just means you don’t need to do it twice.

Yes -- skills live in a special gap between "should have been a deterministic program" and "model already had the ability to figure this out". My personal experience leaves me in agreement that minimal system prompts are definitely the way to go.

See 'genetic programming' for techniques that are sort of based on this idea. Typical approach is to have a problem representation (gene analogues) that can be used to create a population of different individual solutions. Test them all against a fitness function and retain those that are 'best' according to some metric. Then create (breed) some new individuals who have some of the characteristics of the winners, perhaps mutated somewhat, insert these into the population. Repeat until you have solved the problem or have a good enough solution.

Challenges (apart from the time taken) are coming up with a good enough gene representation that captures the essence of the problem, building an efficient fitness function, and avoiding local maxima - i.e. a solution that is almost but not quite good enough, but from where you can't breed a better solution.,

Same experience here.

Documentation-based skills don’t really work in practice. They tend to waste tokens instead of adding value.

CLI skills are also redundant when the CLI already provides clear built-in help messages. Those help messages are usually up to date, unlike separate skills that need to be maintained independently.

If the CLI itself is confusing (and would likely be confusing for humans as well) then targeted skills can serve as a temporary workaround, a kind of band-aid.

Where skills truly shine is when agents need to understand non-generic terms and concepts: unique product names, brand-specific terminology, custom function names, and other domain-specific language.

Skills are only loaded when you need them, so you’ll probably use fewer tokens overall compared to MCP servers or including them manually in your main AGENTS.md/CLAUDE.md file, which are always loaded in the system prompt.

> is there a mechanism to pin a version, or is it always HEAD? Skills that evolve can silently break downstream workflows.

don't forget these skills are just text that goes into the llm for it to read, interpret, and then produce text that then gets executed in bash. The more intricate and specific the skill definition the more likely the model is to miss something or not follow it exactly.

See the discussions at https://github.com/agentskills/agentskills/discussions

For example, on Proposal: AgentFile — Declarative Agent Composition from Skills + Filesystem-Native Skill Delivery

https://github.com/agentskills/agentskills/discussions/179

I actually think SKILLS.md is such a janky way of doing this sort of thing, let alone the fact that's reliant on the oh-so-brittle Python ecosystem. Also way too much context/tokens being eaten up by something that could be piece-wise programmatically injected in the token stream.

Imo a bad idea, but alas.

I think you are spot on there, and I am not sure such things exist (yet), but I may be wrong. Some random thoughts:

1. Using the skills frontmatter to implement a more complex YAML structure, so e.g.

  requires:
     python:
       - pandas>=2.1
     skills:
       - huggingface-cli@1.x

2. Using a skills lock file ;-)

  skills.lock

Anthropic made it an open standard: https://agentskills.io/home