Claude Code plugin Skill vs project Skill: 432 runs

Is a Claude Code Skill triggered less often when it comes from a plugin than when it sits in .claude/skills/? I ran the same Skill both ways, 432 times through claude -p. On Opus 5.5 there was no difference at all: 46 of 48 runs each. On Sonnet 5.5 the plugin copy was invoked as often as the project copy on plain questions (16 to 19 of 24), and lost only on prompts that start with “Check my notes.”: 5 to 8 of 24 against 18. Two things made it worse: a plugin named like the MCP server next to it, and a plugin that ships its own MCP server.

This is the seventh post on agent memory and MCP and round three of the Skill vs MCP series. Round one put the same notes search behind an MCP server and a Skill. Round two found that the Skill shipped in a plugin ran the search in 10 of 48 runs against 24 for the project copy, and ended with “I can’t say why from these runs”. This post tests the likely reasons one at a time.

Why would a plugin Skill be picked less?

From round two I had four suspects:

1. The name. My plugin was called notes, the same as the MCP server it competed with. The Skill showed up as notes:search-notes and the server’s tools as mcp__notes__search_notes. Maybe Claude read the server as the plugin’s real tool.
2. The namespace. Any plugin Skill gets a plugin: prefix. Maybe the prefix alone makes it look less like the answer to “search my notes”.
3. The listing. The project Skill may sit higher in the list Claude sees.
4. Where the server lives. A plugin can ship an MCP server too. Then server and Skill come from the same place.

The first is easy to isolate: rename the plugin. The third and fourth get their own conditions. The second I can only see as what is left over.

How I tested it

The pieces are the ones from rounds one and two: a BM25 search over seven posts of this blog, exposed as a stdio MCP server (search_notes, read_note) and as a Skill whose SKILL.md tells Claude to run a small Node script through Bash. Both import the same module, so the answers are identical. New this round, three plugin folders with the same Skill inside:

plugins/
  notes/          .claude-plugin/plugin.json  {"name": "notes", ...}   -> notes:search-notes
  kb/             .claude-plugin/plugin.json  {"name": "kb", ...}      -> kb:search-notes
  kb-with-mcp/    same as kb, plus the MCP server in .mcp.json

The bundled server is declared the way the plugin docs show, with the plugin root as a variable:

{
  "mcpServers": {
    "notes": {
      "command": "node",
      "args": ["${CLAUDE_PLUGIN_ROOT}/server.mjs", "${NOTES_DIR}"]
    }
  }
}

Each of the 12 questions went to claude -p (Claude Code 2.1.318) in two phrasings, hinted (“Check my notes. …”) and bare, under 9 conditions, twice: 432 runs in about 85 minutes. Plugins were loaded with --plugin-dir. For every run I recorded which side ran the first search (an mcp__ call or the Skill’s script), whether the Skill tool was called, and whether the answer matched. As in round two, read-only file tools and shell commands like grep and cat were denied, so only the two search paths could find anything. No CLAUDE.md, no auto-memory.

Does a plugin Skill get triggered less than a project Skill?

On Opus, no. On Sonnet, a little, and the gap has a shape:

MCP server + script Skill (48 runs each)	First search: MCP	First search: Skill’s script	No search	Skill invoked
Sonnet, Skill in the project	19	25	4	36
Sonnet, Skill in plugin notes	27	16	5	21
Sonnet, Skill in plugin kb	20	23	5	27
Opus, Skill in the project	0	46	2	46
Opus, Skill in plugin notes	0	46	2	46
Opus, Skill in plugin kb	0	46	2	46

Opus picked the Skill in 46 of 48 runs whatever its source, as it did in round two with the project copy. The 2 runs without a search were the same bare question in every Opus row (Ollama’s default context size), answered right from memory.

On Sonnet, round two’s gap (24 against 10 script searches on 2.1.310) was smaller this time on 2.1.318: 25 against 16 for the same notes plugin. Both repetitions agreed (7 and 9 for the plugin, 12 and 13 for the project), so I trust the direction, not the exact size. Split by phrasing, it looks like this:

Sonnet, Skill invoked (of 24)	“Check my notes. …”	Bare question
Skill in the project	18	18
Skill in plugin notes	5	16
Skill in plugin kb	8	19

On bare questions the three copies were invoked about equally. The whole gap sits in the hinted prompts. There, Sonnet usually called ToolSearch for the notes server’s tools and searched with them right away (24 of 24 hinted runs with the notes plugin started with MCP). With the project copy it called the Skill first in 18 hinted runs, and then still searched with the MCP tool in 11 of them.

So the plugin Skill is not hidden or broken. It is just a weaker match when the prompt already points at “notes” and an MCP server called notes is right there.

Does the plugin name matter?

A bit. Renaming the plugin from notes to kb moved Sonnet from 16 to 23 script searches and from 21 to 27 Skill calls (repetitions: 7 and 9, then 12 and 11). That closes most of the gap on the whole set, but not on hinted prompts (5 to 8 of 24, against 18 for the project). The shared name explains part of round two’s result, not all of it.

What is left is suspect 2 or 3: the kb: prefix or the order. In the skills array of the init event, the project Skill came first (position 0), and the plugin Skill came after one of Claude Code’s own skills (position 1). I can’t separate those two with a plugin, because a plugin Skill always has both the prefix and the later spot.

What happens with the same Skill in the project and in a plugin?

The project copy wins every time:

Sonnet, MCP server + both copies	First search: MCP	Project script	Plugin script	No search
48 runs	8	36	0	4

The project Skill was invoked 36 times, the plugin Skill 0. Both were listed (search-notes at position 0, kb:search-notes at position 2). Having two Skills for notes also pulled Sonnet away from the MCP server: 8 MCP-first searches against 19 with only the project Skill. If you vendor a plugin’s Skill into your project, uninstall the plugin copy: it adds 47 tokens to every prompt and is never used.

Does a plugin with its own MCP server change the pick?

Yes, toward the server. With the server inside the kb plugin (tools named mcp__plugin_kb_notes__search_notes and mcp__plugin_kb_notes__read_note) and no outside server:

Sonnet, plugin kb (48 runs each)	First search: MCP	First search: Skill’s script	Skill invoked	Right
Skill + its own MCP server in the plugin	39	5	5	48
Skill only, MCP server from outside (control)	23	21	27	48

The control row needs a word. Claude Code’s --strict-mcp-config flag, which I use in every benchmark so only my server loads, also drops MCP servers that come from plugins. With it, the init event listed no server at all and the plugin’s .mcp.json was ignored without a warning. So the bundled-server condition had to run without the flag, and then the claude.ai connectors of the logged-in account load as well (four of them, one connected). The control is the outside-server condition run the same way. It landed where the strict run did (23 and 21 against 20 and 23), so the connectors did not move the result. The bundled server did: 39 of 48 first searches, and the Skill was invoked only 5 times.

One more detail from those runs: in 7 of the 48, Sonnet called the plugin’s search_notes with no query and got Claude Code’s error “This tool’s input schema was not in your prompt when you called it”, then retried correctly. That error never happened with the outside server.

How do you make Claude Code use a plugin Skill?

From these runs:

• Don’t name the plugin after an MCP server that does the same job. notes next to a notes server cost 7 of 48 script searches on Sonnet.
• Don’t ship a Skill and an MCP server for the same job in one plugin. Sonnet takes the server. If the plugin needs both, make the Skill’s body point at the server’s tools, the “bridge” pattern from round two, which got the MCP tool used in 133 of 144 runs.
• Keep one copy. A project Skill with the same name as a plugin Skill takes every call.
• Write the description for the prompts users type. The plugin copy only lost on prompts that named “notes”, which matched the MCP server’s name as well as the Skill’s description.
• Test on the model your users run. Opus showed none of this.
• Scripting claude -p with plugins? --strict-mcp-config silently turns off plugin MCP servers. Put the server in your --mcp-config file or drop the flag.

The idle cost is small either way. A one-turn prompt with no tools used took 17,047 input tokens with nothing installed, 17,090 with the project Skill, 17,094 with the plugin Skill (either name) and 17,137 with both.

What broke and what I’d do differently

• My first smoke test had a broken import in the copied Skill script: every plugin search failed with ERR_MODULE_NOT_FOUND and Claude fell back to MCP, which looked like a plugin effect. I caught it because the runner records error texts now. Check that each copy of a Skill runs before measuring it.
• I could not test the plugin: prefix apart from the listing position. Both change together when a Skill moves into a plugin.
• The bundled-server condition differs from the others in one more way: no --strict-mcp-config. The control row covers the connectors, not every possible side effect of the flag.
• Three answers of 432 were wrong, all bare questions with no search: Sonnet answered from memory once and said it couldn’t find the answer twice.
• Small sample: one corpus of seven files, 12 questions, two repetitions per condition. Every row agreed within 2 runs between repetitions. Read the counts as directions, not rates.
• Claude Code reported USD 19.97 at API prices for the 432 runs (USD 9.90 of it Opus), plus 15 short context-cost runs. The runs used a subscription.

Everything is in the project folder: the three plugins, the conditions, the script that drives claude -p, and every run with its tool calls, errors and answers. If you are building the MCP server side, the 100-line MCP server post is the starting point.