Last month I lost most of a Friday to a "battle-tested" Claude Code skill someone dropped in a Discord. Four hundred stars. The README had a diagram. It also swallowed every error from the API it wrapped, so when that API started returning 429s, my agent kept right on going. Confidently. Writing garbage to a real table I had to clean up by hand.
That's a slop skill. And catching one before it touches anything that matters has quietly become part of the job. There are thousands of these now, and they all look great in a screenshot.
Here's the checklist I run before I trust an AI agent skill, an MCP server, or a Cursor rule. Not theory. The stuff I wish I'd checked before that Friday.
What a slop skill actually is
A slop skill performs competence it doesn't have. It's built to get adopted, not to survive contact with production. Good name, confident README, a demo gif where the sun is always shining. Then it folds the second reality stops being the happy path.
The tell is the gap between how polished it looks and how little it's actually been hurt. Code that's run in production for a while has scars in it: weird guard clauses, a comment that says "don't remove this, learned the hard way," a retry someone clearly added at 2 a.m. Slop has none of that. It has good copywriting.
You see it across Claude Code skills, Cursor rules, MCP servers, agent templates. Anyone can publish one in an afternoon, and a model will happily write a convincing README to go with it. The supply went vertical. The quality didn't move.
How to vet an AI agent skill: the 5-point check
I don't read the whole thing top to bottom. I go hunting for specific smells, in this order.
1. Does it handle the unhappy path? I grep for error handling before I read any of the main logic. No try/catch, no error branches, no retries, no timeout? Then it was written against a demo and tested against a demo. A skill that can't tell the model "this failed, here's why, stop" will let your agent hallucinate straight through the failure. That's the dangerous kind. It doesn't crash. It lies, and it sounds great doing it.
2. Does it tell the model when NOT to act? Almost nobody checks this. A good skill bakes boundaries into its instructions: when to bail, when to ask first, what it must never touch. Slop only describes the thing it does well. There's no "if you're unsure, stop." Hand that to an autonomous agent and you've given a power tool to something with no idea where its own hands are.
3. Are inputs validated or just assumed? Find where data comes in. Is anything checked? Slop assumes the model always returns clean JSON, the file always exists, the field is always populated. Production is exactly where that field comes back null while you're asleep.
4. What's the last commit, and what was it? A skill untouched for eight months, sitting on top of tooling that ships breaking changes monthly, is a fossil. Read what the commits actually say, too. All "update README" and no "fix edge case"? Nobody's running it for real. Real use generates real bug fixes. The commit log will tell you the truth the README won't.
5. What's the blast radius? Dumbest question, biggest payoff. What can this thing actually touch? A skill that claims to format your commit messages but quietly holds write access to the whole repo plus network egress is a liability in a helpful costume. Scope should match the job. When it doesn't, that's the whole story right there.
Most of this is fast. A handful of greps and a glance at the commit history catches the obvious junk before I've sunk real time into it.
The part that actually costs you
Here's the thing about vetting skills properly. The auditing isn't hard. It's the volume that wears you down, and it doesn't stop.
You're never checking one. You're picking between nine that all claim to do the same job, and they're scattered. GitHub. Two different Discords. A couple of "awesome-skills" lists that went stale in spring. A Reddit thread. Somebody's blog post. Each candidate wants the grep, the commit sniff, the scope check, the issues tab to see if anyone already hit the bug you're worried about. Each one is a fresh pile of tabs.
I tried to actually account for one. I needed a single decent skill for one MCP integration. Finding the candidates, opening each repo, reading the suspicious bits, checking issues, working out which fork was the live one and which was abandoned. I counted somewhere around nine hours across two days. For one slot in my toolchain.
And the answer goes stale. The skill I picked was solid in May. Is it still? No idea, and nothing tells me. Stars are a popularity vote frozen in the past. Download counts don't know whether the thing broke last Tuesday when the upstream API changed its response shape. There's no signal anywhere for "this one started failing this week." So I'll do the whole audit again next quarter, from scratch.
Every builder I've talked to about this describes the same routine. Open it, grep it, squint at the commits, hope. On their own. Nobody's notes carry over to anybody else. We're each privately rediscovering, one wasted Friday at a time, that the same popular skill quietly eats its errors.
The takeaway
The checklist works. Error handling. Stop conditions. Input validation. The truth in the commit log. Blast radius. Run those and you'll dodge most of the wreckage. Tape them above your monitor if you want.
But run them for a few months and the real problem surfaces. The looking isn't the expensive part. The hunting is, and the fact that you do it alone, every time, against a list nobody keeps current.
A skill that looks production-ready and one that is are about ten minutes of reading apart. The catch is that almost nobody has ten clean minutes and a trustworthy place to start.
Check the blast radius first. It's the cheapest question, and it's the one that's saved me when none of the others did.