AI Guides › Skills & Agents

Building A Skill From A Support Ticket Pattern, Not A Hypothetical

By Nigel Guy · 2 min read

A lot of skills get built from an imagined version of a problem — "people probably struggle with X" — rather than from evidence that X is actually what's happening. The imagined version is faster to start from and almost always wrong in some detail that only shows up once real cases arrive, by which point the skill's already built around the wrong shape of the problem.

The mechanism: the Ticket-to-Skill Pipeline

1. Pull a real sample, not a memorable handful. Recall bias makes the three tickets you remember feel like the pattern. Pull an actual set — the last month, the last fifty, whatever's practical — before deciding what the pattern is.
2. Group by what was actually asked, not by category label. Support categories are often broader than the real pattern inside them. Read the tickets themselves and group by the specific recurring question or action, which is frequently narrower than the label suggests.
3. Write the skill's job as the resolution to the most common group, specifically. Not "helps with billing questions" — "handles the specific case where someone's asking why they were charged twice after a plan change," if that's what the tickets actually show.
4. Pull the exact language customers used, and keep it. The skill's trigger condition should be built from how people actually phrased the problem, not from how you'd phrase it internally — those are often different enough to matter.
5. Test the skill against a held-out set of tickets it hasn't seen. Not the sample you built it from — a separate batch, to check the pattern generalises rather than just fitting the specific tickets you read.

What to skip

Skip building from a hypothetical edge case someone raised in a meeting, however plausible it sounds, until you've checked it against real tickets and found it's actually common — plausible and common are not the same thing, and skills built for the former often sit unused while the real, less obvious pattern goes unaddressed. Skip building the skill around the single worst ticket you've seen; the worst case and the common case are usually different problems, and optimising for the former can leave the latter, more frequent one, no better handled than before.

Guardrails

• A pattern in past tickets isn't a guarantee it'll keep recurring the same way — check back after the skill's been live a while to confirm the pattern held, not just that it existed historically.
• This mechanism needs an actual ticket sample to work from. Where none exists yet — a new product, a new support line — you're building from a hypothesis by necessity, and should treat the resulting skill as unverified until real tickets can confirm or correct it.
• Watch for tickets that were miscategorised or resolved by a workaround rather than a real fix — those can inflate a pattern that isn't actually what it looks like on the surface.

All 751 AI guides · JulieMango plans from £17/mo