Testing and Improving a Skill

A skill isn't finished when you write it. It's finished when it works on examples you didn't write it for. Build a small test set: five to ten real inputs, including a few awkward ones, such as missing information or an unusual request.

Run each input and score the output against a short checklist: correct, complete, in the right format, nothing invented. Note every failure and what caused it. Most failures trace to a missing instruction, a vague rule or an unclear description.

The Improvement Loop

• Change one thing in the skill at a time.
• Rerun the whole test set, not just the case that failed.
• Keep a log of what you changed and what happened.
• Stop when it passes your checklist on all inputs, then add two new awkward inputs and see if it still holds.

Exercise

• Build a test set of six inputs for your skill, including an injection input.
• Run it and log the results.
• Make one change and rerun all six.
• Pass condition: a log with at least one change and a rerun result, the injection input did not change the output, and all six inputs pass on the final rerun.