Deleting half a skill and proving nothing broke
Shortening two caliper skills, grill-skill and evaluate-skill, by about 45% using a non-inferiority eval bar: prove the shorter version is no worse before shipping the cut.
Tagged
skills
3 posts
Teaching an agent to stop sounding like an agent
A case study in using Caliper to evaluate blader/humanizer, tighten voice calibration, and turn the improvement into an upstream contribution with regression coverage. With a postscript on what the maintainer did instead.
Skills without evals are just vibes
Tribal knowledge encoded as an AI skill is still just text until you evaluate it. Ablation baselines, routing regression tests, trajectory autoraters, and the gotchas flywheel keep encoded knowledge from rotting.
