Cutting a skill in half without making it worse
Shortening two caliper skills, grill-skill and evaluate-skill, by ~45% using a non-inferiority eval bar: prove the shorter version is no worse before shipping the cut.
agents evaluation skills
Tagged
#agents
A small eval loop for the humanizer skill
A case study in using Caliper to evaluate blader/humanizer, tighten voice calibration, and turn the improvement into an upstream contribution with regression coverage.
agents evaluation skills writing
Skills without evals are just optimism
Tribal knowledge encoded as an AI skill is still just text until you evaluate it. Ablation baselines, routing regression tests, trajectory autoraters, and the gotchas flywheel keep encoded knowledge from rotting.
agents evaluation skills
Why traditional access control fails with agents
Everyone is worried about AI reading things it shouldn't. That's the wrong threat model. The problem starts after the agent reads.
ai-safety agents security access-control