Match your coding-agent question to a bounded Stet workflow
Each workflow changes one meaningful harness surface at a time, uses a matched
task slice, and returns a Trial Result you can interpret before deciding what to
keep. Choose the question that matches the decision in front of you.
Use a manifest-backed A/B comparison when you have a proposed instruction change
and want the model and other relevant settings held fixed.Recommended prompt
Use the Stet skill to A/B test my proposed AGENTS.md change. Keep the modelfixed, plan before launch, and return the canonical Trial Result with itsrecommendation, evidence limitations, and next action.
Evidence expectation: A credible result compares baseline and candidate on
the same retained repository tasks, with the intended instruction lever
isolated and the required tests and graders present. A quick smoke can guide
iteration but is not a rollout decision.
Use the iterative loop when you want the agent to propose bounded instruction
edits and measure them one at a time. Set a search space and stop rule before
launching; holdout evidence and promotion remain explicit decisions.Recommended prompt
Use the Stet skill to improve AGENTS.md within a bounded search space and stoprule. Test one change at a time and ask before using holdout evidence orpromoting a finalist.
Evidence expectation: Each iteration should name the single changed lever,
preserve the task and grader contract, and stop when the result is inspect-only,
not improving, or otherwise unable to support the next decision.
Evaluate a skill against the right baseline: an effectively absent skill for a
new skill, or the committed skill for a revision. Keep repository work and
behavior-relevant graders matched.Recommended prompt
Use the Stet skill to test whether this repo-managed skill helps. Plan beforelaunch, use the right baseline, and report promote, hold, or inspect.
Evidence expectation: The baseline must actually represent skill absence or
the committed version, and the result should include behavior-relevant grader
coverage. Missing coverage turns a comparison into an inspectable diagnostic,
not a keep decision.
Compare models on the same retained tasks, or hold the model fixed while
comparing reasoning settings. Keep the rest of the harness stable enough to
attribute the observed difference.Recommended prompt
Use the Stet skill to compare these configurations on the same repositorytasks. First ask whether I want a cheap diagnostic read or a gateable rolloutdecision. Keep diagnostic evidence labeled directional; for a gateable choice,use matched runs and return the canonical Trial Result.
Evidence expectation: Read correctness, quality dimensions, cost,
uncertainty, validity, and residual risk separately. Do not choose a default from
stale, partial, incomparable, or inspect-only evidence.