| Node | Comparison design | Control versus variable |
|---|---|---|
| Using the four presets | Tool surfaces of the four presets | Different preset, same task |
| PTC versus standard | Same task three times each | Different presentation, same task |
| Context and compaction | Same follow-up before and after | Whether compaction happened |
| Instruction structure and placeholders | One-liner versus full recipe | Different instruction structure |
| Glossary | No comparison, sources only | — |
| Context index | (this page) | — |
Shared requirements in this group
- Same task: comparison requires identical tasks including output format. Fix the format or you cannot compare.
- One variable: change the preset or the instruction, not both.
- At least three runs: a single result is dominated by variance.
- Record three numbers: tool calls, model round trips, and whether outputs are byte-identical. Time is reference only.
The easiest way to fake this group is comparing while outputs differ. If the outputs differ, it was not the same task and every number downstream is void. So step one of any experiment log is: confirm the two outputs are byte-identical.