Honest note: no paired measurements yet. What follows is an executable experiment design plus expectations derived from structure, not results. The numbers go in once both groups have three runs each.
Why the two shapes diverge follows from structure:
| Task shape | native | ptc | Expected |
|---|---|---|---|
| Batched repetition (one action across 50 files) | 50 tool calls, 50 full round trips | One loop, one run_code, one round trip | PTC clearly cheaper |
| Small job (read 2 files, summarise) | 2 calls, 2 round trips | Still generates the SDK prompt, then a program: saves one round trip, pays a fixed prompt cost | Roughly break-even, maybe slightly worse |
| Each step depends on the previous result | The model inspects each result before deciding | Branches must be written in advance; judgment thins out | PTC not worth it |
| Independent, parallelisable operations | Serial calls | The program can overlap them (maxParallelSubCalls, default 10) | PTC may be faster |
Designing the experiment
- Batch group: 50 structurally identical files, one action (take the first 100 characters of each). Three native runs, three PTC runs.
- Small group: two files, one summary. Three runs each.
- Record only: tool calls, round trips, input/output tokens, whether the output is byte-identical.
- Log time but do not make it the lead metric — network variance swamps the signal.
# batch group (50 files) Read all 50 markdown files under files/, take the first 100 characters of each, and compile table.md # small group (2 files) Read a.md and b.md and merge them into summary.md
One variable people miss
PTC has a maxParallelSubCalls setting, default 10, capping concurrent sub-calls within one run_code program. The comments say it follows native's scheduling contract: only tools classified as concurrency-safe overlap, exclusive calls act as barriers, and setting it to 1 restores strict serial dispatch.
So the comparison is not 'one mode versus another' but 'one mode versus a mode with a tunable knob'. Any write-up must state the maxParallelSubCalls value, or the numbers cannot be reproduced.