Run the same task twice, and lay the bill open
Controlled experiments, raw data, no hidden failures.
Each lab has six fixed sections: question, control, environment, raw data, conclusion, and what it can't generalize to. The last one is mandatory—one experiment doesn't cover every scenario.
The community has seen opposite conclusions (same mode comparison—cheaper vs. pricier depends on the task shape). We don’t presume a conclusion, and we don’t write numbers without raw records.
What we are adding
Claim one and submit the raw records by the template.
- Lab / ModeStandard vs PTC: same task, which saves moreClaim in progress
- Lab / PluginContext plugins: do they actually save tokensAwaiting claim
- Lab / VersionUpgrade drill: what breaks, and how long a rollback takesAwaiting claim
- Lab / ModelLocal model vs API model: quality and time on the same taskAwaiting claim
- Lab / ModeStreaming vs one-shot generation: difference on the same taskAwaiting claim
- Lab / PromptSame task, Chinese vs English prompts: how much differenceAwaiting claim
- Lab / ContextBefore vs after context compaction: how much info is lostAwaiting claim
- Lab / ToolTool-call failures: retry-strategy comparisonAwaiting claim