Before vs after context compaction: how much info is lost
Experiment design set, results await raw records
This page gives only the experiment design, not the results. Numbers we haven’t run are never written; paste raw records into the table after running, then write the conclusion.
Question to answer
After a long session triggers context compaction, how do early-detail recall and downstream-task accuracy change?
Control groups
Pre-compaction baseline / post-compaction measurement; verify with the same set of 'early facts' via follow-up questions.
Environment & version
- Host version
- To fill
- Model ID
- To fill
- Mode / permissions
- Keep both groups identical
- Input files
- Record hashes so both runs use the same files
Fields to log
| # | Field | Result |
|---|---|---|
| 1 | Host and compaction plugin | to be tested |
| 2 | Session length (tokens) | to be tested |
| 3 | Early-fact recall after compaction | to be tested |
| 4 | Downstream-task accuracy | to be tested |
| 5 | Tokens saved by compaction | to be tested |
Conclusion
To be tested. Any conclusion must also state what it can't generalize to.
Why we don’t write results yet
Compaction plugins like billion-context claim retrievable details; this site records measured recall, not vendor phrasing.