Measured numbers, results before method:
| Batch | Total | OK | Failed | Time | Per file |
|---|---|---|---|---|---|
| Full run | 100 | 80 | 20 | 2.46 s | 25 ms |
| First 10 | 10 | 8 | 2 | about 0.25 s | 25 ms |
The key finding: two of the first ten failed, already exposing a 20 percent failure rate. You do not need the full hundred to know the data has a problem. That is the entire value of trying ten — not saving time, but saving the discovery that the direction was wrong.
Check the equation
A batch job must satisfy exactly one equation:
total = successes + failures
It sounds trivial, but it is the only check that catches silent skipping. Here 80 + 20 = 100 holds. If your script says it processed 78, two files vanished — worse than failing, because you do not know which.
How to phrase the recipe
Process the first 10 PDFs under pdfs/, extracting the first 200 characters of body text from each. Output: - A table with columns: filename, character count, first 200 characters - A final line: total / ok / failed / list of failures Requirements: files with no extractable text must still appear, marked 'read failed'. Do not skip them. Do only these 10 first; I will confirm the format before you continue.
That last sentence is the heart of it: one all-or-nothing gamble becomes two small confirmations.
What to change when scaling
- Change only the range (first 10 to all). Do not change the output format — changing it throws away the confirmation you did.
- If the full run must be batched, state the batch boundaries and how results merge.
- First thing after the run: check the equation, then argue about content.