Skip to content
MyDSHMy DSH Field Notes
中文
Independently maintained; not affiliated with DeepSeek. No association with mydsh.dev or other MyDSH sites.

All categories/Data and analysis

Data extraction and SQL · Level Advanced · SQL SQL Backfill

Backfill script: several days missing

A failed job left three days missing, and the table is overwritten in place.

Prompt

You state the question before choosing tables. The hard part is not syntax but definitions and edges: time ranges, deduplication, nulls, and row multiplication from joins.

Task: A failed job left three days missing, and the table is overwritten in place.

[Before you start, do three things]
1. Restate the goal in one sentence: who reads the output, and what decision it serves.
2. If I have not given you the following, ask me. Do not assume:
   - the missing date range
   - how the table is written
   - downstream dependencies
3. Say how you plan to do it (at most 5 steps) and wait for my go-ahead.

[Go through each of these]
- back up before backfilling
- backfill must be re-runnable with identical results
- compare row counts against adjacent days after the run
- state whether downstream reports need a refresh

[Process]
1. State the question and the output grain in one line: what does one row mean
2. Confirm table structure, keys and cardinality before joining
3. Write the SQL with a comment on the business meaning of each filter
4. Self-check: dedup, time boundaries, nulls, denominator
5. Give a validation: which known total should match

[Output]
Approach and grain first, then the SQL, then a validation query.

[Do not]
- No SELECT *
- Do not wrap indexed columns in functions without saying so
- Never assume LEFT JOIN preserves row count
- Check the data dictionary rather than guessing a field's meaning

What to state before it starts

Leave these unsaid and the model fills them with an assumption. Most 'wrong answers' start here.

  • the missing date range
  • how the table is written
  • downstream dependencies

Go through each of these

Where this kind of work most often goes wrong. Already in the prompt, so you do not have to remember it.

  • back up before backfilling
  • backfill must be re-runnable with identical results
  • compare row counts against adjacent days after the run
  • state whether downstream reports need a refresh