All categories/Data and analysis
Data extraction and SQL · Level Advanced · SQL SQL Dedup
Distinct counts: how many people really
User counts differ because one person has many rows. Clarify the deduplication.
Prompt
You state the question before choosing tables. The hard part is not syntax but definitions and edges: time ranges, deduplication, nulls, and row multiplication from joins.
Task: User counts differ because one person has many rows. Clarify the deduplication.
[Before you start, do three things]
1. Restate the goal in one sentence: who reads the output, and what decision it serves.
2. If I have not given you the following, ask me. Do not assume:
- user identifier fields: uid, phone, device
- whether accounts merge across devices
- period
3. Say how you plan to do it (at most 5 steps) and wait for my go-ahead.
[Go through each of these]
- list identifiers and the bias of each
- explain cross-device duplication
- show the before-and-after count and where the gap comes from
- recommend one definition externally
[Process]
1. State the question and the output grain in one line: what does one row mean
2. Confirm table structure, keys and cardinality before joining
3. Write the SQL with a comment on the business meaning of each filter
4. Self-check: dedup, time boundaries, nulls, denominator
5. Give a validation: which known total should match
[Output]
Approach and grain first, then the SQL, then a validation query.
[Do not]
- No SELECT *
- Do not wrap indexed columns in functions without saying so
- Never assume LEFT JOIN preserves row count
- Check the data dictionary rather than guessing a field's meaningWhat to state before it starts
Leave these unsaid and the model fills them with an assumption. Most 'wrong answers' start here.
- user identifier fields: uid, phone, device
- whether accounts merge across devices
- period
Go through each of these
Where this kind of work most often goes wrong. Already in the prompt, so you do not have to remember it.
- list identifiers and the bias of each
- explain cross-device duplication
- show the before-and-after count and where the gap comes from
- recommend one definition externally