Research design · Level Advanced · Research Causality Methods
Causal claims: what licenses them
A correlation in observational data and an urge to call it causal.
Prompt
One question: if the claim is true, what would the data look like, and if false. If you cannot tell them apart, there is no design.
Task: A correlation in observational data and an urge to call it causal.
[Before you start, do three things]
1. Restate the goal in one sentence: who reads the output, and what decision it serves.
2. If I have not given you the following, ask me. Do not assume:
- data source and temporal structure
- confounders
- identification strategies available
- outcome measurement
3. Say how you plan to do it (at most 5 steps) and wait for my go-ahead.
[Go through each of these]
- list three confounders and how they touch both X and Y
- one feasible identification strategy
- state the assumptions it needs
- if impossible, reframe as association
[Process]
1. State the question as something falsifiable
2. Name the observation that separates true from false
3. Check whether sample, measurement and control can support it
4. List three things that could overturn it
5. State the scope
[Output]
Question, criterion, method, counter-evidence, scope.
[Do not]
- Never equate correlation with cause
- Never let 'significant' stand for 'important'
- No conclusions without a control
- Do not hide unfavourable resultsWhat to state before it starts
Leave these unsaid and the model fills them with an assumption. Most 'wrong answers' start here.
- data source and temporal structure
- confounders
- identification strategies available
- outcome measurement
Go through each of these
Where this kind of work most often goes wrong. Already in the prompt, so you do not have to remember it.
- list three confounders and how they touch both X and Y
- one feasible identification strategy
- state the assumptions it needs
- if impossible, reframe as association