DAGs I + II

POL51

Haley Daarstad

University of California, Davis

August 26, 2026

Plan for today

Identifying effects

Controlling for confounds

Intuition

Limitations

Why DAG

We want to identify the effect of X (waffles) on Y (divorce)

We can use our model to identify that effect, BUT:

We also know that lurking variables might make things go awry (the South)

Why DAG

We know that the DAG on the left will produce the spurious correlation on the right

Regardless of whether or not waffles cause divorce

Why not control for everything?

Controlling for the wrong thing can close a perplexing pipe – this erases part or all of the effect that X has on Y

Or open up an exploding collider – creates weird correlation between X and Y

Example: Bias in police use of force

Are the police more likely to use deadly force against people of color?

Black Americans are 3.23 times more likely than white Americans to be killed by police (Schwartz and Jahn, 2020)

Yet there are big debates about how exactly to estimate this bias (and the extent to which it exists)

Fryer (2019) finds that Blacks and Hispanics are 50% more likely to be stopped by police, but that conditional on being stopped by the police, there are no racial differences in officer-involved shootings

Bias in use of force

Fryer used extensive controls about the nature of the interaction, time of day, and hundreds of factors that I’ve captured with Confounds

Bias in use of force

Fryer shows that once you account for the indirect effect, the direct effect is basically not there – once the police has stopped someone, they do not use deadly force more often against Minorities than Whites

Bias in police use of force

But what if police are more likely to stop people they believe are “suspicious” AND use force against people they find “suspicious”? THEN conditioning on the stop is equivalent to conditioning on a collider

Tough!

We’d like to know if Minorities are killed more than Whites in police interactions once they are stopped

But controlling for being stopped creates collider bias

Super tough to estimate the effect of race ➡️ police abuse with observational data!

What do we do?

We have to be careful and slow

Think carefully about what the DAG probably looks like

Use the DAG to figure out what we need to control

(and what must be left alone)

Next (in a few slides): how to actually control for stuff

Why experiments work

DAGs can also help us see why experiments “work”:

Person Shown an ad? Democrats thermometer
1 Yes 65.5
2 No 37.57
3 Yes 34.99
4 No 53.92
5 No 88.69

Why experiments work

Experiments seem simple…

Why experiments work

But the outcome can be very complex …

And yet we can still identify the effect because nothing causes you to receive the experimental treatment; it is random!

When experiments go wrong

Say the ad experiment was implemented on TikTok, and younger people are more likely to use TikTok than older people

This means Age is now a fork

# Identifying effects

Front-doors and back-doors

  • Judea Pearl’s back-door criterion ties this all together

  • Confounding caused by existence of an open “back door” path from X to Y

  • A backdoor path is a non-causal path from X to Y

  • Need to close back-doors and keep front-doors open

Backdoor paths

A backdoor path can involve a chain of variables – like the fork, but with more steps

Here we have a backdoor path between X and Y that runs through a, b, and m

Breaking the path

We can identify X \(\rightarrow\) Y by controlling for any variable in the backdoor path to break the chain: m, a, or b

Solve the DAG

Solve the DAG

Solve the DAG

Solve the DAG

Solve the DAG

Now back to -> Do waffles cause divorce?

Remember, we have strong reason to believe the South is confounding the relationship between Waffle Houses and divorce rates:

A problem we share with Lincoln

We need to control (for) the South

It has a bad influence on divorce, waffle house locations, and the integrity of the union

But how do we do control (for) the South? And what does that even mean?

We’ve already done it

One way to adjust/control for backdoor paths is with multiple regression:

In general: \(Y = \alpha + \beta_1X_1 + \beta_2X_2 + \dots\)

In this case: \(Y = \alpha + \beta_1Waffles + \beta_2South\)

In multiple regression, coefficients (\(\beta_i\)) are different: they describe the relationship between X1 and Y, after adjusting for the X2, X3, X4, etc.

What does it mean to control?

\(Y = \alpha + \beta_1Waffles + \beta_2South\)

Three ways of thinking about \(\color{red}{\beta_1}\) here:

  • The relationship between Waffles and Divorce, controlling for the South

  • The relationship between Waffles and Divorce that cannot be explained by the South

  • The relationship between Waffles and Divorce, comparing among similar states (South vs. South, North vs. North)

Comparing models

Let’s fit two models of the effect of GDP on life expectancy, one with controls and one without:

no_controls = lm(lifeExp ~ gdpPercap, data = gap)
controls = lm(lifeExp ~ gdpPercap + pop + year, data = gap)

Population and year might be forks here: they affect both economic activity and life expectancy

Note

I’ve re-scaled gdp (10,000s) and population (millions)

The regression table

We can compare models using a regression table;

many different functions, we’ll use huxreg() from the huxtable package

huxreg("No controls" = no_controls, "With controls" = controls)

Note

Format: huxreg("label for model 1" = model_1, "label for model 2" = model_2, ...)

The regression table

huxreg("No controls" = no_controls, "With controls" = controls)
No controlsWith controls
(Intercept)53.956 ***-411.450 ***
(0.315)   (27.666)   
gdpPercap7.649 ***6.729 ***
(0.258)   (0.244)   
pop        0.006 ** 
        (0.002)   
year        0.235 ***
        (0.014)   
N1704        1704        
R20.341    0.440    
logLik-6422.205    -6282.869    
AIC12850.409    12575.737    
*** p < 0.001; ** p < 0.01; * p < 0.05.

Comparison

  • Model 1 has no controls: just the relationship between GDP and life expectancy

  • Model 2 controls/adjusts for: population and year

  • the effect of GDP per capita on life expectancy changes with controls

  • The estimate is smaller

No controlsWith controls
(Intercept)53.956 ***-411.450 ***
(0.315)   (27.666)   
gdpPercap7.649 ***6.729 ***
(0.258)   (0.244)   
pop        0.006 ** 
        (0.002)   
year        0.235 ***
        (0.014)   
N1704        1704        
R20.341    0.440    
logLik-6422.205    -6282.869    
AIC12850.409    12575.737    
*** p < 0.001; ** p < 0.01; * p < 0.05.

Interpretation

No controls: every additional 10k of GDP = 7.6 years more life expectancy

With controls: after adjusting for population and year, every additional 10k of GDP = 6.7 years more of life expectancy

No controlsWith controls
(Intercept)53.956 ***-411.450 ***
(0.315)   (27.666)   
gdpPercap7.649 ***6.729 ***
(0.258)   (0.244)   
pop        0.006 ** 
        (0.002)   
year        0.235 ***
        (0.014)   
nobs1704        1704        
*** p < 0.001; ** p < 0.01; * p < 0.05.

In other words: .shout[comparing countries with similar population levels and in the same year], every additional 10k of GDP = 6.7 years more of life expectancy

Controlling for size

What happens if we control for how large a house is?

  • The effect goes very close to zero (by house price standards), and is even negative
no_controls = lm(price ~ bathrooms, data = house_prices)
controls = lm(price ~ bathrooms + sqft_living, data = house_prices)
huxreg("No controls" = no_controls, "Controls" = controls)
No controlsControls
(Intercept)10708.309    -39456.614 ***
(6210.669)   (5223.129)   
bathrooms250326.516 ***-5164.600    
(2759.528)   (3519.452)   
sqft_living        283.892 ***
        (2.951)   
N21613        21613        
R20.276    0.493    
logLik-304117.741    -300266.206    
AIC608241.481    600540.413    
*** p < 0.001; ** p < 0.01; * p < 0.05.

Does this actually work? Back to the waffles

These are examples with real data, where we can’t know for sure if our controls are doing what we think

Only way to know for sure is with made-up data, where we know the effects ex ante:

fake = tibble(south = sample(c(0, 1), size = 50, replace = TRUE), 
              waffle = rnorm(n = 50, mean = 20, sd = 4) + 10 * south,
              divorce = rnorm(n = 50, mean = 20, sd = 2) + 8 * south)

What do we know?

fake = tibble(south = sample(c(0, 1), size = 50, replace = TRUE), 
              waffle = rnorm(n = 50, mean = 20, sd = 4) + 10*south,
              divorce = rnorm(n = 50, mean = 20, sd = 2) + 8*south)

We know that waffles have 0 effect on divorce

We know that the south has an effect of 10 on divorce

We know that the south has an effect of 8 on waffles

Controlling for the South

Fit a naive model without controls, and the correct one controlling for the South:

naive_waffles = lm(divorce ~ waffle, data = fake)
control_waffles = lm(divorce ~ waffle + south, data = fake)

The results

Perfect! Naive model is confounded; but controlling for the South, we get pretty close to the truth (0 effect)

Naive modelControl South
(Intercept)9.242 ***21.182 ***
(2.248)   (2.011)   
waffle0.587 ***-0.085    
(0.087)   (0.097)   
south        9.443 ***
        (1.118)   
nobs50        50        
*** p < 0.001; ** p < 0.01; * p < 0.05.

What’s going on?

In our made-up world, if we control for the South we can get back the uncounfounded estimate of Divorce ~ Waffles

But what’s lm() doing under-the-hood that makes this possible?

What’s going on?

  • lm() is estimating \(South \rightarrow Divorce\) and \(South \rightarrow Waffles\)

  • it is then subtracting out or removing the effect of South on Divorce and Waffles

  • what’s left is the relationship between Waffles and Divorce, adjusting for the influence of the South on each

Visualizing controlling for the South

This is the confounded relationship between waffles and divorce (zoomed out)

Add the south

We can see what we already know: states in the South tend to have more divorce, and more waffles

Effect of south on divorce

\(South \rightarrow Divorce = 10\) How much higher, on average, divorce is in the South than the North

Remove effect of South on divorce

Regression subtracts out the effect of the South on divorce

Next: effect of South on waffles

\(South \rightarrow Waffles = 8\) How many more, on average, Waffle Houses there are in the South than the North

Subtract out the effect of south on waffles

Regression subtracts out the effect of the South on waffles

What’s left over?

The true effect of waffles on divorce \(\approx\) 0

The other confounds

The perplexing pipe

Remember, with a perplexing pipe, controlling for Z blocks the effect of X on Y:

Simulation

Let’s make up some data to show this: every unit of foreign aid increases corruption by 8; every unit of corruption increases the number of protest by 4

fake_pipe = tibble(aid = rnorm(n = 200, mean = 10), 
                   corruption = rnorm(n = 200, mean = 10) + 8 * aid, 
                   protest = rnorm(n = 200, mean = 10) + 4 * corruption)

What is the true effect of aid on protest? Tricky since the effect runs through corruption

For every unit of aid, corruption increases by 8; and for every unit of corruption, protest increases by 4…

The effect of aid on protest is \(4 \times 8 = 32\)

The data

aid corruption protest
11.02 97.12 397.48
10.41 92.95 382.00
9.13 82.51 339.76
10.13 92.60 381.16
10.88 97.00 398.86
10.67 92.67 378.27

Bad controls

Remember, with a pipe controlling for Z (corruption) is a bad idea

Let’s fit two models, where one makes the mistake of controlling for corruption

right_model = lm(protest ~ aid, data = fake_pipe)
bad_control = lm(protest ~ aid + corruption, data = fake_pipe)

Bad controls

Notice how the model that mistakenly controls for Z tells you that X basically has no effect on Y (wrong)

Correct modelBad control
(Intercept)55.345 ***9.015 ***
(3.030)   (1.074)   
aid31.461 ***-0.609    
(0.304)   (0.559)   
corruption        4.079 ***
        (0.071)   
nobs200        200        
*** p < 0.001; ** p < 0.01; * p < 0.05.

The exploding collider

Remember, with an exploding collider, controlling for M creates strange correlations between X and Y:

Simulation

Let’s make up some data to show this:

fake_collider = tibble(x = rnorm(n = 100, mean = 10), 
                   y = rnorm(n = 100, mean = 10),
                   m = rnorm(n = 100, mean = 10) + 8 * x + 4 * y)

  • X has an effect of 8 on M

  • Y has an effect of 4 on M

  • X has no effect on Y

The data

x y m
8.599355 9.685493 118.0025
8.154775 9.549067 114.8474
10.468210 10.598875 136.5320
9.668180 11.408754 131.3963
12.168042 11.236746 153.0428
10.997311 9.620571 136.6360

Bad controls

What’s the true effect of X on Y? it’s zero

Remember, with a collider controlling for M is a bad idea

Let’s fit two models, where one makes the mistake of controlling for M

right_model = lm(y ~ x, data = fake_collider)
collided_model = lm(y ~ x + m, data = fake_collider)

Bad controls

Notice how the model that mistakenly controls for M tells you that X has a strong, negative effect on Y (wrong)

Correct modelCollided!
(Intercept)9.525 ***-2.268 ***
(1.009)   (0.382)   
x0.050    -1.992 ***
(0.100)   (0.056)   
m        0.248 ***
        (0.006)   
nobs100        100        
*** p < 0.001; ** p < 0.01; * p < 0.05.

Colliding as sample selection

Most of the time when we see a collider, it’s because we’re looking at a weird sample of the population we’re interested in

Examples: the non-relationship between height and scoring, among NBA players; the (alleged) negative correlation between how surprising and reliable findings are, among published research

Hiring at Google

Imagine Google wants to hire the best of the best, and they have two criteria: interpersonal skills, and technical skills

Say Google can measure how socially and technically skilled someone is (0-100)

fake_google = tibble(social_skills = rnorm(n = 200, mean = 50, sd = 10), 
                     tech_skills = rnorm(n = 200, mean = 50, sd = 10))

The two are causally unrelated: one does not affect the other; improving someone’s social skills would not hurt their technical skills

The data

social_skills tech_skills
54.75 50.67
30.33 46.83
45.68 48.21
62.76 47.19
51.07 57.09
50.85 54.04
64.30 31.70
45.92 54.46

Simulate the hiring process

Now imagine that they add up the two skills to see a person’s overall quality:

fake_google %>% 
  mutate(total_score = social_skills + tech_skills)
social_skillstech_skillstotal_score
71.668.7140  
49.441.490.8
42.959.6103  
41.139.880.9
47.253.1100  
30.346.877.2
33.758.892.5
66.364  130  
58.333.792  
54.347.8102  
39.960.9101  
58.444.9103  
59.258.5118  
54.959.8115  
57.647.5105  
65.136.4101  
22.638.360.9
53.454.8108  
64.454.8119  
60  45.4105  
45.954.5100  
57.760.3118  
31.846  77.8
54.544.999.4
51.849.4101  
52.435.587.8
45.643.489  
41.557  98.5
64.244.5109  
59.343.8103  
72.336.5109  
55.455.9111  
49.847.897.6
62.662.3125  
58  51.7110  
51.946.698.6
49  68.9118  
53.360.2113  
48.944.293.1
59.426.886.2
62.672.4135  
57  62.9120  
58.947  106  
72.453.2126  
50.436.887.2
65.248.1113  
44.145.589.6
50.841.692.5
48.251  99.3
46.948.795.6
60.766.4127  
40.936.777.6
54.848  103  
46.661.7108  
40.546.487  
63  54.1117  
54.742.397  
65.141.7107  
53.529.282.6
49.450.199.5
46.338.484.7
53.446.499.8
54.372.9127  
63.355.1118  
66.339.2106  
36.550  86.5
51.243.895  
53  41.194.1
47.332.379.6
46.950.497.3
54.750.7105  
53.642.696.2
59.873.1133  
42  43.485.4
51.450.9102  
41.737.178.8
51.149.7101  
62.847.2110  
49  37.786.7
66.264.5131  
54.548  102  
36.150  86  
56.859.6116  
58  57.7116  
54.850.2105  
33.661.495  
46.745.692.3
76.466.6143  
57.163.5121  
46.141.487.5
47.135.682.8
54.542  96.5
63.156.6120  
57.734.892.4
60.152.1112  
50.646.797.4
46.848  94.7
69.843.6113  
51.825.677.4
55.562.9118  
45.648.393.9
68.545.7114  
56.757.7114  
48.947.296.2
58.769.4128  
44.960.2105  
70.954.5125  
47.940.988.9
75.956.8133  
48.169.7118  
64.968.1133  
58.747.2106  
55.663.2119  
39.950.590.4
49.345.795.1
37.955.493.3
67.867.9136  
45.748.293.9
65  50.9116  
20.340.360.7
43.442.786.1
58.770.4129  
57.661.6119  
52.463  115  
44.557.8102  
69.258.2127  
41.135.576.6
54.856.2111  
57.847.6105  
42.448.190.5
43.835.579.4
44.154.898.9
40.345.685.8
49.352.7102  
32  48.380.3
50.552.7103  
56.758  115  
51.652.5104  
56  58.3114  
42.246.188.4
52.548.3101  
50.948.399.2
64.331.796  
47.337.985.3
55.159.9115  
37.942.380.2
46.755.3102  
58.855.2114  
58.333.691.9
44.454.298.7
44  43.787.7
42  52.294.2
30.832.763.6
30.851.782.5
50.854  105  
47.256.7104  
52.151.8104  
70.441.8112  
56.663.8120  
61  53.8115  
49.559.2109  
59  40.299.2
41.554.395.8
63.468.7132  
52.440.793  
62.157.7120  
63.741.5105  
46  52.298.1
44.627.972.5
55.938.394.1
48.558.1107  
30.654.284.8
56.925.682.5
46.356.9103  
57.657.7115  
63.148.5112  
44.944.389.2
53.762.6116  
64.360.4125  
53.635.388.9
62.461.7124  
48.363  111  
60.234.594.6
51.865.1117  
39.962.8103  
47.950.998.8
50.836.687.4
50  70.1120  
51.157.1108  
46.754.9102  
38.345  83.3
39.442.481.8
52.161.7114  
61.356.3118  
57.955.6114  
49.357.1106  
44.760.5105  
42.861  104  
51.144.495.4
55.737.993.6

Simulating the hiring process

Now imagine that Google only hires people who are in the top 15% of quality (in this case that’s 112.8 or higher)

fake_google %>% 
  mutate(total_score = social_skills + tech_skills) %>% 
  mutate(hired = case_when(total_score >= quantile(total_score, .85) ~ "yes", 
                           total_score < quantile(total_score, .85) ~ "no"))
social_skillstech_skillstotal_scorehired
71.668.7140  yes
49.441.490.8no
42.959.6103  no
41.139.880.9no
47.253.1100  no
30.346.877.2no
33.758.892.5no
66.364  130  yes
58.333.792  no
54.347.8102  no
39.960.9101  no
58.444.9103  no
59.258.5118  no
54.959.8115  no
57.647.5105  no
65.136.4101  no
22.638.360.9no
53.454.8108  no
64.454.8119  yes
60  45.4105  no
45.954.5100  no
57.760.3118  no
31.846  77.8no
54.544.999.4no
51.849.4101  no
52.435.587.8no
45.643.489  no
41.557  98.5no
64.244.5109  no
59.343.8103  no
72.336.5109  no
55.455.9111  no
49.847.897.6no
62.662.3125  yes
58  51.7110  no
51.946.698.6no
49  68.9118  no
53.360.2113  no
48.944.293.1no
59.426.886.2no
62.672.4135  yes
57  62.9120  yes
58.947  106  no
72.453.2126  yes
50.436.887.2no
65.248.1113  no
44.145.589.6no
50.841.692.5no
48.251  99.3no
46.948.795.6no
60.766.4127  yes
40.936.777.6no
54.848  103  no
46.661.7108  no
40.546.487  no
63  54.1117  no
54.742.397  no
65.141.7107  no
53.529.282.6no
49.450.199.5no
46.338.484.7no
53.446.499.8no
54.372.9127  yes
63.355.1118  yes
66.339.2106  no
36.550  86.5no
51.243.895  no
53  41.194.1no
47.332.379.6no
46.950.497.3no
54.750.7105  no
53.642.696.2no
59.873.1133  yes
42  43.485.4no
51.450.9102  no
41.737.178.8no
51.149.7101  no
62.847.2110  no
49  37.786.7no
66.264.5131  yes
54.548  102  no
36.150  86  no
56.859.6116  no
58  57.7116  no
54.850.2105  no
33.661.495  no
46.745.692.3no
76.466.6143  yes
57.163.5121  yes
46.141.487.5no
47.135.682.8no
54.542  96.5no
63.156.6120  yes
57.734.892.4no
60.152.1112  no
50.646.797.4no
46.848  94.7no
69.843.6113  no
51.825.677.4no
55.562.9118  no
45.648.393.9no
68.545.7114  no
56.757.7114  no
48.947.296.2no
58.769.4128  yes
44.960.2105  no
70.954.5125  yes
47.940.988.9no
75.956.8133  yes
48.169.7118  no
64.968.1133  yes
58.747.2106  no
55.663.2119  yes
39.950.590.4no
49.345.795.1no
37.955.493.3no
67.867.9136  yes
45.748.293.9no
65  50.9116  no
20.340.360.7no
43.442.786.1no
58.770.4129  yes
57.661.6119  yes
52.463  115  no
44.557.8102  no
69.258.2127  yes
41.135.576.6no
54.856.2111  no
57.847.6105  no
42.448.190.5no
43.835.579.4no
44.154.898.9no
40.345.685.8no
49.352.7102  no
32  48.380.3no
50.552.7103  no
56.758  115  no
51.652.5104  no
56  58.3114  no
42.246.188.4no
52.548.3101  no
50.948.399.2no
64.331.796  no
47.337.985.3no
55.159.9115  no
37.942.380.2no
46.755.3102  no
58.855.2114  no
58.333.691.9no
44.454.298.7no
44  43.787.7no
42  52.294.2no
30.832.763.6no
30.851.782.5no
50.854  105  no
47.256.7104  no
52.151.8104  no
70.441.8112  no
56.663.8120  yes
61  53.8115  no
49.559.2109  no
59  40.299.2no
41.554.395.8no
63.468.7132  yes
52.440.793  no
62.157.7120  yes
63.741.5105  no
46  52.298.1no
44.627.972.5no
55.938.394.1no
48.558.1107  no
30.654.284.8no
56.925.682.5no
46.356.9103  no
57.657.7115  no
63.148.5112  no
44.944.389.2no
53.762.6116  no
64.360.4125  yes
53.635.388.9no
62.461.7124  yes
48.363  111  no
60.234.594.6no
51.865.1117  no
39.962.8103  no
47.950.998.8no
50.836.687.4no
50  70.1120  yes
51.157.1108  no
46.754.9102  no
38.345  83.3no
39.442.481.8no
52.161.7114  no
61.356.3118  no
57.955.6114  no
49.357.1106  no
44.760.5105  no
42.861  104  no
51.144.495.4no
55.737.993.6no

General population

No relationship between social and technical skills among all job candidates

Collided!

If we only look at Google workers we see a trade-off between social and technical skills:

Limitations

It’s cool that we can control for a confound, or avoid colliders/pipes and get back the truth

But there are big limitations we must keep in mind when evaluating research:

  • We need to know what to control for (confident in our DAG)
  • We need to have data on the controls (e.g., data on Z)
  • We need our data to measure the variable well (e.g., # of homicides a good proxy for crime?)

Stuff that’s hard to measure

Ability is a likely fork for the effect of Education on Earnings; but how do you measure ability?

💥 Your Turn: Make a DAG 💥

  • On one piece of paper, draw a DAG for your team’s project as a group

    • Identify the treatment and control

    • identify possible confounders

      • Also note which type: fork, collidier, and pipe.

  • Make sure to put your names! This is your AE for the week.