Air pollution has been shown to have negative health outcomes such as lower life expectancy, increased illness and hospitalization rates. This is relatively straightforward. But given that pollution can penetrate the blood flow and the circulatory system, it may even have effects on cognition.
If pollution can indeed affect cognitive performance, even in the short run, then the harm from pollution is not limited to “simple” health issues. This means that days with higher air pollution can have lower labor productivity, more injuries in the workplace, and even lower test scores on various exams taken by students.
The sample consists of around 490,000 Bagrut exams taken in 2000-2002 by 70,000 Israeli students in grades 10-12. This is a very important, high-stakes examination which serves as a college entrance test for high school students. The exam covers various subjects (and as such there are multiple test days for each student). This means that we’re dealing with a test (1) whose outcome is very important for the students, and (2) which each student takes multiple times (though for different subjects). Passing the tests is not trivial, only around half of the students succeed.
This test data is used in the paper in conjunction with data on pollution to investigate the hypothesis put forward above. The authors consider two pollutants: PM2.5 (particulate matter, less than 2.5 micrometers in diameter) and CO (carbon monoxide).
PM2.5, also called fine particulate matter, is basically dust. Its main sources in Israel are sand storms and coal-burning power plants. CO, on the other hand, is mostly the result of motor vehicle emissions, and to a lesser extent of fossil fuel-furnaces and fires.
Monitors measuring these pollutants were all located within 2.5 km of the schools in the sample. It is also worth noting that given that the sources of PM2.5 and CO are very different, their levels in the air are pretty much uncorrelated.
As I will be referring to pollutants in Air Quality Index (AQI) units, it’s worth mentioning that AQI units are standardized across pollutants and can thus be easily interpreted using the table below (source).
First, the authors simply estimated the effects of a pollutant on test scores for the whole sample. The preferred specification includes student fixed effects, which are possible because one student takes multiple exams. A 10 AQI unit increase in PM2.5 decreases test scores by around .46, which is about 2% of the test scores’ standard deviation. A similar increase in CO decreases scores by .85, or about 4% of the standard deviation.
These effects are significant at the 1% level, but nevertheless are relatively small quantitatively in my opinion. But these numbers also hide the fact that heavily polluted days have a much larger effect on average. In other words, the relationship between pollution and test scores appears to be nonlinear: at first, we have a quantitatively small (but statistically significant) negative effect, but this effect then suddenly increases at levels of heavy pollution.
If on the day of the exam, PM2.5 levels exceed the unhealthy threshold (AQI > 100), then test scores will be on average 1.95 points lower, that’s about 8.2% of the standard deviation of test scores. For CO, if the day is among the top 5% most polluted days in the sample, then exam scores will be 10.16 points lower on average, which is about 42.8% of the standard deviation.
The evidence also indicates that effects for PM2.5 are largely linear: a 1 unit increase in PM2.5 will result in a relatively similar decrease in test scores, no matter the PM2.5 levels. For CO, the effects are much more nonlinear. While at low levels, CO may be effectively harmless, at higher levels a 1 unit increase in CO levels can create huge drops in test scores.
The effects are also consistent with the intra-day fluctuations of pollution. PM2.5 levels are roughly constant over the day, while CO levels usually peak in the afternoon. Indeed, it seems if we divide the sample into morning and afternoon exams, we find that for morning exams CO is about 10 times less harmful than for afternoon exams. As for PM2.5, its effect – as expected – is roughly constant across the day.
Another hypothesis of the authors is that PM2.5 is more harmful for those who have respiratory problems, while CO shouldn’t discriminate on this basis. They find evidence for this as well: boys – who in Israel are about 25% more likely to have asthma than girls – are about 2-4 times more sensitive to PM2.5. Similarly, Ashkenazi (Jews of European/American descent) students have a 63% higher incidence of asthma than Shephardi (Jews of African/Asian origin) students. And the authors do find that Ashkenazis are about 30-70% more sensitive to PM2.5 than Shephardis. These differences do not appear robustly for CO, lending support to the authors’ hypothesis.
Finally, the authors look at more long-term effects such as whether overall success at the Bagrut exam is affected by pollution. More specifically, they look at how the probability of failing the Bagrut is affected by the average pollution across all days a particular student had an exam on.
First, they find that the probability of failing goes up by 2.4% when PM2.5 levels are abnormally high, and by 12.3% when CO levels are abnormally high. Second, even the composite Bagrut score is affected negatively: a 10 unit increase in average PM2.5 reduces the score around 1.66 (10% of a standard deviation) for PM2.5. The effect is about half of this for CO.
There are also several robustness checks in the paper. All in all, my opinion is that the results are robust. The most reliable conclusion is that days heavily polluted with PM2.5 can have quantitatively significant negative effects on test performance, especially for those with asthma. Also, days with particularly bad CO (which seem to be relatively rare) can have some large negative effects on mental acuity.
On the other hand, whether one should worry about relatively small (< 100 AQI units) fluctuations in PM2.5 or CO as long as this fluctuation is happening below the unhealthy level of 100 AQI units is not so clear. As long as the air is not heavily polluted, it seems that even if PM2.5 and CO have negative effects on cognitive function, these effects are quantitatively small.
Still, given the results of this study, and the fact that heavy pollution – while relatively rare – is not nonexistent, it is quite possible that the risks of air pollution are underestimated. Besides the “usual” health problems, it appears air pollution can have significant detrimental effects on anything requiring mental acuity.
The results also bring in to question whether over-reliance on few high-stakes tests is a good idea. Such tests determine the fates of children in many countries. It would be a shame if external factors such as pollution could significantly influence one’s performance in these tests. Or maybe it’s time to invent the pollution-corrected SAT score?
In the spirit of this article, I made two graphs of the top 10 US metros by heavy air pollution. I ranked metros by the fraction of days on which their air pollution exceeded some threshold. The threshold for PM2.5 is an AQI of 100 (= 40.4 micrograms per cubic meter following Table A1 in the paper). For CO, a day is considered heavily polluted if its CO level is among the top 5% of all CO levels across the country; i.e. the cut-off is the 95th percentile.
EDIT: apparently, the high pollution of mountain cities in the top 10 lists above as well as the situation in the photo at the beginning of the article can be explained by temperature inversion (see here and here).