Reliability of a Meta-analysis of Air Quality−Asthma Cohort Studies

  •  S. Stanley Young    
  •  Kai-Chieh Cheng    
  •  Jin Hua Chen    
  •  Shu-Chuan Chen    
  •  Warren B. Kindzierski    


What may be a contributing cause of the replication problem in science – multiple testing bias – was examined in this study. Independent analysis was performed on a meta-analysis of cohort studies associating ambient exposure to nitrogen dioxide (NO2) and fine particulate matter (PM2.5) with development of asthma. Statistical tests used in 19 base papers from the meta-analysis were counted. Test statistics and confidence intervals from the base papers used for meta-analysis were converted to p-values. A combined p-value plot for NO2 and PM2.5 was constructed to evaluate the effect heterogeneity of the p-values. Large numbers of statistical tests were estimated in the 19 base papers – median 13,824 (interquartile range 1,536−221,184). Given these numbers, there is little assurance that test statistics used from the base papers for meta-analysis are unbiased. The p-value plot of test statistics showed a two-component mixture. The shape of the p-value plot for NO2 suggests the use of questionable research practices related to small p-values in some of the cohort studies. All p-values for PM2.5 fall on a 45-degree line in the p-value plot indicating randomness. The claim that ambient exposure to NO2 and PM2.5 is associated with development of asthma is not supported by our analysis.

This work is licensed under a Creative Commons Attribution 4.0 License.
  • ISSN(Print): 1927-7032
  • ISSN(Online): 1927-7040
  • Started: 2012
  • Frequency: bimonthly

Journal Metrics

  • h-index (December 2021): 20
  • i10-index (December 2021): 51
  • h5-index (December 2021): N/A
  • h5-median(December 2021): N/A

( The data was calculated based on Google Scholar Citations. Click Here to Learn More. )