Risk and Evidence of Bias in Randomized Controlled Trials

Alex Eble, Brown University
Peter Boone, Effective Intervention

The randomized controlled trial (RCT), a common tool in the medical literature, has enjoyed recent popularity in the social sciences, where it has been used to evaluate questions of both policy and theory. Surprisingly, social science RCT reports seem to ignore a large body of the medical literature which identifies a set of common mistakes in trial design and links these to biased (exaggerated) treatment effect estimates. We use this medical literature to design a tool to evaluate adequacy of reporting and risk of bias in RCT reports and use it to evaluate RCT reports published in a set of major economics journals between 2000 and 2009 alongside a sample of RCT reports published in medical journals over the same period. We find economics RCTs fall short of the recommendations for reporting and conduct put forth in the medical literature, suggesting risk of exaggerated treatment effects in the economics literature.