The Logic of Adaptive Sequential Experimentation in Policy Design

Book excerpt: Inspired by the wide adoption of rigorous randomized controlled trials (RCTs) in medical research, economists and other social scientists have increasingly used RCTs in their research. As researchers pick up projects amenable to the RCT methodology, they likely leave out important questions to which RCTs cannot be directly applied. As a result, RCTs have been criticized for the proclivity of addressing trivial questions. As a matter of fact, in medical research RCTs are an integral part of adaptive sequential experiment design—a few steps must be taken to screen out drugs that have toxins and strong side effects before running any RCTs on humans. In this paper, we argue that economists can learn a great deal from the design principles implemented in medical research. We develop a theoretical model to show the logic of adaptive sequential experiment design in the presence of uncertainty over negative effects and discuss how to choose samples in a population to minimize the experiment cost. We also point out the applications of our proposed framework in the economic domain, such as economic reforms and new product design.