Fighting for Reliable Evidence

Book excerpt: Once primarily used in medical clinical trials, random assignment experimentation is now accepted among social scientists across a broad range of disciplines. The technique has been used in social experiments to evaluate a variety of programs, from microfinance and welfare reform to housing vouchers and teaching methods. How did randomized experiments move beyond medicine and into the social sciences, and can they be used effectively to evaluate complex social problems? Fighting for Reliable Evidence provides an absorbing historical account of the characters and controversies that have propelled the wider use of random assignment in social policy research over the past forty years. Drawing from their extensive experience evaluating welfare reform programs, noted scholar practitioners Judith M. Gueron and Howard Rolston portray randomized experiments as a vital research tool to assess the impact of social policy. In a random assignment experiment, participants are sorted into either a treatment group that participates in a particular program, or a control group that does not. Because the groups are randomly selected, they do not differ from one another systematically. Therefore any subsequent differences between the groups can be attributed to the influence of the program or policy. The theory is elegant and persuasive, but many scholars worry that such an experiment is too difficult or expensive to implement in the real world. Can a control group be truly insulated from the treatment policy? Would staffers comply with the random allocation of participants? Would the findings matter? Fighting for Reliable Evidence recounts the experiments that helped answer these questions, starting with the income maintenance experiments and the Supported Work project in the 1960s and 1970s. Gueron and Rolston argue that a crucial turning point came during the 1980s, when Congress allowed states to experiment with welfare programs and foundations, states, and the federal government funded larger randomized trials to assess the impact of these reforms. As they trace these historical shifts, Gueron and Rolston discuss the ways that strategies for resolving theoretical and practical problems were developed, and they highlight the strict conditions required to execute a randomized experiment successfully. What emerges is a nuanced portrait of the potential and limitations of social experiments to advance empirical knowledge. Weaving history, data analysis and personal experience, Fighting for Reliable Evidence offers valuable lessons for researchers, policymakers, funders, and informed citizens interested in isolating the effect of policy initiatives. It is an essential primer on welfare policy, causal inference, and experimental designs.