Enhancing Evaluation Use

Book excerpt: Enhancing Evaluation Use: Insights from Internal Evaluation Units offers invaluable insights from real evaluators who share strategies they have adopted through their own experiences in evaluation. Readers will learn about the challenges, solutions, and lessons drawn from the experience of evaluators working in a wide range of organizations. Referencing the latest literature, contributors discuss factors that help or undermine attempts to foster an evaluative thinking and learning culture within an organization. Applicable in a wide range of situations, their accounts demonstrate the initiative and innovative thinking they use to address challenges in various, sometimes complex, evaluation settings. Questions at the end of each chapter stimulate thought and discussions about the issues raised and allow readers to apply their findings to their own situations. "This book speaks to a cutting-edge topic, that is, the potential to generalize program evaluation expertise to larger organizational questions, and the cases from multiple international contexts represent a unique feature." —John Clayton Thomas, Georgia State University "The use of actual cases to highlight major concepts in evaluation in the public sector is a great feature." —Danica G. Hays, Old Dominion University "The text provides practical information from a variety of organizational contexts and the integration of international experiences provides for expanded discussion of evaluation theory and practice." —Kathleen Norris, Plymouth State University "The key strengths of this book lie in its national, supra-national and international organizational contexts, its consistency in insider perspectives, and the detailed examples provided." —Donna Haig Friedman, University of Massachusetts, Boston "The book of essays reviewed here was edited by two eminent evaluators. It fills an important gap in the literature: in pursuit of improved quality of evaluation products, evaluation thinkers have lavished attention on evaluation methods, ethics and use but they have sorely neglected evaluation governance issues and have largely failed to probe the workings of evaluation within organizations. Yet, most evaluations are commissioned by (or undertaken within) organizations. The choices organizations make in structuring evaluation functions and designing evaluation processes have a major bearing on the relevance, validity and usefulness of evaluations. Refreshingly, the book offers fresh mental models and practical lessons about evaluation systems and practices adopted in diverse organizational settings. All contributors to the book are seasoned practitioners. They hail from national, supranational and international organizations and many of them have trespassed across these thematic and organizational boundaries. They all are equipped to draw on a vast reservoir of hands---on experience as evaluation commissioners, managers, internal evaluators or external practitioners. Remarkably, they also display considerable familiarity with relevant themes treated by the organization management and evaluation literature. Given its pragmatic focus the book is bound to elicit broad based interest among evaluation practitioners. While it addresses familiar dilemmas and challenges (evaluation independence, evaluation utilization, organizational learning, nurturing of an evaluation culture, etc.) it does so from the distinctive perspective of "insiders" who have had to contend with a variety of organizational constraints and management pressures. Revealingly most contributors find good reasons to be optimistic about the possibility of positive change. The stage is set by John Mayne whose introductory and authoritative chapter identifies the critical importance of organizational factors in the utilization of evaluation results. This is followed by Bastiaan de Laat’s chapter which puts forward an ingenious analytical construct: the "tricky triangle" of relationships that links commissioners, evaluators and evaluands. Specifically De Laat unmasks and assesses the different and complex configurations that result from this triangular interplay. Penny Hawkins’ elegant contribution underlines the importance of evaluation independence. It highlights the considerable impact that the electoral cycle may have on evaluation approaches. It also reveals that indigenous cultures may present formidable obstacles to the very notion of external evaluative oversight. The next chapter penned by Marlene Laubli’s is equally perceptive in tracing the creative adaptation that the evaluation function underwent in response to changes in the organizational force field of the Swiss Federal Office of Public Health’s Evaluation Unit. In a similar vein Erica Wimbush provides a lucid exposition of the strategies used to bridge evaluative knowledge and management action in Scotland’s public health agency. Still within the health field, Nancy Porteous and Steve Montague show convincingly that transformative organizational change in Canada’s Public Health Agency was facilitated through judicious evaluation programming and what has come to be called "double loop learning". Finally, Bastiaan de Laat and Kevin Williams summarize important findings from of two empirical studies of evaluation use in the European Commission that go a long way in identifying good practices for evaluation commissioning in a supra--- national organizational context. The next two chapters of the book focus on two specialized agencies of the United Nations. They tread gently in what is admittedly a highly sensitive and contested terrain. Maria J. Santamaria Hergueta, Alan Schnur and Deepak Thapa discuss how the World Health Organization’s evaluation function evolved towards greater independence over several decades while Janet Neubecker, Matthew Ripley and Craig Russon focus on the innovative techniques used within the International Labour Organization to enhance utilization of self evaluation findings. The final chapter by Marlene Laubli Loud misses the opportunity to compensate for the "good news" bias that understandably characterizes the previous contributions. Surprisingly, it displays ambivalence towards evaluation independence which it equates with externality, a concept soundly rejected by de Laat. But setting this dimension aside, the chapter is interesting and valuable since it reflects the co---editor’ vast experience as an evaluator, evaluation trainer and evaluation manager in diverse national and international contexts. Thus the chapter provides a useful compilation of "take away" messages focused on good evaluation management practices. It is on solid grounds when it describes how to make effective use of advisory committees, motivate potential evaluation users, design good terms of reference, select the right evaluators, craft judicious evaluation policies and keep an eye out for political "windows of opportunity". Each chapter of the book concludes with a crisp exposition of overarching findings, discussion topics and references. Thus the volume should be of practical value to teachers, students, professional evaluators as well as evaluation commissioners and programme managers. All in all, this is a book that belongs on your shelf if you are intent on enhancing the role that evaluation plays in your organization." —Robert Picciotto, UKES Council Member