The Science of Evaluation
Share

The Science of Evaluation
A Realist Manifesto



© 2013 | 240 pages | SAGE Publications Ltd
Evaluation researchers are tasked with providing the evidence to guide programme building and to assess its outcomes. As such, they labour under the highest expectations - bringing independence and objectivity to policy making. They face huge challenges, given the complexity of modern interventions and the politicised backdrop to all of their investigations. They have responded with a huge portfolio of research techniques and, through their professional associations, have set up schemes to establish standards for evaluative inquiry and to accredit evaluation practitioners. A big question remains. Has this monumental effort produced a progressive, cumulative and authoritative body of knowledge that we might think of as evaluation science? This is the question addressed by Ray Pawson in this sequel to Realistic Evaluation and Evidence-based Policy. In answer, he provides a detailed blueprint for an evaluation science based on realist principles.
Preface: The Armchair Methodologist and the Jobbing Researcher  
 
PART ONE: PRECURSORS AND PRINCIPLES
 
Precursors: From the Library of Ray Pawson
 
First Principles: A Realist Diagnostic Workshop
 
PART TWO: THE CHALLENGE OF COMPLEXITY - DROWNING OR WAVING?
 
A Complexity Checklist
 
Contested Complexity
 
Informed Guesswork: The Realist Response to Complexity
 
PART THREE: TOWARDS EVALUATION SCIENCE
 
Invisible Mechanisms I: The Long Road to Behavioural Change
 
Invisible Mechanisms II: Clinical Interventions as Social Interventions
 
Synthesis as Science: The Bumpy Road to Legislative Change
 
Conclusion: A Mutually Monitoring, Disputatious Community of Truth Seekers

This work provides a real breakthrough. It teaches us to appreciate the hugely varied interactions that can occur when our policy interventions are inserted into the complex environments in which we live. More importantly, it teaches us how to extract general lessons from the diverse occurrences we observe in various policy settings, so that we can have a better understanding and better results in the future.
Nancy Cartwright
Professor of Philosophy, University of California San Diego


Pawson has lost none of his touch for the pithy phrase and no-nonsense argument, but this time the style is put to service in a work of architectural proportions. I hope and expect this book to become required reading within and beyond the evaluation community.
Christopher Pollitt
Emeritus Professor, Katholieke Universiteit Leuven, Belgium


Ray Pawson's new book is a treat for evaluators, particularly in a world full of checklists, protocols, guidelines, and best practices. Evaluation is back to where it belongs: a full consideration of the theory and method of applied research with its scientific roots taken seriously.
Frans Leeuw
Professor of Law, Public Administration and Social Research, Maastricht University, The Netherlands


Like his earlier books, Realistic Evaluation, with Nick Tilley, and Evidence-Based Policy: A Realist Approach, this book has the potential to transform our understanding of what evaluation can be and how it can be used. Pawson addresses common questions about realist evaluation and shows clearly that it is more than simply identifying patterns in results but essentially involves iteratively building and testing theories to explain these patterns. His revisiting of classic evaluation texts demonstrates their realist underpinnings and he convincingly reclaims the Campbell mantle from those who would invoke his name only for a narrow range of experimental research designs. The book deserves an audience among evaluators and users of evaluation, especially those who are sceptical of realist evaluation who can create the "mutually monitoring, disputatious community of scholars" to properly review and revise these ideas.
Patricia Rogers
Professor of Public Sector Evaluation, Royal Melbourne Institute of Technology, Australia


I really enjoyed the evident relish which Ray Pawson takes into methodological debates, and the disputatious but constructive style of the whole book - a good example being the section on the MRC complex interventions evaluation framework. One particular highlight in the book for me was the chapter on 'clinical interventions as social interventions' which should be required reading for all biomedicalinical researchers, if only because they'll disagree with it.
Kieran Walshe
Professor of Health Policy and Management, Manchester Business School


Ray Pawson's The Science of Evaluation completes the realist evaluation trilogy composed of Realist Evaluation (co-author NickTilley) and Evidence-based Policy. His approach has beguiled many in the field but has left us with many puzzles: How to deal with complex programs? How to choose and develop program theories? How to deal with uncertainty? This new work provides answers, with characteristic brio, by moving backwards into the theoretical roots of realism, with its generative view of causation, and the legacy of the evolutionary epistemology - and also by moving forward presenting illustrations of the logic of inquiry of evaluations that provide a "realist response to complexity". Pawson's argument for "middle range theories" and "partial knowledge" is an invitation to engage in a "never-ending journey" that will take evaluation to the core of policy-making and social change.
Prof Nicoletta Stame
Sapienza: Università di Roma


Ray Pawson is quite a rare creature - a professor of social research methodology, and this is a book about methodology, the 'how to' of the evaluation of public policies and programmes. It both argues for a 'realist' approach to these activities and explores in some detail what such an approach entails. It is written in a lively, vivid, personal and often humorous way that will be familiar to those who read the author's previous works...I would say that, whether or not you intend to embark on this particular voyage, whether or not you like academic texts which are sprinkled with humour, you should consider reading this book. It strikes me as genuinely important, and very challenging. And certainly all those who think they believe in (or at least believe it is judicious to pretend to believe in) 'evidence-based policymaking' should explore Pawson's book - in order to get a taste for just what that would actually entail.
Christopher Pollitt
The International Review of Administrative Sciences


 

'The Science of Evaluation deserves to be widely read and discussed. In a literature that often struggles to be lucid, accessible and engaging, writing with clarity, honesty and passion, Pawson thoughtfully outlines how evaluation research can embrace a realist perspective without fearing it will lose its methodological ardour'

Lawrence Buhagiar
LSE Review of Books

Pawson calls for an endless journey into evaluation, organised with interventiontheories as the unit of analysis, so as to generate a phalanx of middle-range researchprogrammes. His second suggestion is for maintaining an ever-shifting and improvingmiddle ground that can only be achieved collectively and through constant criticalscrutiny of each other’s work. Thus his hope is that ideas and problems raised inthis book will stimulate and encourage more research about the scientific status ofevaluation.

Mae Keary, Scott-Keary Consultants
Online Information Review

The Science of Evaluation is a key text for anyone interested in research in the social sciences. It is a brilliant book which will justifiably become a fixture on students’ reading lists, as it brings together in one place Pawson’s thinking and ideas in relation to Realist Evaluation.

Donald Forrester, Tilda Goldberg Centre, University of Bedfordshire
Journal of Children's Services

Sample Materials & Chapters

Chapter 1


Preview this book

For instructors

This book is not available as an inspection copy. For more information contact your local sales representative.

Purchasing options

Please select a format:

ISBN: 9781446252437
ISBN: 9781446252420

SAGE Research Methods is a research methods tool created to help researchers, faculty and students with their research projects. SAGE Research Methods links over 175,000 pages of SAGE’s renowned book, journal and reference content with truly advanced search and discovery tools. Researchers can explore methods concepts to help them design research projects, understand particular methods or identify a new method, conduct their research, and write up their findings. Since SAGE Research Methods focuses on methodology rather than disciplines, it can be used across the social sciences, health sciences, and more.

With SAGE Research Methods, researchers can explore their chosen method across the depth and breadth of content, expanding or refining their search as needed; read online, print, or email full-text content; utilize suggested related methods and links to related authors from SAGE Research Methods' robust library and unique features; and even share their own collections of content through Methods Lists. SAGE Research Methods contains content from over 720 books, dictionaries, encyclopedias, and handbooks, the entire “Little Green Book,” and "Little Blue Book” series, two Major Works collating a selection of journal articles, and specially commissioned videos.