This Series provides participants with a detailed understanding of different types of impact evaluation techniques as well as an introduction to what factors need to be considered with evaluating a policy or program. Impact evaluations involve quantifying the effect of a policy or a program by estimating what would have taken place if it had not been implemented (i.e. the ‘counterfactual’).

The first day of the training focuses on the ‘gold standard’ of impact evaluation, while on the second day participants will gain insights about different kinds of policy evaluations (both qualitative and quantitative). To conclude participants will be guided through the process of designing an impact evaluation of one of the policies or programs they work on and will have an opportunity to pitch some of their ideas. The most promising idea will be matched with potential implementing partners.

The course is interactive in nature and is customised to the level and needs of the participants.

Policy Evaluation: Understanding the Evaluation Process

What the main factors that need to be considered when implementing an evaluation of public policies and what approaches should be used? This one-day course will equip participants so that they can learn how to identify the potential for, manage the implementation of and interpret the findings of different kinds of policy evaluations. They will learn the difference between outputs, outcomes and impact, as well as how to build and apply a theory of change. In addition, participants will be guided through the process of planning an impact evaluation of one of the policies or programs they work on.

Quantifying Policy Impact Using Randomized Control Trials

How can you know with certainty whether policies and programs are having their intended impact? There’s no simpler or more powerful tool for finding out than a Randomised Control Trial (RCT). RCTs help build evidence about what works – and what doesn’t. They are not just about maximising impact and saving money, they also help policy makers avoid unintended consequences. But how do they work? Are there any ethical issues? And how are RCTs being used by policy makers around the world? This one-day course provides participants with an overview of how RCTs are used to quantify policy impact and explains why they are considered the ‘gold standard’ compared to other types of policy evaluation

Topics to be covered:

• Overview of randomized control trials
• Applications of randomized control trials
• Determining the appropriate sample size
• Identifying opportunities for potential RCTs
• Introduction to the different types of evaluations
• Selecting an evaluation approach
• Managing the implementation of an evaluation
• Brainstorming ideas for potential impact evaluations

Outcomes:

Policy makers are often unsure if the intended impact of policies and/or programs are achieved because commonly used monitoring and evaluation approaches focus on input and outputs not outcomes. Public sector organisations often describe impact in terms of the amount of money spent or what that money is spent on as opposed to what the outcomes of policies/programs are. For example, the funding set aside for schools or the number of teachers trained is focused on as opposed to learning outcomes and the opportunities students have after school. This course on impact evaluations addresses this issue by providing policy makers with techniques to be able to measure the outcomes and impact of the policies and programs their organisations implements.

Anticipated behavioural and business impacts of the course include:

Participants will be equipped so that they can learn how to determine whether policies and programs have their intended outcomes and understand what changes can be made to increase their impact. The brainstorming and group work activity that is integrated into this course will provide participants with tangible examples that they can return back to their work places with and implement immediately. Furthermore, we will assist with matching the most promising idea that is proposed with a potential implementing partner (such as a researcher or non-government organisations) to fast track the anticipated impact.

Who should attend?

Policy makers and program managers between the levels of APS4 to EL2 (or equivalent). This course has been designed for those without a background in economics and participants do not require any quantitative skills. The content will be accessible so that participants without any exposure to monitoring and evaluation will still be able to benefit from the course. All that is required is some experience with working on public policy issues.