COVID-19

Quantifying policy impact using randomized control trials

Crawford School of Public Policy | Executive course
Policy Essentials

Summary

How can you know with certainty whether policies and programs are having their intended impact? There’s no simpler or more powerful tool for finding out than a Randomised Control Trial (RCT). RCTs help build evidence about what works – and what doesn’t. They are not just about maximising impact and saving money, they also help policy makers avoid unintended consequences. But how do they work? Are there any ethical issues? And how are RCTs being used by policy makers around the world? This one-day course provides participants with an overview of how RCTs are used to quantify policy impact and explains why they are considered the ‘gold standard’ compared to other types of policy evaluation.

See related courses: Policy Evaluation: Understanding the Evaluation Process

Course overview

Participants will be provided with a detailed understanding of different types of impact evaluation techniques as well as an introduction to what factors need to be considered with evaluating a policy or program. Impact evaluations involve quantifying the effect of a policy or a program by estimating what would have taken place if it had not been implemented (i.e. the ‘counterfactual’). The day will focus on the ‘gold standard’ of impact evaluation, which is using randomised control trials.

The course is interactive in nature and is customised to the level and needs of the participants.

Topics to be covered:

  • Overview of randomized control trials
  • Applications of randomized control trials
  • Determining the appropriate sample size
  • Identifying opportunities for potential RCTs

Outcomes:

Policy makers are often unsure if the intended impact of policies and/or programs are achieved because commonly used monitoring and evaluation approaches focus on input and outputs not outcomes. Public sector organisations often describe impact in terms of the amount of money spent or what that money is spent on as opposed to what the outcomes of policies/programs are. For example, the funding set aside for schools or the number of teachers trained is focused on as opposed to learning outcomes and the opportunities students have after school. This course on impact evaluations addresses this issue by providing policy makers with techniques to be able to measure the outcomes and impact of the policies and programs their organisations implements.

Anticipated behavioural and business impacts of the course include:

Participants will be equipped so that they can learn how to determine whether policies and programs have their intended outcomes and understand what changes can be made to increase their impact. The brainstorming and group work activity that is integrated into this course will provide participants with tangible examples that they can return back to their work places with and implement immediately. Furthermore, we will assist with matching the most promising idea that is proposed with a potential implementing partner (such as a researcher or non-government organisations) to fast track the anticipated impact.

Who should attend?

Policy makers and program managers between the levels of APS4 to EL2 (or equivalent). This course has been designed for those without a background in economics and participants do not require any quantitative skills. The content will be accessible so that participants without any exposure to monitoring and evaluation will still be able to benefit from the course. All that is required is some experience with working on public policy issues.

Course presenter(s)

Dr Christopher Hoy

Christopher Hoy is a Research and Policy Fellow at the Crawford School of Public Policy. He has conducted over 20 impact evaluations in 15 countries, including the first large-scale randomised control trials in Papua New Guinea and Fiji. He has worked as an impact evaluation expert for the World Bank and Asian Development Bank as well as a lecturer for a subject on impact evaluations at the University of Sydney. Christopher holds a PhD in Economics from the Australian National University and a Masters in International and Development Economics from Yale University.

Updated:  4 December 2024/Responsible Officer:  Crawford Engagement/Page Contact:  CAP Web Team