Evidence-based policy-making requires not only evidence, but also for policymakers to update appropriately based on that evidence. We examine how policymakers, researchers, and others update in response to results from multiple studies, using a unique opportunity to run an experiment on policymakers. We find evidence of ‘variance neglect’, a bias similar to extension neglect in which confidence intervals are ignored. We also find evidence of increased updating on positive results. This means that policymakers might be biased towards those interventions with a greater dispersion of results. We test ways to mitigate the observed biases.