Ithaca College

The Many Labs 5 project assessed the replicability of 10 psychology studies in more than 100 labs all over the world.

According to critics, a well-known replication project in psychology (The Reproducibility Project: Psychology; Open Science Collaboration, 2015, Science) failed to replicate findings because of problems with the replication studies. The Many Labs 5 project conducted replications of these replication studies, and the results suggest a different conclusion.

A team of 171 researchers at 104 institutions around the world took part in the Many Labs 5 project, which worked with the original authors of the studies to design better replications. A massive increase in sample size and close collaboration with the original authors before conducting the 10 replication studies did not increase the replicability of the original findings.

One of the institutions that contributed to the Many Labs 5 project was Ithaca College, where Psychology Professor Leigh Ann Vaughn and her Social and Personality Research Team students replicated one of the studies. They found that neither the original nor revised version of the study replicated at Ithaca College. Across the 13 institutions that replicated that study, the revised version of the study produced the same results as in the original study, but much less strongly.

Studies like this are important because they can test common assumptions about the reproducibility of research findings. For example, the results of Many Labs 5 provide strong evidence that deficiencies in power and adherence to expert feedback do not explain failures to replicate these 10 studies. This has been a well-publicized assumption about why these findings did not replicate.

In addition to collecting data for Many Labs 5, Vaughn also helped design the project and write the papers describing its results. The Many Labs 5 project was published as 11 articles comprising Fall issue of the Association for Psychological Science journal Advances in Methods and Practices in Psychological Science on Nov. 13, 2020.