February 25, 2025
The Chronicle of Higher Education recently published an article co-authored by Andy King, Allen and Kelli Questrom Professor in Strategy and Innovation, discussing how the social science publication process is facing a credibility crisis, with flawed data and questionable research often going uncorrected.
Many studies with sensational results are published despite being scientifically unsound, sometimes causing harm before they are corrected. Proposals to improve the system, such as better reviews and replication studies, have had limited success. King and Gelman provide a new approach, “replay review,” suggesting additional reviews for highly cited studies after they’ve had a significant impact.
“As with “booth review” in sports, replay review in science should include analysis from multiple angles, but also a clear assessment of how the “play” should be called, and whether the original conclusions were justified or not”, King adds.
This method would allow for more efficient use of resources and help address influential but flawed research, ultimately improving the reliability of social science publications.
