Replication Crisis

By Douglas, February 12, 2017

This is the first of a series of blog posts I plan to write that address some common questions or discussion points with WikiLogic.

The replication crisis is something that frequently comes up when I talk to academics about WL. This is the name given to a recently exposed problem where many different studies, some quite famous, have been shown to fail upon replication. Initially this should not seem like an issue, as a cornerstone of science involves reproducing studies to test for consistency which we do under the expectation that some will likley fail due to unforeseen or unaccounted variables. So why is this a crisis in the scientific world?

The issue is that some of the experiments that have failed to be reproduced are heavily cited and may have numerous and/or highly important works based upon them.

To tackle this we need to funnel resources into going back and reproducing our experiments which adds an economic stress.  The question is how best to allocate our finite resources between new research and replication of existing ones?

This is where WL offers a powerful service. Using it will create a  database that can report back how much other information is reliant on any given claim. We can also take something we know to be highly important like Germ theory and ask WL to tell us which experiments were used to back this. These two methods give us both how much information relies on a study and if its foundational for some critical claim. Future additions can take into account other factors such as cost of replication and expertise available.

This will allow grant makers the ability to make intelligent choices about what gives the most benefit. It may also have other benefits such as helping guide students towards upcoming relevant areas of study and specialization.