IT IS to the running of the scientific enterprise what democracy is to the running of a country. , the UK’s national academy, where the took place in 1665.
Peer review is the system through which scientists vet each other’s work to judge whether or not it is fit to be published or funded. That makes it the very cornerstone of all research. And yet this foundation is cracked. described it as “biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish and frequently wrong”. that it is “complex, expensive and prone to abuse”.
These long-running concerns take on fresh resonance in the light of New Scientist‘s analysis of a highly competitive area of stem cell biology, which shows that researchers in the US get their papers published faster, and in higher-profile journals, than rivals elsewhere (see “Inside the stem cell wars”).
Advertisement
“US labs get their papers published faster, and in higher-profile journals, than rivals elsewhere”
We focused on stem cells because a group of scientists in that field – mostly from outside the US – recently took the unusual step of to leading journals protesting at “unreasonable or obstructive” peer reviews. But share a drink with scientists in other disciplines and you’ll hear similar accusations of researchers sabotaging competitors by raising excessive objections – or worse, stealing ideas. If there is any truth to these allegations, it could undermine scientific progress.
The problem with peer review is also its strength. Reviews are typically seen only by journal editors and then sent in anonymous form to authors of a submitted paper. This allows scientists to comment without fear of retribution, but the scope for abuse is obvious.
Even so, authors of the letter were surprised that our analysis found such a difference in the fate of papers from researchers in the US compared with those from elsewhere. Our study shows that it is possible to use publicly available information to probe the dynamics of publication, but fully explaining our findings would require journals to lift the veil of secrecy.
Are papers from scientists outside the “in-crowd” of top US labs given unfair, harsh treatment? Or are US-based scientists simply doing better work? Without seeing reviewers’ comments and learning their identities, we’ll never know. Hopefully our analysis will prompt journals to conduct internal audits to ensure that each paper, whether from Harvard University or a little-known lab in China, is considered on its merits.
Complete disclosure of peer review is unlikely. But the authors of the protest letter suggest a compromise, which should make abuses easier to spot while preserving the main strength of peer review: keep anonymity, so scientists aren’t discouraged from being critical, but publish the reviews and editorial correspondence. Indeed, already does this.
For justice to be seen to be done more widely, other journals should follow this lead. Like democracy, peer review may be flawed but it’s the least-worst system we’ve got. It should be nurtured, through exposure to a little more sunlight.