Free tools Windows power users keep installed
One-click scans. No signup required.
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Some subjectivity in peer review may help a scientific community resist “herding”—but the finding comes from a mathematical model, not a test of real journals. In a 2013 paper published in Nature’s 2014 issue, In-Uck Park, Mike W. Peacey and Marcus R. Munafò modeled how reviewer decisions can influence what researchers believe and publish. Their results suggest that a non-excessive degree of subjectivity can help the community estimate truth more accurately under the conditions they modeled.
How peer review could contribute to a false consensus
Peer review is often treated as a filter for assessing individual papers. Park, Peacey and Munafò examined a broader possibility: review and publication can also shape what researchers infer from one another’s decisions. Their model explores how that influence affects the scientific community’s ability to identify which of competing hypotheses is more likely to be true. The Nature paper describes peer behavior as a potential influence even when researchers are motivated to promote truth.
What herding means in the model
Herding occurs when researchers’ choices are shaped, or eventually dominated, by information they infer from peers’ behavior rather than by their own prior beliefs alone. If those signals lead people to follow an early, mistaken conclusion, later decisions can reinforce it. In some modeled conditions, the community can therefore converge on an incorrect answer.
What the researchers modeled
The paper models reviewer decision-making and the way publication affects the aggregation of knowledge. A contemporaneous Chemistry World account describes a simplified scenario in which researchers start with beliefs about competing hypotheses, submit papers, and update their beliefs as they encounter peer review and publication. This is a theoretical account of decision dynamics, not an experiment tracking researchers or journals in the field.
#1 Best Overall
Objective-only review versus some subjectivity
The model compares reviewer decisions that are objective-only with decisions that allow some subjectivity. In the Chemistry World account of the model, objective-only review leaves the modeled probability of backing a false hypothesis at a stable minimum. With a non-excessive degree of subjectivity, that probability can continue to fall, as herding is curbed and information is processed more effectively.
The authors summarize the proposed benefit in their abstract: “We further demonstrate that exercising some subjectivity in reviewer decisions, which serves to curb the herding process, can be beneficial for the scientific community in processing available information to estimate truth more accurately.” This is a conditional result of their model—not a finding that any subjective judgment improves review.
Why the result is not proof that subjectivity prevents false findings
- It is a model, not a field test. The paper investigates possible behavior under modeled assumptions; it does not show that actual journals reliably reduce false findings by allowing subjective decisions.
- The proposed benefit is limited. The account describes helpful subjectivity as non-excessive. The result does not support the claim that more subjectivity is always better.
- Timing matters in the modeled account. Chemistry World notes that stopping publication too soon may leave a modeled reversal of herding incomplete. That is a feature of the described scenario, not a measured rule for when real journals should publish.
- It does not establish a universal effect. The available sources explain this paper’s model and its contemporary reception; they do not establish that the predicted benefit holds across disciplines or review systems.
Munafò, identified in the contemporaneous account as a University of Bristol researcher, put the result cautiously: “Our results suggest that peer review may not perform as badly as sometimes thought.” That statement reflects the paper’s modeled conclusion, not evidence that peer review always works well. Chemistry World also reported a practitioner’s view from Nicola Nugent, then the Royal Society of Chemistry’s assistant manager for peer review: “A healthy balance of objectivity and subjectivity in the review process is crucial.” This is a separate professional comment, not a result of the model.
When the paper appeared
Park, I.-U., Peacey, M. W., and Munafò, M. R., “Modelling the effects of subjective and objective decision making in scientific peer review,” appeared online on 4 December 2013 and in Nature volume 506, issue 7486, pages 93–96, dated 6 February 2014. The PubMed record lists the bibliographic details and DOI 10.1038/nature12786.
Quick Recap
Best Value
Rank #3
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
