No AI Agents, No Scalability in Research Validation

Researchers have estimated that academics collectively spent over 100 million hours (~15,000 people years) reviewing ~4.7 million manuscripts and pre-prints every year for possible publishing in 30,000 scientific journals. That research only considered papers submitted to journals and did not consider the millions of papers that are posted to various preprint repositories. When those are considered, it becomes clear that

Read More »
Gaming Referee

Gaming is an expected outcome of any valued and successful system. This page anticipates numerous scenarios that may occur within Referee and how these techniques will be countered.

Read More »
Academic collegiality is a bug, not a feature (and it’s largely not true)

When I listen to people talk about improving peer review, I often detect a paradox: the importance of embracing academic collegiality while acknowledging the cutthroat nature of academia. They highlight concerns about potential retaliation if peer reviewer anonymity were compromised and relate researcher worries about competitors scooping their preprints. In addition, all the evidence suggests that most academics are reluctant

Read More »