What if AI can make Science Better, not just Faster?

Producing science has never been cheaper. Trusting it has never been harder.

I believe that the current model of academic publishing in journals and conferences is soon to be over. It’s been struggling for a while but I think it’s hard to look at the trends and think that the process can be saved. It’s a bigger question whether it’s worth trying.

Many AI/ML conferences have seen submissions multiply 2-3x while journals are struggling to find reviewers and maintain quality. Desk rejection rates, where public, have risen dramatically. Authors report longer times to publication and receiving AI-generated/supported feedback anyway. Preprints are quickly becoming an authoritative source.

I think many people believe there is a silver bullet for this somewhere. I haven’t seen any that scale or are not just stopgaps. Detection can be defeated and focusing on it creates markets where people will invest in tools to make this easier. Disclosure does nothing to indicate if changes are material to findings.

These ideas assume that filtering out AI preserves a high-quality status quo. Randomised trials of reviewers find them to miss intentionally placed errors. Even if caught in writing, it is plainly impossible for reviewers to check every line of code (if it’s available) and test if claims hold up on data (if it’s accessible). The process relies on trust and an implicit assumption that someone investing time and effort into producing a manuscript means that similar effort has been expended on the analysis and verification.

AI usage by itself says nothing about the underlying work. Fully AI-generated code with a human-written manuscript defeats almost all detection (and may have far better code than a majority of research). In the inverse, Claude can write a Nature Med paper from the published analysis. The writing will sound off and may be ‘load-bearing’ or rather dense, but should it be lesser if the evidence and conclusions are the same?

These problems are going to get worse. The growth of agentic tooling is going to make it easier than ever before to produce code, papers, grant proposals, and more in a time where verification demand far outstrips supply.

Reviewers donate over 175 million hours a year to review papers for publishers. This estimate is conservative as it doesn’t consider conferences and other forms of review that academics likely perform. Journal publications are growing 2-3x faster than the number of academics. Preprint volumes are growing 4-8x faster. By any number this ratio is unsustainable, even without AI acceleration.

We cannot know how much review capacity we have available, but if we assume we’ve used some fraction of it, we can determine how long we can sustain current growth rates before that capacity would be exhausted. arXiv growth is used as a proxy for what submission volumes may soon be.

The problem is one of a growing number that deal with alignment. The goals of producing more research are not inherently the same as producing more knowledge or better science. This has been true longer than AI has been around and means that acceleration widens the divide.

Current research challenges mix new problems with the old, many of which have not been meaningfully solved.

Fixing this is going to require hard choices. We need to rethink how we verify work at scale, what the future of scientific communication looks like, and how we align incentives for production with quality and reproducibility. It may even require us to give up the notion of a ‘paper’ and adopt a different taxonomy of evidence and ways of sharing knowledge. I don’t believe this will be simple, but the benefits are likely to far outweigh the cost. Think about how much value could be created if implementations were reusable by default and findings could be trusted to be built upon. How many times have methods been found to be weaker than reported and are still widely used? How many hundreds of hours have been spent using approaches that don’t generalise and undermine downstream work?

My fear is that easy solutions will prevail. That we will deploy agents to simulate an already deficient peer review system and keep the current system running with bandaids and prayers, rather than investing in the hard work to think what could (or should) be.

Accelerating science should be an easy net good for the world. If code is cheap, it should be required to produce reproducible implementations and claims that can be verified. It should be effortless to produce ways of communicating findings that can be easily understood and shared. This will require new standards of practice and mechanisms of review that focus human attention where it is needed, rather than treating reviewers as a verification sink.

AI presents an extraordinary possibility to learn and do more at a faster pace than at any point in history. We need to work on methods and thinking that align this with what makes science better, rather than what produces more incremental papers.

Slides and ideas are an excerpt from my talk: “What if AI can make science better, not just faster?” from the Encode: AI for Science cohort 2 kickoff.