Panic about AI slop has reached the labs: delve is everywhere, hallucinated references slip into bibliographies, and editors worry that models are homogenizing scientific writing. A four-author team at the New Jersey Institute of Technology chose measurement over debate. They pulled the full texts of roughly 2.5 million papers published from 2015 to 2025 across five disciplines — Computer Science, Math and Physics from arXiv, Health and Life Sciences from PubMed Central — and compared word usage, lexical diversity, terminology density, topic diversity, paper structure and collaboration before and after late 2022. Their verdict: LLMs rewrote the words of science, but have not yet transformed science itself. (Paper: https://arxiv.org/abs/2611.22975 )
Only the words moved
The team fitted segmented regressions to 14,484 English lemmas with a breakpoint at 2022: 796 lemmas surged after LLMs arrived, 199 declined, and 71 rose then dropped. Surge words such as establish and underscore belong to the analytic register LLMs prefer; decline words such as good and happen are the plain alternatives that model-edited prose avoids. The emblem is delve: nearly absent before 2023, peaking in 2024, falling sharply in 2025 — which the authors attribute to model updates and growing awareness of AI slop. The signal is strongest in Computer Science and close to zero in Math, where formal notation constrains the prose. Zoom out, though: the affected list holds slightly over a thousand words against tens of millions of unique words in the corpus, so the vocabulary of scientific writing kept its composition.
Diversity did not collapse; the measurement misbehaved
The scarier indicators do not survive contact either. Lexical diversity, measured by a moving-average type-token ratio, shows no statistically significant slope change in 2022 for any of the five disciplines. The density of scientific terminology keeps rising in all five — papers read as more scientific over time, not more diluted. The post-2022 plunge in biomedical topic diversity looks alarming until you notice where the similarity comes from: publishing metadata, licensing statements and contributor-role boilerplate. Filtering the top 105 boilerplate keyphrases removes most of the decline. The convergence of Computer Science topics is real, but it started well before 2022 as the field coalesced around deep learning, and it has slowed slightly since 2021.
Structure and collaboration: old trends, same slope
Structure tells the same story. Across 437,980 CS papers, the share with only 1 to 3 sections fell from 17% to 4%, papers with 8 or more sections grew from 5% to 30%, Related Work coverage rose from 32% to 66%, and the standard five-section template grew from 25% to 55% of papers; introductions were cut from 2141 to 1236 words on average while references grew from 682 to 1261 words. Every one of these rides a trend laid down before LLMs. Collaboration kept its long climb in team size after 2022. One exception stands out: the number of coauthors on Computer Science papers from Asia accelerated after 2022, and paper counts in CS and Physics look amplified. The authors place this inside the frame of LLMs lowering cross-lingual friction, while stressing that it extends an old trend rather than starting a new wave.
Independent corroboration, and cold water
This is not a lone datapoint. A separate team, whose study of 2.1 million preprints across four repositories (arXiv, bioRxiv, medRxiv, SocArXiv) was submitted in October 2025, concludes that LLMs act less as universal disruptors than as selective catalysts: submission and revision cycles accelerated, linguistic complexity rose modestly, and AI-related topics expanded disproportionately. Two independent teams, two multi-million-document corpora, one direction. (Cross-check: https://arxiv.org/abs/2510.17882 )
The cold water: every number above is self-reported by the papers, with no third-party replication; the data ends in 2025 while the authors themselves call the speed of LLM penetration concerning; and the AI paradox documented by Hao et al. in Nature — individual output expanding while the collective focus of science contracts — lives on a layer that word statistics cannot referee. The words are fine; that does not settle the science.
So the next time a soaring delve count is offered as proof that AI ruined science, ask for the baseline. The real contribution of this study is methodological: calibrate a decade of trend first, then argue about whether a revolution happened.