A. Feder Cooper

To my knowledge, 2 papers of mine have been put through LLMs and reposted on arXiv/ResearchGate as novel work. 1 was published in a Nature subsidiary (+ verbatim copies figures from my 2021 paper). The 2nd popped up today in my arXiv digest, is a sloppified version of a paper I put up in early July.

*frantically scrolling this grant award announcement to see if my brilliant friends got their bag*

In our work on extracting memorized pieces of books from open-weight LLMs, we measure how likely models are to output training text verbatim. But near-verbatim cases matter too. E.g., high-probability continuations for this Gatsby quote differ from the original only by punctuation:

The blog post walks through the algorithm with interactive animations; the paper has the full results, including other models and kinds of text. Blog post: afedercooper.info/near-verbatim Paper: arxiv.org/abs/2603.24917

In our work on extracting memorized pieces of books from open-weight LLMs, we measure how likely models are to output training text verbatim. But near-verbatim cases matter too. E.g., high-probability continuations for this Gatsby quote differ from the original only by punctuation:

Why not just sample to compute the near-verbatim probability? Well, Monte Carlo can take thousands (or hundreds of thousands) of samples to settle on a reliable estimate. k-CBS gets a useful lower bound close to the Monte Carlo estimate for the cost of just ~20 samples.

In our work on extracting memorized pieces of books from open-weight LLMs, we measure how likely models are to output training text verbatim. But near-verbatim cases matter too. E.g., high-probability continuations for this Gatsby quote differ from the original only by punctuation:

Our large-scale study of memorization of books in open-weight LLMs (e.g., Llama, Qwen) will appear at the 2026 Conference on Language Modeling as an oral. We made a website for exploring our results on 200 books and 14 models: books-memorization.github.io

We make very carefully scoped claims, supported by an extensive set of baselines on non-training data. Without baselines, it's not possible to rigorously separate a model's ability to generate fluent text from extraction of training data. (We see no false positives for non-training books.)

Our large-scale study of memorization of books in open-weight LLMs (e.g., Llama, Qwen) will appear at the 2026 Conference on Language Modeling as an oral. We made a website for exploring our results on 200 books and 14 models: books-memorization.github.io

3. In some cases, memorization is so extensive that one can deterministically(!) extract a near-pristine copy of a whole book using the book's first few words as an initial prompt. Here's a portion of the diff between the real (British) Harry Potter and the one we generated with Llama 3.1 70B.

Our large-scale study of memorization of books in open-weight LLMs (e.g., Llama, Qwen) will appear at the 2026 Conference on Language Modeling as an oral. We made a website for exploring our results on 200 books and 14 models: books-memorization.github.io

1. Memorization varies both by model and book. With respect to our specific measurement methodology, we find that most LLMs don't memorize most books. In the heatmaps, white means not memorized; blue indicates memorization (with darker blue indicating higher extraction probability) However...

Our large-scale study of memorization of books in open-weight LLMs (e.g., Llama, Qwen) will appear at the 2026 Conference on Language Modeling as an oral. We made a website for exploring our results on 200 books and 14 models: books-memorization.github.io

I think this is a really important read. @masnick.com carefully explains what is so shocking about recent revelations in court about undisclosed COIs concerning research on copyright and LLMs. www.techdirt.com/2026/09/25/o...

In general, I think disclosing COIs in science is a necessary feature of all papers, and that failing to do so calls into question the legitimacy of the work. I think it's especially important in this subfield because of the broader harm non-disclosure can potentially bring about in the real world.

Like many of my colleagues, I'm receiving dozens of EOI emails a day from prospective students. It's worth reading those emails before clicking send. (I expect most are using AI to draft emails. This tiny prompt injection is just to see if the sender bothered to even read the output.)

Prospective copyright plaintiffs have started asking me my opinion about citing "Alignment Whack-a-Mole" in litigation. It reports that fine-tuning makes frontier LLMs reproduce up to 85-90% of copyrighted books. I don't think the headline results hold up: afedercooper.info/whack-a-mole/

My blog post isn't an argument for or against fair use. I'm also not claiming that LLMs don't memorize books. I've spent years showing that they do. It's about whether this particular paper supports the enormous claims it makes. (It doesn't.)

Prospective copyright plaintiffs have started asking me my opinion about citing "Alignment Whack-a-Mole" in litigation. It reports that fine-tuning makes frontier LLMs reproduce up to 85-90% of copyrighted books. I don't think the headline results hold up: afedercooper.info/whack-a-mole/

That headline number comes from a metric the authors call bmc@5. bmc@5 counts very short matches between generations and books as evidence for memorization. Far shorter than what the field standard considers valid.