Tech firms are buying and destroying real books to train AI

SHAH ALAM – To make artificial intelligence (AI) models write better, tech companies have been buying millions of physical books, cutting off their spines and destroying them after scanning the pages, all to feed the huge amounts of data these systems need.

At the heart of the controversial methodology is artificial intelligence startup Anthropic.

Based on international news reports, through a secret internal initiative dubbed "Project Panama," internal documents unsealed in an author-led copyright lawsuit showed that the company purchased millions of used books to train its flagship Claude model.

Rather than relying on standard non-destructive scanning or traditional digital formatting, court filings reveal workers utilised a "hydraulic powered cutting machine" to "neatly cut" off book spines before running pages through "high speed, high quality, production level scanners.”

Once ingested, the stripped volumes were promptly handed over to a recycling company.

According to a report by science and technology news website Futurism, internal documents unsealed from the lawsuit showed Anthropic leadership viewed pre-2023 human-authored books as essential data.

High-ranking executives noted that physical literature would teach AI models "how to write well" instead of mimicking "low-quality internet speak.”

As highlighted by data platform ISBNdb in 404 Media, older print volumes offer a massive structural advantage because physical books published before the generative AI boom are "structurally clean of modern poisoning tools" and guaranteed to be free from synthetic "AI slop."

It was reported that despite the aggressive data harvesting, Anthropic knew the public backlash would be severe if the secret got out.

The court filings were unsealed in connection with a copyright lawsuit against Anthropic, which detailed Project Panama, which began in early 2024.

"We don’t want it to be known that we are working on this.", Anthropic said in internal planning documents.

Company leaders also recognised that "an AI company destroys two million books" was a public relations nightmare capable of ruining public trust.

The destructive process relies heavily on a legal loophole known as the first-sale doctrine, which allows buyers to do as they wished with legally purchased physical items.

As Anthropic transformed physical texts into digital files without republishing or redistributing the originals, a federal judge in San Francisco ruled in the Bartz v. Anthropic PBC case that the scanning constituted transformative fair use.

However, that legal victory was accompanied by a separate $1.5 billion settlement Anthropic agreed to pay after utilising pirated digital copies from shadow libraries like LibGen to train its models.

Anthropic’s destructive approach has now created a booming cottage industry, reshaping the secondhand book market globally.

Data intermediaries and platforms like ISBNdb now assist AI companies in sourcing anywhere between 1,000 to one million books per order, offering absolute anonymity to hide the practice from the public eye.

An independent bookseller told 404 Media that his weekly sales suddenly jumped from 20 books to several hundred, saying that while it benefits him financially by clearing old inventory, he genuinely dislikes that "uncommon books are being pulped."

Critics and industry players, including SpaceX AI lead Elon Musk, have voiced strong opposition, with Musk stating on X, that he instructed his team to preserve rare books in a library and "scan them the hard way vs just cutting off the spine and scanning."