OpenAI and Microsoft knew they were starting a ‘doom loop’ for the web

Recently unsealed court documents in the New York Times’ case against OpenAI and Microsoft are pretty damning. The companies’ own documentation warned that it was starting a “doom loop” that would damage the web, characterized its scraping of data to train its models as the “largest theft of labor in human history,” and that it made a “complete mockery of the idea of fair use.”

Many of the most eye-catching quotes from the document come from Microsoft’s Director of Applied Science, Brent Hecht. Though, the company has tried to distance itself from Hecht’s assertions. Microsoft spokesperson Alex Haurek told The Verge that “These comments reflect one employee’s individual perspective, are not a legal analysis, and do not represent the company’s views.”

In a separate court filing, Jordan Usdan, GM for Data Strategy and Ops at Microsoft AI, characterized Hecht’s role as adversarial. He said that Hecht “holds divergent, academic, and forward-looking views about how data ecosystems for AI should operate and is employed at Microsoft to bring asymmetrical, futuristic, and academic points of view … nor is he someone who speaks for Microsoft specifically as to his theoretical views on AI’s potential effect on content creators.”

But whether or not Microsoft wants to own these comments, it’s clear that this came true. Google Zero is real! AI is eating the web!

There are plenty more wild statements in NYT’s filing from a variety of figures, including Satya Nadella, Sam Altman, and other OpenAI employees. Here are some highlights from the 92 page document.

“An astonishing theft”

The introduction quotes Hecht and OpenAI’s Head of ChatGPT (presumably Nick Turley) in a way that seems to show the companies knew they posed an “existential threat” to publishers like the New York Times. Hecht calls ChatGPT and Copilot’s harvesting of data the “largest theft of labor in human history” and says that Microsoft’s defense makes a “complete mockery of the idea of ‘fair use.’”

Satya Nadella admits that chatbots have basically replaced search and removed the need to go straight to the source for info. But perhaps more damning is an internal Microsoft document that says, “Our AI content strategy has started a ‘doom loop’ that will hurt the performance of our models and the entire web at the same time: It is highly unusual that an end-product threatens the economic foundations of its essential suppliers, but that is the situation we have created for our LLM business with respect to its ‘content supply chain.’”

That’s not even a real number

Don’t be fooled by OpenAI or Microsoft’s claims of altruistic intent. OpenAI cofounder Greg Brockman is more interested in the “gazillions” of dollars it he could potentially make through commercial AI.

Despite Nadella later being quoted as saying, “anything that is paywalled should be licensed,” An OpenAI representative admitted that he was “unaware” of any effort to detect or remove paywalled content from training data.

“Insanely good at regurgitation”

Internally, it seems that OpenAI was well aware of ChatGPT’s tendency to simply reproduce copyrighted material “verbatim.” Even though it acknowledged that the “prevention of memorization” was important to “minimize copyright violations,” employees admitted that GPT-4 “memorized a ton of data and therefore will be insanely good at regurgitation.”

The filing then goes on to cite several examples of ChatGPT outputting long strings of copy straight from articles in the Times, Mercury News, The Denver Post, LifeHacker, and Eurogamer in response to queries.

“‘Hoovering up’ all their work”

Microsoft knew how its wholesale scraping of the internet would be perceived and admitted that “almost no one intended for they [sic] content they created to be used in this fashion, nor are they compensated for its use.”

A “substitute for the labor of people”

OpenAI Policy Director Jack Clark saw the writing on the wall, saying that it was “creating systems that substitute for the labor of the people that define the ‘culture’ of society.” Internal documents described ChatGPT as “the modern newsstand.” OpenAI’s Nick Turley is later quoted as saying that once you get an answer from its chatbot, there is “no good reason to click” on a link to the source.

Destroying their own supply chain

Microsoft is quoted as admitting that “LLMs are a product that destroys its own supply chain” because it’s a substitute for its own training data in many cases.

OpenAI knows its killing referral traffic

OpenAI’s own media and economic experts attributed the drop in referral traffic for sites like the Times directly to AI summaries like Google’s AI Overviews. They’ve speculated that search referrals may be down as much as 60 percent.

Microsoft spokesperson Haurek cautioned that “Satya’s testimony and Microsoft’s position in this case are perfectly consistent. He spoke to broad principles and changes underway in how people find and consume information. Those observations should not be confused with conclusions about copyright questions before the Court, which Microsoft addresses in its filings.”

But it seems pretty clear based on this newly unsealed document that both Microsoft and OpenAI knew they were going to irreparably harm the publishing industry, the “millions of people” it employs, and, by extension, damage their own product, but carried forward anyway in pursuit of “gazillions” of dollars — doom loop be damned.