(Credit: James Leynse / Contributor via Getty Images)
The New York Times has sent a cease-and-desist letter to Jeff Bezos-backed Perplexity demanding the AI startup stop using its content.
The paper is alleging "unlawful use" of its articles—and wants to know how and why Perplexity's AI is still citing it, according to a copy of the Oct. 2 letter viewed by PCMag.
"Perplexity and its business partners have been unjustly enriched by using, without authorization, The Times’s expressive, carefully written and researched, and edited journalism without a license," the newspaper wrote in its letter, which gives Perplexity a deadline of Oct. 30 to respond. The Wall Street Journal first reported the news on Tuesday. A New York Times spokesperson declined to comment further beyond the content of the letter itself.
The Times accuses Perplexity of circumventing its anti-scraping and anti-bot measures. Its robots.txt page specifically disallows "PerplexityBot," the startup's scraping bot, though it's unclear if Perplexity uses others, as well, or other intermediary methods that indirectly pull from the outlet's content like a pre-collected dataset (PCMag's own robots.txt also disallows Perplexity). Robots.txt pages are rules that can be broken, however. So even if a site has one, "bad" bots can still scrape it.
In response, Perplexity CEO Aravind Srinivas says the firm wants to collaborate with the Times. "We are very much interested in working with every single publisher, including The New York Times," Srinivas told WSJ. "We have no interest in being anyone’s antagonist here."
A Perplexity rep tells PCMag via email that the firm will respond to the letter by the deadline. "We believe in transparency and have a public page on our website that clarifies our content policies and how we use web content," the rep says. "We aren't scraping data for building foundation models, but rather indexing web pages and surfacing content as citations to inform responses when a user asks a question."
In June, Perplexity told PCMag that its PerplexityBot "respects robots.txt." Because the bot runs on Amazon Web Services, Perplexity also said its scraping bot isn't "crawling in any way that violates AWS Terms of Service."
But some tech and AI firms have also taken the stance that scraping any site they want constitutes "fair use," though that has yet to be proven in court. Many AI firms may also be desperate for fresh, human-generated data pilfered for free. One professor has warned that AI companies are "running out of text" on which to train their chatbots.
Regardless of what Perplexity says it is or isn't doing, news outlets aren't happy. Condé Nast, which owns Wired, The New Yorker, and Vogue, previously sent Perplexity a cease and desist, alleging it's been scraping its sites and using that content for its own financial gain. Forbes has also fired shots at the AI firm, accusing it of theft and creating "knockoff stories" based on Forbes articles.
Other AI firms have also come under fire for using copyrighted content without consent or payment. While many continue to scrape the web anyway, some have decided to strike content licensing deals with news outlets too. Associated Press, The Atlantic, Financial Times, Semafor, Business Insider, Dotdash Meredith, Vox Media, and even the WSJ itself are all part of AI licensing deals struck by their respective leaders or parent companies.
Other news outlets, however, are trying to hold AI firms accountable for swiping their content without permission. The New York Times filed its lawsuit against OpenAI and Microsoft late last year, which remains ongoing. They've since exchanged some jabs, with OpenAI accusing NYT of "hacking" ChatGPT, which the paper denies. Microsoft, similarly, has defended OpenAI, claiming the paper is boosting "doomsday futurology" with the lawsuit.
In April, a collection of over half a dozen newspapers including the Orange County Register and the New York Daily News also sued OpenAI and Microsoft for similar reasons.
Perplexity, specifically, has been criticized for using news outlets' stories to generate its own, without adequately citing or linking to its source material in a way that's clearly visible. Unsurprisingly, however, Perplexity doesn't see it that way.
Editor's Note: This story has been updated to include comment from Perplexity and The New York Times.