(Credit: Anadolu / Contributor / Andadolu via Getty Images)
A California attorney made an expensive mistake when trying to cut corners on a legal brief.
Amir Mostafavi submitted an appeal in an employment-related case, but 21 of the 23 cases he cited to support his argument were fake—hallucinated by AI—or included phony quotes from existing cases. Judge Lee Smalley Edmon sanctioned Mostafavi and fined him $10,000.
"To state the obvious, it is a fundamental duty of attorneys to read the legal authorities they cite in appellate briefs," Judge Edmon says in a strongly worded opinion. "Plainly, counsel did not read the cases he cited before filing his appellate briefs: Had he read them, he would have discovered, as we did, that the cases did not contain the language he purported to quote, did not support the propositions for which they were cited, or did not exist."
Mostafavi claims he wrote the first draft of the brief but then used AI tools such as ChatGPT, Grok, Gemini, and Claude to "enhance" it. He did not read through the final version before filing it, and says he should not be fined because he did not know AI tools can make up information.
Mostavi passed the California bar exam in 2012, a decade before OpenAI claimed in 2023 that ChatGPT was smart enough to master that test. Mostavi was also a part-time law professor at the People's College of Law in Los Angeles, an unaccredited school that shut down in 2023, according to his website. He told the court he has educated himself about AI hallucinations since being called out.
Judge Edmon does not condemn the use of AI in the legal profession, and in fact says, "there is nothing inherently wrong with an attorney appropriately using AI in a law practice." But attorneys who use AI must carefully fact-check every citation, and "cannot delegate that role to AI, computers, robots, or any other form of technology."
The use of inaccurate, AI-generated text in legal briefings is becoming "far too common," the judge adds. In 2024, a Stanford law professor was caught using AI-generated citations in a legal argument that was, ironically, in support of a bill to curb deepfakes. In 2023, two New York lawyers were sanctioned after submitting a brief that included fake ChatGPT citations.
The issue isn't limited to lawyers. Alaska's top education official used AI to draft a state-wide school policy on cell phone use in 2024, and four of the six studies cited were made up. Education Commissioner Deena Bishop admitted to using generative AI to draft the resolution and says it was an early version posted online prematurely.
AI companies don't have a solution to stop hallucinations. In a paper published this month, OpenAI said models make up information when they do not know the answer, and they are trained to reward guesswork. They also want to please users, who may not be happy if the AI cannot answer their question, The Register reports. So, they make stuff up.
In a legal context, if an AI cannot find a suitable case for an attorney to cite, it might make one up. Judge Edmon argues this wastes the court's time and resources to double-check every citation in submitted documents, and is not a tenable long-term solution for the legal system.