California Lawyers Grapple with AI Hallucinations in Legal Work
San Francisco (AP) — California attorneys are increasingly encountering AI-generated "hallucinations" in legal research tools, prompting warnings about overreliance on artificial intelligence in court filings and document preparation, according to a recent report by the San Francisco Chronicle.
The issue has emerged as law firms and legal departments adopt AI tools to streamline tasks like case research and drafting motions. However, attorneys report encountering fabricated case citations and distorted legal precedents generated by AI systems, with one lawyer noting "dozens" of hallucinated references in recent filings.
"AI tools are powerful but not infallible," said UC Hastings College of Law professor Emily Zhang, who has studied AI adoption in legal practice. "When systems generate plausible-sounding but entirely false legal citations, it creates risks for practitioners who don’t verify sources."
The California State Bar has issued guidance urging lawyers to maintain "human oversight" when using AI, particularly for tasks involving judicial filings. Some firms are implementing verification protocols requiring manual cross-checking of AI-generated references against official court databases.
Legal tech companies are responding with updates to their systems. Casetext, a legal research platform, announced this month it would integrate "hallucination detection" features into its AI tools, flagging citations that don’t match records in state and federal court systems.