Why checking comes first
Trusting the output
- Fluent but possibly invented
- Fake cases look real
- Errors reach the filing
Verifying the output
- Each source opened and read
- Quotes matched to the original
- Errors caught before filing
Language models can produce confident text that is simply false. Researchers call this hallucination, content that is fabricated or unsupported by sources but delivered as fluently as accurate output.
Legal tools built for lawyers reduce the problem but do not remove it. Stanford researchers found that products from two major legal publishers each produced wrong information more than 17% of the time.
Lawyers know the risk. In the ABA's 2024 technology survey, three-quarters of those holding back on AI named hallucinations as the reason.