Short answer: to understand a notion, yes. To cite case law, no, not on its own. ChatGPT, Claude and Gemini produce references that look accurate but do not exist, a risk already noted by French courts in 2025. A specialised legal AI like Plato avoids it: every reference points to a real, verifiable decision.
A hallucination is a piece of information generated by an AI that looks credible but is false: an invented ruling, an appeal number that matches nothing, a citation attributed to a non-existent decision. The model does not "lie", it statistically completes a plausible answer, with no verifiable base of truth.
In law, the problem is acute: a false reference has the exact appearance of a real one (court, date, number), which makes it hard to spot without systematic verification.
Yes, and it is no longer theoretical. Courts have found it in recent decisions. The table below sums up the documented cases in France and abroad.
| Decision | Date | What happened | Penalty |
|---|---|---|---|
| Administrative court of Grenoble (no. 2509827, no. 2512468) | Dec. 2025 | Fictitious case-law references from a generative AI, in the submissions of unrepresented litigants | None (finding) |
| Administrative court of Orléans (no. 2506461) | 29 Dec. 2025 | Around fifteen entirely false references cited by a lawyer | Warning, no financial penalty |
| Mata v. Avianca (SDNY, United States) | June 2023 | Six decisions invented by ChatGPT in a brief | 5,000 dollar fine |
| Johnson v. Dunn (N.D. Alabama, United States) | July 2025 | False AI-generated citations | Lawyer removed from the proceedings |
In France, the judges have so far taken an educational approach: they note the phenomenon and warn, without sanctioning. Nothing guarantees this tolerance will last.
Beyond the credibility lost before the judge, the lawyer engages their duty of competence. Using a tool without knowing it can hallucinate, then producing its results without checking them, is a professional failing. The responsibility stays the lawyer's, not the tool's: technology does not exempt anyone from the obligation to verify their sources.
Three simple rules cut the risk to almost zero.
A general-purpose AI draws on broad, unverified knowledge. A vertical legal tool works on a base of identified decisions and ties each statement to its source. For valuing a personal injury or an employment dispute, where every amount must be enforceable, the second approach avoids hallucination by design.
Plato, the only AI specialised in personal injury, is built on this principle: every amount is tied to real decisions from its base of more than 84,000 quantified decisions, and stays traceable back to the exhibit it comes from. Another practical difference: where a general-purpose AI like Claude caps the number of documents (around 20 PDFs), Plato handles an entire file, often several hundred exhibits. For the detail of the differences, see our comparison Plato against ChatGPT, Claude and Gemini and the page Plato for personal injury.
On the substance, the guard against hallucination is methodological: knowing where the decisions are and how to verify them, database by database: see how to verify case law cited by an AI. See also how to find quantified case law in personal injury, which legal AI to choose by use and, on the tooling side, which software to value the Dintilhac heads. Reliability is only one side of the problem: on confidentiality, see whether a legal AI can respect the lawyer's professional secrecy.
Can ChatGPT cite case law that does not exist?
Yes. It generates plausible references from probabilities, with no base of truth. French courts (Grenoble, Orléans) noted false AI-generated references in December 2025.
Has a lawyer ever been sanctioned for this?
In France, not yet: the judges have warned without sanctioning. In the United States, yes: in Mata v. Avianca (2023), two lawyers were fined 5,000 dollars.
How do you know whether an AI has invented a decision?
By verifying the reference in a primary source (Légifrance, Judilibre or the court's base). A reference that cannot be found should raise a flag; but Légifrance lists only a selection of the lower-court decisions and Judilibre covers appeal courts only since April 2022, so "not found" does not always mean "invented". When in doubt, a reference that cannot be verified is not cited.
Does a specialised legal AI hallucinate too?
The risk drops sharply when the tool ties its answers to real, verifiable decisions instead of generating from memory. This is the principle of vertical tools like Plato.
Can you use ChatGPT for law anyway?
Yes, to explain a notion, structure reasoning or rephrase a text. Not to produce references without verifying them one by one.
Is the lawyer responsible for the AI's errors?
Yes. The responsibility stays the lawyer's, under their duty of competence. Using a tool does not exempt them from verifying their sources.