No. Retrieval-augmented generation (RAG) makes the AI search for documents before it drafts its answer, but it does not eliminate hallucinations. A Stanford and Yale study (2024) measured 17% to 33% of hallucinated answers across three US legal research tools, on questions chosen to be demanding.
A paper published in 2020 by researchers from Facebook AI Research, University College London and New York University (Lewis et al., arXiv 2005.11401) describes retrieval-augmented generation models that combine two memories:
The authors started from an observation: large language models store knowledge, but knowing where an answer comes from and updating that knowledge remained open problems. In their text generation tests, RAG models produced more specific and more factual text than a baseline model without retrieval.
A study by researchers at Stanford and Yale sums up the process in two stages (Magesh et al., arXiv 2405.20362):
The authors describe it as working "open book": the model can rely on the text it is given, and not only on its training memory.
Two studies, which partly share the same authors, have measured these hallucinations, both on US law.
| Study | What is tested | Published result |
|---|---|---|
| Dahl et al., "Large Legal Fictions", 2024 (arXiv 2401.01301) | General-purpose models, including GPT-4, questioned on decisions of US federal courts | Hallucinations at least 58% of the time, on the model versions tested |
| Magesh et al., "Hallucination-Free?", 2024 (arXiv 2405.20362) | Lexis+ AI, Westlaw AI-Assisted Research and Ask Practical Law AI, RAG-based legal tools, and GPT-4 | In tests run from March to May 2024, each legal tool hallucinated between 17% and 33% of the time, against 43% for GPT-4 |
The second study looks directly at legal research tools. It was preregistered, covers more than 200 legal questions, and expressly targets the commercial claims that RAG "eliminates" or "avoids" hallucinations. Its conclusion: those claims are overstated. The tools hallucinated less than GPT-4 but remained far from zero. Two of them answered fewer questions than GPT-4 without giving significantly more reliable answers when they did answer. The authors specify that their evaluation only holds for the test period: the tools' answers changed during the study. They also specify that their questions were chosen to be demanding and that these rates are not meant to estimate the average frequency of errors in everyday use: they serve to check whether RAG had made hallucinations disappear, as the vendors claimed.
Neither of these studies tested a French tool or a question of French law: their percentages cannot be transposed as they stand. Their typology of errors can, however, serve as a framework for testing a tool.
The Stanford and Yale study defines hallucination more broadly than an invented decision. A hallucinated answer contains either incorrect information or the false assertion that a source supports a proposition.
It distinguishes two dimensions:
| Dimension | What is checked | Classification by the study |
|---|---|---|
| Correctness | Is the answer correct and relevant to the question? | Incorrect answer: hallucination; refusal or off-topic answer: incomplete answer |
| Groundedness, for a correct answer | Do the key assertions point to relevant documents that support them? | Source cited but misread or inapplicable, a "misgrounded" answer: hallucination; key assertions without a source: incomplete answer, not a hallucination |
With a misgrounded answer, the cited source does not say what the answer makes it say, or does not apply. A check that stops at whether the reference exists does not catch this flaw. The study also describes the case of a tool which, asked about the notable decisions of a fictitious judge, cited a real decision that had not been written by that judge.
The authors identify three difficulties specific to the legal domain:
They also note that when a special rule conflicts with a general rule that is far more present in the model's training data, the model may stay more faithful to that data than to the retrieved documents.
To spot a misgrounded answer, the study notes that you have to open the cited references, read the sources, assess their authority and compare them with the propositions in the answer. The first three criteria below follow from this; the last two concern the database queried and the premises of the question. To test them, questions to which the lawyer already knows the answer work best:
The method for checking a decision, database by database, is set out in our article on verifying case law cited by an AI. For the risk specific to general-purpose tools, see Is ChatGPT reliable for law?, and for choosing a tool, our comparison of the best legal AI for lawyers.
Plato meets the same need for verification through traceability: every figure points to the document it comes from, which lets the lawyer check the source themselves before using it.
What does RAG stand for in artificial intelligence?
Retrieval-Augmented Generation (in French, génération augmentée par récupération). The system searches for documents in a database, then supplies them to the language model together with the question so that it drafts its answer.
Can a RAG-based legal AI still hallucinate?
Yes. In tests run in 2024, the Stanford and Yale study measured between 17% and 33% of hallucinated answers across three RAG-based US legal research tools, far from zero, on questions chosen to be demanding.
What is a "misgrounded" answer?
A correct answer whose key assertions cite a source that does not support them: the source is misread or inapplicable. The study counts it as a hallucination.
Do the Stanford and Yale figures apply to French law?
No, not as they stand: both studies concern questions of US law. Their typology of errors can serve as a framework for testing a tool.
Why is RAG harder in law?
Because the answer depends on the jurisdiction, the date and the conditions under which a rule applies, and a document close in wording may be inapplicable. The authors also point to the difficulty of putting several sources together.
How do you test a legal AI tool before adopting it?
With questions to which you know the answer: check that each source is cited, accessible and applicable and that it really says what the answer attributes to it, and see how the tool reacts to a false premise.