The current discourse surrounding Artificial Intelligence is often clouded by hype and misunderstanding. To leverage AI effectively—particularly in high-stakes fields like law—it is essential to separate the technology's actual capabilities from the persistent myths that surround it.

1. The Reasoning Trap: AI is Not Human

The most fundamental misunderstanding is the belief that AI thinks like a human. In reality, AI does not reason, understand, or possess intention; it is a sophisticated system for recognizing patterns and predicting outcomes based on data.

As expert Martin Milani notes, we often fall into a "category error" by mistaking fluency for reasoning. While a Large Language Model (LLM) can generate a "chain of thought," this is merely a sequence of generated steps rather than an execution trace of an actual inferential process. Just as a parrot can say "therefore" without being a logician, an AI can produce the language of reasoning without the underlying structure of thought. This distinction has legal teeth: in California, Civil Code section 1714.46 now prohibits the "AI did it" defense, holding developers and implementers responsible for the AI's actions.
 

2. The Iceberg Effect: Beyond Drafting

While AI is frequently used to draft, outline, and summarize, drafting is just the tip of the iceberg. For law firms, the most significant ROI (Return on Investment) does not come from document creation but from administrative and preparatory work. This includes:

• Intake and document organization
• EDIscovery triage
• Routine review and identifying workflow bottlenecks

3. The Efficiency Fallacy: AI and Legal Costs

There is a common assumption that AI automatically reduces legal costs, but installing the technology does not guarantee instant savings. In practice, AI can actually accelerate existing inefficiencies if workflows are not redesigned.

True cost savings and ROI are dependent on training, governance, and integration rather than the tool itself. Firms must move beyond simply "adding" AI and instead focus on process change to see tangible gains.
 

4. Transformation, Not Replacement

The fear that AI will take over entire professions is perhaps the oldest and most durable myth. The data indicates that AI replaces tasks, not judgment.

AI excels at pattern recognition, research, and summarization, but it cannot perform strategy, negotiation, client counseling, or ethical reasoning. Successful adoption involves using AI to "shorten the runway" to the point where human judgment begins, allowing professionals to focus on high-level advocacy.

Strategic Consideration: When comparing adoption, large and small firms face different strengths and weaknesses. While large firms may have more resources for integration, smaller firms may be more agile in redesigning workflows—a critical step for realizing AI’s true value.
 

5. The "Truth" Delusion

Finally, it is a mistake to believe that AI always knows the right answer. AI can be highly confident while being entirely wrong, fabricating information or reflecting biases from its training data.

The Stanford Reg Lab study (https://reglab.stanford.edu/publications/hallucination-free-assessing-the-reliability-of-leading-ai-legal-research-tools/) , found that Lexis+ AI hallucinated ~17–20% of the time, while Westlaw AI‑Assisted Research hallucinated ~33–34% of the time. That means Westlaw hallucinated roughly twice as often as Lexis.

Why? Because LLMs are trained to always produce an answer, even when the context is empty or wrong. Much like the old days of multiple-choice testing, LLM’s penalize an unanswered question. Even worse, if no answer is found, they make one up.

A further part of this myth is a RAG can fix hallucinations when retrieval fails silently. Retrieval‑Augmented Generation systema are an AI architecture that lets a model look up real, external information at the moment of a query and then generate an answer grounded in that information. But even with a RAG, LLMs can still:

• Misinterpret retrieved text
   
• Produce incorrect conclusions from correct sources
   
• Hallucinate if retrieval fails or is low‑quality

Why?  This is the biggest reason.

If the retriever pulls:

• irrelevant cases
• the wrong jurisdiction
• outdated law
   
• nothing at all

the model does not know the retrieval failed. It just keeps talking.

Further, Shepard’s only works on real citations. If the AI invents a case — e.g., Smith v. Johnson, 2014 — Shepard’s has nothing to check. It can’t say “this case doesn’t exist.” It just returns nothing.

AI models interpret “no result” as permission to continue, not as a red flag. This is the #1 failure mode in the Stanford study.

Crucially, there is no "cure" for these errors; AI will always have the potential to hallucinate. If the model cannot find a citation, it must be forced to say: “No authority found.”, This is how you stop invented tests, invented standards, and invented holdings.
 

BONUS MYTH: DOCUMENT DRIFT

Document drift in AI is the phenomenon where an AI system gradually alters the meaning, scope, or legal effect of a document as it drafts, rewrites, or reviews it. The key idea: the text still looks correct and reads smoothly, but its legal meaning has shifted, sometimes in subtle but high‑risk ways.

This shows up in two distinct but related forms in legal work:

Machine‑introduced legal drift during drafting or editing, and

Model drift / semantic drift during document review systems over time.

The scale of this decay was recently quantified by Microsoft Research. In a study involving 19 frontier models across 310 professional documents, researchers simulated a standard workflow: iterative editing across 20 successive interactions. By the final interaction, models had corrupted 25% of the document content! Human verification remains essential to ensure accuracy and ethical standards are met.

 

About the Author...

Tom O'Connor
Gulf Coast Technology Center
E-Discovery Committee Chair