In May 2026, researchers from the Simons Foundation published a paper on arXiv that challenged a core assumption in enterprise AI. The paper, titled "A Few Good Clauses," tested a domain-trained small language model called Olava Extract against five frontier models on structured contract extraction. Olava Extract is a self-hosted legal-domain Mixture of Experts model. Its parameter count is a fraction of what frontier models use.
Olava Extract achieved the strongest aggregate performance in the study, with a macro F1 of 0.812 and a micro F1 of 0.842. It reduced inference cost by 78% to 97% compared with the frontier models tested. It also achieved the highest precision scores, producing fewer hallucinated and unsupported extractions.
A model that costs a fraction of the price runs on your own infrastructure. It produces fewer hallucinations than GPT-4 on contract extraction. That is not a compromise. That is a better tool for the job.
The Size Fallacy in Legal AI
Legal tech has operated under a straightforward assumption. Legal documents are complex, so you need the biggest model available. Law firms and corporate legal departments default to GPT-4, Claude Opus, and other frontier models for contract review, clause extraction, and risk analysis.
The assumption is wrong.
A separate March 2026 study evaluated nine sub-10B parameter models across three legal benchmarks — ContractNLI, CaseHOLD, and ECtHR — using five prompting strategies. The researchers ran 405 experiments with three random seeds per configuration.
A Mixture-of-Experts model activating only 3B parameters matched GPT-4o-mini in mean accuracy. It surpassed GPT-4o-mini on legal holding identification. The study found that architecture and training quality matter more than raw parameter count. The largest model tested, at 9B parameters, performed worst overall.
On legal reasoning tasks, smaller and better-architected models can outperform larger ones. The assumption that bigger is better does not hold in legal AI.
The Cost Gap That Changes Everything
The performance gap is striking. The cost gap is even more so.
Olava Extract reduced inference cost by 78% to 97% compared with frontier models. That is not a marginal improvement. That is a structural difference that changes what is possible.
For a law firm processing thousands of contracts per month, the math is straightforward. Frontier models charge per token. Legal documents are token-heavy. A single 100-page contract can cost $50 to $100 to process with GPT-4. At scale, that becomes a seven-figure annual expense.
A self-hosted legal SLM has no per-token cost. You pay for the infrastructure once. Every additional contract is essentially free. The upfront investment is real. The payback period is measured in months, not years.
The March 2026 study also highlighted a striking detail about evaluation accessibility. All 405 experiments were conducted via cloud inference APIs at a total cost of $62. Thorough LLM evaluation is now accessible without dedicated GPU infrastructure. The barrier to entry for legal AI has dropped dramatically.
The Hallucination Problem Nobody Wants to Talk About
Legal hallucinations are not academic. They create operational risk and downstream review burden. A hallucinated clause in a contract review can lead to missed obligations, incorrect risk assessments, or compliance failures.
The Olava Extract study found that the domain-trained SLM produced fewer hallucinated and unsupported extractions than the frontier models it was tested against. This result is counterintuitive. The smaller, cheaper model was also the more reliable one.
The reason is straightforward. Frontier models are trained on the open internet. They know about everything. But they lack precision on the specific language of legal contracts. A model trained exclusively on legal documents knows the patterns and stays within them. A generalist model is more likely to invent details when the text is ambiguous.
For legal workflows, this distinction matters more than benchmark scores. A model that produces fewer hallucinations requires less human review. Human review is the most expensive part of legal work.
The Production Reality
The research is clear. What does this look like in production?
Harvey, a legal AI platform, published a Contract Intelligence benchmark in November 2025. It encompassed more than 4,000 data points measuring extraction accuracy on varying contract types. The benchmark compared results to human experts. The initial finding: out-of-the-box LLMs struggled on contract extraction, identifying only around 65-70% of valid deal points.
That is the baseline. Out-of-the-box frontier models, without specialized tuning, miss 30-35% of valid contract terms. For a law firm, that is not a productivity tool. That is a liability.
The solution, as Harvey found, is not a larger model. It is a specialized system — one that combines domain-trained models with better retrieval, better prompting, and better evaluation. The goal is not to have the biggest model. The goal is to have the most accurate system for the specific task.
Why Your Lawyer's AI Is Probably Smaller Than Yours
If you are a technology professional, you are probably using GPT-4 or Claude for daily work. You assume that better means bigger.
Your lawyer is probably using something smaller.
Not because law firms are behind the curve. Because they have done the math. A smaller, domain-trained model that runs on-premise, produces fewer hallucinations, and costs a fraction of the price is simply a better tool for contract extraction than a general-purpose frontier model.
The research from 2025 and 2026 is consistent. Specialized small models are closing the performance gap with frontier models. They maintain a massive cost advantage while doing so.
The era of "bigger is always better" in AI is ending. In legal tech, it may already be over.
Tomorrow Morning, Do This
If you are building a legal AI product or evaluating tools for your legal team, stop assuming you need the biggest model.
Ask the vendor: what model are you using? Is it domain-trained? Can you show me the hallucination rate on my specific contract types? What is the total cost of ownership, not just the per-token price?
The answers might surprise you. Your lawyer's AI is probably smaller than yours. And that is a good thing.
📊 Know your real AI cost structure
Our cost engineering briefings show you exactly where AI spending leaks — model selection, inference optimization, and the hidden costs that never appear on an invoice.
Get the frameworks →
The Olava Extract study tested a single SLM against five frontier models on a specific task (structured contract extraction). Results may not generalize to other legal tasks such as litigation strategy, legal research, or drafting. Benchmark performance (F1 scores) does not guarantee real-world performance. Production results depend on data quality, prompt engineering, system integration, and human oversight. The cost savings of 78-97% reflect inference-only costs and may not account for infrastructure setup, model maintenance, staff training, or ongoing operational expenses associated with self-hosted models. The March 2026 study of sub-10B models used 405 experiments across three benchmarks, which is a limited sample. Larger-scale studies may produce different conclusions. Small language models have known limitations in reasoning depth, context window size, and multi-step task execution compared to frontier models. These trade-offs must be evaluated for each use case. The field of legal AI is advancing rapidly. Claims, benchmarks, and product capabilities referenced here may become outdated.
Sources:
1. Lincoln, N. et al., "A Few Good Clauses: Comparing LLMs vs Domain-Trained Small Language Models on Structured Contract Extraction," arXiv:2605.05532 (May 2026)
2. "Can Small Models Reason About Legal Documents? A Comparative Study," arXiv:2603.25944 (March 2026)
3. Harvey Contract Intelligence Benchmark (November 2025)
4. ForageAI 2026 industry benchmarks on purpose-built contract extraction vs general-purpose LLMs (2026)
Disclaimer:
The information provided in this article is for general informational and educational purposes only. It does not constitute legal, financial, or professional advice. All trademarks and references to third-party products, services, or organizations are the property of their respective owners. The performance data and benchmarks discussed are based on specific research studies and may not generalize to all use cases or environments. As of the publication date, the AI landscape continues to evolve rapidly, and readers should verify current information independently.