AI and Data Licensing · Memo

AI Vendor Contracts: Indemnity for Model Hallucinations

Enterprise AI vendor contracts are starting to have real liability allocation around model hallucinations. I will walk through the legal theories the indemnity has to address, the standard vendor positions, and the customer-side language I push for.

A hallucination, in the practitioner-friendly definition, is an AI model output that asserts something that is not true. The legal exposure depends on what the user does with the false assertion. If the user is a consumer relying on a personal chatbot, the exposure is bounded by the user's reliance. If the user is a business deploying the model in a workflow that produces customer-facing outputs, the exposure stacks: defamation, false advertising, professional malpractice, products liability, contract misrepresentation, regulatory violations, and tort claims by third parties who relied on the output.

Three matters that have shaped my drafting in the last year. The Mata v. Avianca matter, where counsel filed a brief with hallucinated case citations and was sanctioned, did not generate a vendor lawsuit but did demonstrate that the user, not the vendor, bears the immediate professional exposure. The Air Canada chatbot matter in 2024 (Tribunal decision) held the deploying business responsible for false statements made by its customer-service chatbot. The Walters v. OpenAI defamation matter, dismissed by the Georgia trial court, was the first reported defamation action against an LLM provider and the decisional analysis is worth reading even where it does not bind. These matters are not a coherent doctrine, but they have shifted vendor negotiating posture.

The legal theories the indemnity has to cover

• Defamation. If a model output asserts a false fact of and concerning a real person, the deploying business may face defamation exposure. Section 230 of the Communications Decency Act, 47 U.S.C. section 230, may or may not apply; the case law is unsettled on whether an AI output is third-party content under section 230(c)(1). The vendor's first draft typically does not address defamation. The customer should push for it.
• False advertising / UCL. If a model output makes false statements that the deploying business uses in marketing material, exposure under FTC Act section 5, California Bus. and Prof. Code section 17500, and the UCL section 17200 may attach. The indemnity should reach this.
• Professional malpractice. For legal, medical, financial, or other professional deployments, hallucinations can produce malpractice exposure to clients. The professional, not the vendor, owes the duty, but the indemnity can shift the loss back to the vendor where the hallucination was the proximate cause.
• Contract misrepresentation. If the deploying business uses a model to communicate with customers and the model makes a false representation that the customer relies on, the deploying business may face a misrepresentation or contract-formation claim. Air Canada is the example.
• Regulatory exposure. Sector-specific regimes (HIPAA for health, GLBA for financial, COPPA for children's data, ECOA for credit, FCRA for consumer reports) impose obligations on regulated entities. A model that produces output that violates a sector-specific obligation creates exposure for the regulated entity.

Standard vendor positions in 2026

Most enterprise AI vendors now provide some form of indemnity, but the structure varies. The patterns I see most often:

1. IP-only indemnity. The vendor indemnifies for IP infringement claims arising from the model's outputs (typically subject to carve-outs for customer prompts, customer training data, and combination claims). Hallucinations producing non-IP exposure are not covered.
2. IP plus limited hallucination indemnity. The vendor indemnifies for IP claims and adds a narrow indemnity for hallucination-related claims, typically subject to a use-restriction (the customer must implement specific guardrails) and a cap (often the standard liability cap, sometimes a super-cap).
3. Full output indemnity with use restrictions. The vendor indemnifies for all third-party claims arising from model outputs, conditional on the customer's compliance with specified use restrictions (no use for legal advice, no use without human review, no use in regulated workflows without additional approval). The cap is typically the standard cap or a super-cap.

The use-restriction framing is the operational mechanism the vendors have settled on. The vendor is willing to indemnify if the customer has implemented the recommended guardrails. The customer is willing to implement the guardrails if the indemnity is real. The negotiation is on what counts as adequate guardrails.

The customer-side language

For customer-side counsel, the language I push for:

• Indemnity that covers all third-party claims arising from outputs. Not just IP. Defamation, false advertising, products liability, professional malpractice, regulatory violations.
• Use-restriction language drafted as best-efforts compliance, not strict compliance. The customer cannot guarantee that every deployment will hit every guardrail. The indemnity should not be defeated by an isolated guardrail miss.
• Carve-outs from the indemnity that are narrow and specific. Vendors will request carve-outs for customer-modified prompts, customer-tuned models, customer use outside the documented scope. Each of these is reasonable in principle and dangerous when broadly drafted.
• A separate hallucination super-cap. The general liability cap is too low for substantive hallucination exposure. A super-cap of three to five times the annual fees, with willful-misconduct uncap, is what I aim for.
• A defense-control mechanic. The customer typically wants the vendor to defend, but with reasonable input from the customer on settlement and on positions taken in the litigation. The control-of-defense clause should be drafted with reasonable bilateral input.

The vendor-side defense

For vendor-side counsel, the reasonable positions:

• Narrow the indemnity to vendor-caused output errors. If the customer modified the model, supplied risky training data, or deployed the model in an environment the vendor did not bless, the loss should not be the vendor's.
• Insist on use restrictions with operational meaning. The restrictions should be specific enough to be auditable. 'Reasonable guardrails' is not enough. 'Human review of outputs before publication in marketing channels' is.
• Cap exposure. An uncapped output indemnity is not commercially viable for most vendors. A super-cap is the negotiating space.
• Address customer prompts. Customer prompts that elicit hallucinations or harmful outputs (jailbreaks, adversarial prompts) should be carved out. The carve-out language has to be careful, because some hallucinations occur on benign prompts.

The 47 U.S.C. section 230 footnote

I am going to flag uncertainty on section 230. The text of section 230(c)(1) shields a provider of an interactive computer service from being treated as the publisher or speaker of information provided by another information content provider. The question whether an AI model is acting as an interactive computer service publishing third-party content, or as an information content provider producing its own content, has not been authoritatively resolved. The Walters trial court treated the model output as the AI provider's own content for defamation purposes; other courts may rule differently. Counsel relying on section 230 as a shield for hallucination liability are relying on an unsettled defense. The contractual indemnity is the more reliable allocation.

Outcomes in actual hallucination disputes depend heavily on the facts. The matters I have advised on have not produced reported decisions, and I would not draw lessons from a settled docket that overstate predictability. The drafting is what counsel can do reliably. The litigation is where the law will set the rules.

Notice and remediation provisions worth adding

One operational provision I add to current drafts that often gets overlooked. When a hallucination is identified in production, the deploying business needs to act fast to remediate (correct the output, notify affected parties, document the response). The vendor's cooperation in that remediation matters. The clause: the vendor will, upon notice from the customer of a material hallucination affecting customer operations, provide reasonable cooperation in identifying the cause, modifying the model or prompts to prevent recurrence, and supporting the customer's remediation communications. The cooperation obligation is independent of the indemnity and survives termination.

The other operational piece: a vendor obligation to disclose known hallucination patterns. If the vendor has internal knowledge that the model produces specific categories of hallucinations under specific conditions, the customer should know. A representation that the vendor has disclosed all known material hallucination patterns at contract execution, with an updating obligation, gives the customer the information it needs to design appropriate guardrails. The vendor will negotiate the scope of the obligation; the underlying logic is hard to argue against.

Sergei Tokmakov, Esq., CA Bar #279869. This memo is attorney commentary on legal questions and is not legal advice. Reading it does not create an attorney-client relationship. Past matter outcomes depend on facts and the responding party; nothing here is a prediction of result.