Somewhere in your organization, right now, someone is pasting patient information into an AI tool. Maybe it’s a clinician summarizing a note, a support rep drafting a reply, or an engineer debugging with production data. They’re not malicious — the tools are genuinely useful, and the compliance guidance most companies have given their staff amounts to a shrug.
HIPAA wasn’t written with large language models in mind, but its logic maps onto them cleanly once you strip away the novelty. This post lays out that mapping in plain language: when an AI vendor needs a BAA, what to ask before signing one, what your internal policy should say, and where the genuinely unsettled questions are.
The core principle: AI vendors are just vendors
Under HIPAA, any entity that creates, receives, maintains, or transmits PHI on your behalf is a business associate, and PHI cannot flow to them without a signed business associate agreement (45 CFR §164.502(e), §164.504(e)). Nothing about a model changes this. An AI transcription service is a transcription service. A chatbot platform handling patient messages is a communications vendor. The BAA analysis is identical to the one you’d run for a cloud host or a billing service.
What AI does change is the risk profile behind the paperwork: training on your data, human review of prompts, long retention of conversation logs, and the sheer ease with which staff can move PHI into a consumer tool with one paste. So: same legal framework, sharper diligence.
When you need a BAA — a practical sorting
BAA required — PHI flows to the vendor:
- AI scribes and ambient documentation tools that record patient encounters and draft clinical notes. Audio of a patient visit is PHI from the first second.
- LLM APIs processing identifiable data — summarizing records, drafting patient communications, extracting codes from clinical text.
- Patient-facing chatbots handling scheduling, symptom intake, billing questions, or anything else tied to an identifiable person’s care or payment.
- AI features inside tools you already use — your EHR’s new assistant, your helpdesk’s reply-drafter, your meeting recorder joining a clinical call. These ride on existing vendor relationships but may be excluded from existing BAAs (more below).
No BAA needed — because no PHI should flow:
- Consumer AI tools (free or standard-tier chatbots). No BAA is offered, so PHI is prohibited, full stop. They’re fine for writing job postings and debugging code that contains no real data.
- Properly de-identified data. Data de-identified under the Safe Harbor method (all 18 identifier categories removed — 45 CFR §164.514(b)) or Expert Determination isn’t PHI, and HIPAA doesn’t restrict it. The trap: free-text clinical notes resist reliable de-identification, and “I deleted the name” is nowhere close to the standard.
- Self-hosted open-weight models running entirely inside your own environment. No disclosure to a vendor means no BAA question — though every Security Rule obligation (access control, encryption, audit logging) applies to that infrastructure just as it does to your database.
The good news on availability: major AI providers now offer BAAs on specific enterprise and API tiers, and healthcare-specific AI vendors generally sign them as table stakes. The existence question has largely been solved; the scope question hasn’t.
Read the AI vendor’s BAA for scope, not just signature
The most common AI compliance failure we see isn’t a missing BAA — it’s a BAA that doesn’t cover the way the tool is actually used. Check four things:
- Covered products and tiers. A provider’s BAA may cover its API and enterprise plan but not its consumer app — same company, same model, different legal reality. Staff using personal accounts of a “covered” vendor are outside the agreement.
- Configuration requirements. BAAs frequently require specific settings: training opt-outs, zero-data-retention modes, disabled human review. The agreement is conditional on the configuration; verify it’s actually set, and record who verified it.
- Feature exclusions. Beta features, plugins, web browsing, and connected third-party tools are often carved out. A covered chatbot calling an uncovered plugin is a data flow your BAA doesn’t reach.
- AI features appearing inside existing vendors. When your ticketing system or EHR ships an AI assistant, check whether your existing BAA covers it or whether it runs on a subprocessor under separate terms. Vendors are generally answering this in their trust documentation now — but only if you ask.
Training, retention, and human review: the three questions that matter most
Beyond standard vendor diligence, three AI-specific issues deserve written answers before PHI flows:
Is our data used to train models? Training on your PHI means it’s absorbed into an artifact served to other customers — a use far outside “on your behalf.” Credible healthcare-facing vendors contractually commit to no training on customer data. Get that in the agreement, not a marketing page.
How long are prompts and outputs retained, and where? Some vendors retain conversation logs for abuse monitoring for a fixed window; some offer zero-retention configurations. Retained logs full of PHI are part of your breach surface — they belong in your risk analysis and your data map like any other PHI store.
Do humans review inputs? Several providers sample conversations for safety and quality review. Human review of PHI by vendor staff is a disclosure that your BAA and the vendor’s internal controls need to account for — or you need it switched off.
Round out diligence with the ordinary questions you’d ask any PHI vendor: encryption in transit and at rest, access controls, audit logging, subcontractor list and flow-down BAAs, breach notification timelines, independent security audits, and data deletion at termination. Our HIPAA templates include a vendor vetting checklist with an AI addendum covering all of the above.
What your internal AI policy should actually say
A policy that says “don’t use AI” will be ignored, silently, at scale — and shadow AI usage is far riskrier than governed usage. A workable policy does four things:
- Names approved tools and approved uses. “The following tools, under our enterprise accounts with BAAs, may be used with PHI for these purposes. Consumer accounts and unlisted tools may not touch PHI or any patient-related information.”
- Gives a bright-line data rule. Staff won’t parse the 18 identifiers in the moment. Give them: “If it’s about a real patient — even without the name — it doesn’t go into an unapproved tool.”
- Creates a fast approval path. A lightweight request process (tool, use case, data involved) turned around in days keeps people inside the tent. Slow governance manufactures shadow IT.
- Handles output quality. AI text can be confidently wrong. For clinical documentation, require clinician review before notes are finalized — AI scribe vendors themselves frame outputs as drafts. Fabricated content in a medical record is a patient-safety and data-integrity problem, and data integrity is explicitly within the Security Rule’s scope.
Then train on it. Security awareness training is already required under §164.308(a)(5); adding a concrete AI module — with examples of what pasting PHI into a consumer chatbot looks like and why it’s a reportable event — is the highest-yield training update you can make this year. Fold approved AI tools into your access reviews and offboarding like any other PHI system.
Update your risk analysis — this is where regulators will look
Your risk analysis must be accurate and thorough for the environment you actually run (45 CFR §164.308(a)(1)(ii)(A)). If AI tools now process PHI and your risk analysis predates them, it’s no longer thorough. Add:
- Each approved AI tool as an asset, with the PHI it handles and where prompts/outputs are retained
- Shadow AI usage as a threat, with your policy, training, and any technical controls (network filtering, DLP, browser controls) as mitigations
- Vendor-side risks: training misconfiguration, subprocessor chains, log breaches at the AI provider
- Output integrity risk for clinical documentation workflows
If you haven’t got a current risk analysis at all, start with our step-by-step risk analysis guide and the free HIPAA readiness assessment — AI is a chapter of that book, not a separate book.
Honest notes on the unsettled parts
Practitioner honesty requires flagging what’s genuinely unresolved:
- Regulatory guidance is thin. OCR has not issued comprehensive AI-specific HIPAA guidance; the analysis in this post applies existing rules to new tools, which is what everyone — including regulators — is doing right now. Expect refinement.
- Model memorization is an open research question. Whether and when models can regurgitate training data is actively studied. The contractual “no training on customer data” commitment is precisely how you avoid needing an answer.
- De-identification is getting harder. Re-identification techniques improve alongside AI. Safe Harbor remains the legal standard, but treat aggressive “it’s anonymized” claims — from vendors or your own team — with professional skepticism.
- “HIPAA compliant AI” is a marketing phrase, not a status. No tool is compliant in itself; compliance describes how an organization deploys it. And beware any vendor advertising HIPAA “certification” — HHS runs no certification program and recognizes none. What a serious vendor offers is a BAA, configuration documentation, and independent audit reports. What a serious buyer maintains is a risk analysis, a policy, and diligence records. If you want an outside check on yours, that’s what an independent gap assessment is for.
A 30-day plan to get ahead of this
- Week 1 — Discover. Survey staff (amnesty explicitly granted) and check expense reports and SSO logs for AI tools in use. You will find more than you expect.
- Week 2 — Sort. For each tool: does PHI flow? Is there a BAA, on the right tier, in the right configuration? Kill or contain the uncovered flows first.
- Week 3 — Paper and configure. Sign BAAs where offered and needed, set training opt-outs and retention modes, document verification. Draft the policy with named approved tools.
- Week 4 — Train and integrate. Roll out the policy with training, add AI assets to the risk register and data map, and put a quarterly review on the calendar — this landscape shifts fast enough that annual isn’t enough.
None of this requires banning useful tools. The organizations handling AI well aren’t the ones prohibiting it; they’re the ones that made the approved path easier than the shadow path.
Want to see how your AI usage fits into your broader compliance posture? Take the free HIPAA readiness assessment for a scored snapshot, work through the HIPAA checklist to see where AI touches your required safeguards, or book a 20-minute call — bring the list of tools your team is actually using, and we’ll sort it with you.