Risks of AI in Healthcare: What 70%–77% Privacy Concern Reveals About System Design

Most discussions about the risks of AI in healthcare begin with the model: its accuracy, bias, or hallucination rate. In the HealthTech projects I have worked on, the decisions that mattered most were made earlier, when teams settled what patient data the system could access, what authority its output would carry, and whether anyone could reconstruct an AI-assisted action afterward.

That is why I see healthcare AI risk as both a model problem and a system-design problem.

A TechnBrains secondary evidence review of 14 selected US survey waves published between 2024 and 2026 found no single defensible percentage for concern about healthcare AI. Privacy was the clearest exception. Two independent national surveys found that 70% and 77% of US adults expressed concern about health-data privacy in connection with AI.

What Are the Risks of AI in Healthcare?

The risks of AI in healthcare fall into three groups by origin. Model risks arise from system behavior: hallucination, bias, inaccuracy, drift, and missing clinical context. Data risks arise from excessive access, exposure, retention, secondary use, and weak consent. Workflow and decision risks arise from automation bias, unreviewed actions, unclear accountability, skill erosion, and AI-supported coverage denials.

Grouping by origin points to an owner. A hallucination is a clinical-validation problem, an over-broad payload is a data-architecture problem, and an unreviewed AI draft reaching a record is a workflow problem. Calling all three “AI risks” hides which one you have.

Where Does AI Create Value In Healthcare Without Adding Unnecessary Risk?

The benefits of AI in healthcare are real, particularly in bounded tasks with reviewable outputs. Ambient documentation, summarization, and drafting can reduce administrative work that pulls clinicians away from patient care.

I have seen AI create the most reliable value in bounded tasks where the output is reviewable and a single mistake does not immediately become a clinical decision. Risk rises with autonomy and consequence, not simply with technical sophistication.

How Concerned Are Americans About AI and Health Data Privacy?

Health-data privacy drew the most consistent concern of any category reviewed, with two independent national surveys of US adults landing between 70% and 77%. No survey supported a single percentage for concern about healthcare AI overall.

TechnBrains Evidence Review

A secondary review of 14 selected US survey waves published from 2024 through 2026, covering public, patient, physician, nurse, and healthcare-professional populations. Populations, sampling methods, and question wording were checked before any comparison.

TechnBrains did not conduct the original surveys. No mean, median, or weighted average was calculated.

What the TechnBrains Evidence Review Tells about Risks of AI in Healthcare

The level of concern changes with the question and the AI application being described. Annenberg/SSRS reported in 2025 that 49% of US adults were uncomfortable with providers using AI rather than professional experience alone in care decisions. General questions produced mixed sentiment, while questions specifically involving medical-data privacy generated substantially higher concern.

Why Privacy Was the Clearest Cross-Survey Finding

Ohio State Wexner Medical Center / SSRS reported 70% of about 1,006 US adults surveyed in May 2024 had concerns about data privacy, though that release does not publish the complete question wording, so the construct is documented but the exact item is not. KFF found 77% of 1,343 US adults surveyed February 24 to March 2, 2026 were very or somewhat concerned about the privacy of personal medical information provided to AI tools.

A Wolters Kluwer/Ipsos online panel of 254 US patients fielded in March 2026 reported 74%. It supports the pattern rather than serving as a third endpoint: the panel was smaller, recruited online, and unpublished in full.

What the 70%–77% Finding Does and Does Not Mean

It measures concern about health-data privacy among US adults. It does not show that 70% to 77% oppose healthcare AI, does not measure every healthcare AI risk, is not a TechnBrains survey, and does not combine public and clinician samples.

What Is The Difference Between Model Risk And System-Design Risk?

Model risk is the risk of a wrong, biased, fabricated, or stale output, measured against what the AI produces, often referred to as AI hallucinations. System-design risk comes from what surrounds the model: what data it can reach, what its output can change, what a vendor retains, and whether an action can be reconstructed.

Why the Distinction Changes What a Team Fixes

Model risk already attracts scrutiny through clinical validation, device pathways, and clinician sign-off. System-design risk attracts less, yet a model can pass an accuracy test while the system around it exposes too much data, grants too much authority, or leaves no record of how a decision was made. Not every failure originates here, but this is the part a delivery team can usually remove directly.

I do not open an implementation risk review by asking only whether the model is accurate. I ask what it can reach, what its output can change, and whether the team can reconstruct an AI-assisted action afterward. Those answers are architectural, decided during design, and expensive to retrofit.

Where Each Healthcare AI Risk Enters The System

Each risk enters through a specific decision about data, architecture, integration, workflow, or governance.

Most are set during design, so a risk review scheduled after vendor selection arrives too late to change them.

What Recent AI Data Exposures Teach Healthcare Teams

Three recent incidents exposed sensitive information through different mechanisms. None involved a model producing a wrong answer, and calling them AI leaks would obscure the design decision each implicates.

Xsolis: Vendor Access Became Part of the Healthcare Risk Surface

In June 2026, Xsolis, a healthcare technology vendor whose utilization-management services include AI-supported technology, disclosed that a January 2026 phishing attack led to unauthorized access and acquisition of files containing protected health information.

HHS Office for Civil Rights reporting listed 1,396,519 affected individuals, and no AI model was implicated. An AI-enabled workflow can extend patient data into a vendor environment whose identity and access controls become part of the healthcare organization’s risk surface.

DeepSeek: AI Logs Became a Sensitive Dataset

Wiz Research disclosed in January 2025 a publicly accessible ClickHouse database associated with DeepSeek that required no authentication and exposed over a million lines of log streams containing chat history, secret keys, and backend details.

It was secured after Wiz reported it, and the count refers to log lines, not individuals. Prompts, outputs, and logs are sensitive datasets needing their own retention limits.

Public Conversation Links: Sharing Controls Created Unexpected Exposure

An opt-in setting allowed some deliberately shared ChatGPT conversations to be indexed by search engines, and OpenAI removed it at the end of July 2025. Private unshared conversations were not shown to have been exposed. A sharing control can be technically accurate and still produce consequences users do not anticipate.

What Are The Seven Boundaries For Healthcare AI?

These are the seven boundaries I use to structure a healthcare AI risk review.

Data, Access, and Purpose Boundaries

1. Data boundary. What patient information does the system genuinely need, and at what granularity?
2. Access boundary. Which users, services, and vendors can reach it, and for how long?
3. Purpose boundary. For which approved task may it be used, and what new use requires re-approval?

Decision and Oversight Boundaries

4. Decision boundary. May the system summarize, recommend, modify, approve, or deny? Where is the ceiling on its authority?
5. Oversight boundary. Who reviews outputs, when is review mandatory rather than optional, and who monitors behavior over time?

Retention, Accountability, and Traceability Boundaries

6. Retention and reuse boundary. Are prompts, inputs, outputs, logs, or embeddings retained, and can any be used for training?
7. Accountability and traceability boundary. Who owns a failure, and can the decision be reconstructed?

Patient disclosure runs across all seven rather than sitting inside one, because what a patient is told touches every boundary above.

The Seven Boundaries are a TechnBrains analytical framework, not a regulatory standard, and applying them does not by itself establish compliance. The surveys above did not test the framework; they describe the concern that makes the boundaries worth setting deliberately.

How I Apply These Healthcare AI Controls in Practice

The following composite illustrates patterns I have encountered across multiple TechnBrains HealthTech implementations. It is not a single client engagement, and no quantified outcome is claimed.

A typical example is a healthcare platform where an AI feature summarizes patient interactions, drafts follow-up notes, and surfaces next steps for a care team to review.

The Risk Is Wider Than Model Accuracy

The design risk appears before inference. The proposed integration often makes a broad patient-record payload available to the AI service, defines access at the record level rather than the task level, treats human review as an expectation rather than a requirement, and logs that a request occurred without capturing which fields, versions, and reviewer were involved.

How the Data, Decision, and Oversight Boundaries Get Narrowed

I start by limiting the data each AI task can access. The system selects only the patient information needed for that specific task, removes anything unnecessary, and sends a controlled request to the AI model. This keeps the model from accessing the full patient record and follows the same data-minimization principles used in custom healthcare app builds.

I separate assistance from authority next: the system may summarize, draft, and suggest, but it may not update the record, finalize a clinical instruction, trigger outreach, or approve anything.

Human review becomes a technical requirement rather than a policy statement, so the reviewer must confirm the draft and approve any follow-up action separately before it affects the record. Retention, vendor training use, and deletion expectations become explicit review items, and logging captures the approved field set, versions, reviewer, and final action without copying unnecessary PHI.

Implementation Insight

A policy saying a human should review an output is not a control if the workflow lets it through anyway. Enforced review changes the failure mode: the system cannot advance until a named person has acted, and the reviewer and their edits are recorded for any later investigation.

The Lesson I Take From These Implementations

Improving the model is not the only way to reduce risk. Narrowing the data, authority, and retention around it is frequently the stronger intervention, and it does not require a better model. These patterns touch six of the seven boundaries.

What Risks Concern Physicians And Healthcare Professionals?

Physician concerns centered on skill loss and AI use in coverage decisions rather than privacy. Every clinician finding below came from voluntary, list-sourced, or non-probability panels, so none is representative of all US clinicians or comparable with the public surveys above.

The AMA’s 2026 physician survey found 88% of responding physicians expressed at least some concern about AI-related skill loss, and 70% were very or somewhat concerned about skill loss among current medical students and residents.

A separate AMA prior-authorization survey found 61% of responding US physicians who regularly completed prior authorizations were concerned that health plans’ use of AI was increasing or would increase denials.

Skill Loss May Develop Before Teams Know How to Measure It

Skill erosion receives less attention partly because it develops slowly. What concerns me is that the effect may not appear in the risk dashboards teams already use, which track incidents, uptime, and accuracy rather than gradual changes in clinical reasoning.

AI-Supported Coverage Decisions Create a Different Accountability Risk

Assisting a coverage decision and making one are not the same act. The control separating them is a mandatory human review with a recorded outcome, placing the decision with a named person rather than a model configuration.

What Should Health Systems Check Before Approving An AI Pilot?

Before I support an AI pilot moving forward, I want clear answers to eight questions.

1. What exact task is the AI approved to perform, and what is out of scope?
2. What data does it need, and what data does it not need?
3. Can it influence or execute a consequential decision, or only produce a draft?
4. When is human review mandatory, and is that enforced by the workflow?
5. Can the vendor retain or train on inputs and outputs, and has that been tested?
6. Can a decision be reconstructed from logs without those logs becoming a second sensitive store?
7. What happens when the model is uncertain or wrong, and does the system fail closed?
8. Who owns investigation, correction, and escalation, by name?

A vague answer usually signals an unassigned decision rather than an unimportant one, most often on question 4.

Why Provider-Side AI Governance May Matter More

Federal transparency requirements for certified health IT are in flux. The HTI-1 final rule, in force since January 1, 2025, requires developers certifying predictive Decision Support Interventions to disclose source attributes, often called model cards, and to maintain intervention risk management practices.

The HTI-5 proposed rule, published December 29, 2025 with a comment period that closed February 27, 2026, proposes reducing that criterion to remove the model card requirements and removing the privacy and security certification criteria, on the grounds that those capabilities are independently required under HIPAA.

As of August 3, 2026 it is proposed, not final, and HIPAA obligations continue to apply. If certification stops requiring vendors to document how a predictive model was built, procurement and system design become the remaining places that question gets asked. This is not legal advice.

Conclusion

Privacy concern at the level two national surveys recorded is a design signal, not evidence of rejection. The risks of AI in healthcare that a team can remove are mostly the ones it designed in: a payload wider than the task needed, authority never capped, retention nobody tested, a log that cannot reconstruct a decision.

My position is not that healthcare organizations should avoid AI. It is that AI should earn access and authority one bounded use case at a time. The model matters, but the organization deploying it remains responsible for the data, workflow, and decisions surrounding it.