You go to a doctor with a concern. But before the doctor reaches a conclusion, a system has already analysed your data.
It has compared patterns, calculated probabilities, and produced a recommendation.
This is no longer experimental. Artificial intelligence is now embedded in diagnostics, screening, and clinical decision support. It promises speed, consistency, and, in some cases, greater accuracy.
But its integration into healthcare raises a question that law has not yet fully answered:
When AI contributes to a medical decision, who is responsible for the outcome?
From Clinical Judgment to Algorithmic Output
Artificial intelligence in healthcare operates through algorithms—structured sets of instructions that process data and generate predictions. In practice, this may involve identifying abnormalities in ECG signals, predicting disease risk, or supporting diagnostic decisions.
These systems are often described as decision-support tools. Yet their role is increasingly substantive. They do not merely assist; they shape the informational basis on which decisions are made.
Crucially, AI systems differ from human clinicians in three respects. They learn from data rather than experience, produce probabilistic outputs rather than definitive conclusions, and improve through iterative optimisation rather than reflective judgment.
This marks a shift from judgment-based medicine to data-driven decision-making.
The legal implications of that shift are significant.
Data Protection and the Limits of Consent
AI systems depend on large volumes of sensitive health data, including patient histories, biosignals, and behavioural information. Within the European context, this engages the GDPR, where health data qualifies as a “special category” under Article 9 and is subject to heightened protection.
In principle, the framework is clear. Processing requires a lawful basis under Article 6, a valid condition under Article 9, and appropriate safeguards and transparency.
However, AI complicates each of these requirements.
First, data is not used in a single, clearly defined act of processing. It is reused, repurposed, and integrated into evolving models. Second, the outputs of AI systems—predictions, risk scores, classifications—may themselves constitute new forms of personal data. Third, the logic of these systems is often opaque, which may raise concerns under Article 22 GDPR where decisions are based solely on automated processing with legal or similarly significant effects.
The result is a structural tension: data protection law assumes relatively static processing, while AI systems are inherently dynamic.
Bias, Uncertainty, and the Problem of Standards
AI systems are frequently framed as objective. In reality, they are shaped by the data on which they are trained.
If datasets are incomplete or unrepresentative, the resulting systems may produce systematically biased outcomes. Certain populations may be underdiagnosed, misclassified, or excluded altogether.
At the same time, AI operates under conditions of uncertainty. Many systems produce probabilistic outputs, estimates of likelihood rather than determinate conclusions.
This presents a direct challenge to medical negligence law.
Traditionally, liability is assessed against the standard of the reasonable clinician, often articulated through the Bolam test. However, following Bolitho, the professional opinion relied upon must also withstand logical scrutiny.
But AI does not “exercise judgment” in this sense. It optimises statistical performance.
This raises a difficult question: Can a probabilistic system satisfy a standard designed for human professional judgment?
The Black Box Problem and Explainability
A further difficulty lies in explainability.
Many machine learning systems—particularly neural networks—are not easily interpretable. They may produce highly accurate outputs, but the internal reasoning process is not transparent.
This creates friction across multiple areas of law.
Informed consent requires that patients understand the basis of decisions affecting them. Accountability requires that responsibility can be traced. Judicial review requires that decisions can be evaluated.
Where the reasoning process cannot be meaningfully explained, each of these becomes harder to sustain.
Transparency, in this context, is not merely desirable. It is foundational to legal accountability.
Ethical Frameworks and Their Limits
International efforts have begun to address these concerns. The UNESCO Recommendation on the Ethics of AI (2021), for example, adopts a human rights-based approach, emphasising dignity, autonomy, non-discrimination, and accountability.
Within healthcare, this translates into a clear principle: AI should support decision-making, not replace it.
More recently, the EU AI Act adds a binding regulatory layer. Many AI systems used in healthcare—particularly those integrated into medical devices—are classified as high-risk, triggering obligations relating to risk management, data governance, transparency, human oversight, and conformity assessment.
Yet even with these developments, the central issue—liability—remains only partially addressed.
The Liability Gap
If an AI-assisted decision leads to harm, responsibility may be distributed across multiple actors: the clinician who relied on the system, the institution that deployed it, and the developer who designed it.
Existing legal frameworks struggle with this distribution.
Medical negligence law assumes a human decision-maker, a definable standard of care, and a traceable reasoning process.
AI disrupts each of these assumptions. Decision-making becomes distributed, standards become statistical, and reasoning becomes opaque.
This creates what can be described as a liability gap.
The problem is not simply that harm may occur. It is that the structure of responsibility becomes difficult to map onto existing legal categories.
Human Oversight, and Its Limits
For now, the dominant legal approach is conservative. AI is treated as a tool, and clinicians remain responsible for interpreting outputs and making final decisions.
This preserves a clear point of accountability.
But it also raises a practical concern. As AI systems become more accurate and more integrated into clinical workflows, the extent to which clinicians can meaningfully “override” or critically assess these systems may diminish.
If reliance becomes de facto unavoidable, the distinction between tool and decision-maker begins to blur.
Conclusion
Artificial intelligence is reshaping healthcare in ways that are both promising and legally disruptive.
It improves diagnostic capacity, expands access, and enhances efficiency. But it also challenges foundational assumptions in law—about responsibility, decision-making, and accountability.
Data protection law struggles to capture dynamic systems.
Negligence law struggles to evaluate probabilistic outputs.
And liability law, in particular, struggles to allocate responsibility where decisions are no longer fully human.
The key question is no longer whether a decision was wrong.
It is: who—or what—made it, and whether the law can meaningfully assign responsibility in such a system.
Because even when machines assist in diagnosis,
the consequences remain human.
Leave a Reply