AI security review for healthcare

Clinical AI systems make or assist in decisions that affect patient safety. An AI that assists in diagnosis, triages clinical data, or interacts with clinical workflows is not a standard enterprise software deployment — it carries direct patient risk that standard AppSec processes are not designed to assess.

The EU AI Act explicitly classifies AI systems used in healthcare as high-risk under Annex III — requiring documented risk management, technical documentation, and ongoing monitoring. The EU Medical Device Regulation (MDR) requires a clinical evaluation and post-market surveillance for AI that qualifies as a medical device.

Healthcare AI faces threat surfaces that general security reviews don't cover:

• Prompt injection through clinical notes. An AI that processes clinical notes, discharge summaries, or patient-submitted data can be hijacked through injected instructions in those documents — a direct patient safety risk if the AI affects care decisions.
• PHI leakage through RAG over clinical data. A clinical knowledge assistant retrieves from a corpus that includes patient records. Insufficient retrieval authorisation can surface PHI to users who should not see it — an immediate HIPAA and GDPR violation.
• Bias in clinical AI outputs. LLM-assisted triage or diagnostic support can produce systematically biased outputs for specific demographic groups. The threat model must include adversarial evaluation for demographic fairness alongside technical security.
• Agentic AI with EHR or prescription tool access. An agent that can write to EHR records, trigger alerts, or interact with prescription systems requires the most stringent blast-radius controls and human-in-the-loop approval boundaries.

For healthcare AI classified as high-risk under Annex III, the EU AI Act requires:

• Article 9: documented risk management system with evidence
• Article 11: technical documentation file (system description, design specs, test results)
• Article 17: quality management system
• Article 72: post-market monitoring plan

Drel produces the risk management evidence (Article 9) and the structured technical documentation that supports the Article 11 file. The re-assessment trigger system supports Article 72 by flagging when the system changes in ways that require the risk record to be updated.

Drel is a design-time AI security review tool. For medical devices, MDR conformity assessment and clinical evaluation require separate processes — often involving notified bodies. Drel produces the security evidence that feeds into these processes; it does not replace them. For HIPAA, Drel produces the risk analysis documentation; it does not perform the full HIPAA Security Rule risk assessment or replace your compliance programme.