UK review recommends 'L plates' for new healthcare AIs

The National Commission into the Regulation of AI in Healthcare has delivered its keenly anticipated report, which gives recommendations on creating a "safe, fast, and trusted" system for deploying AI tools in the NHS.

The overall conclusion of the report, commissioned by the Medicines and Healthcare Products Regulatory Agency (MHRA), is that the current regulatory framework for healthcare AIs is inadequate for AI, as it was set up for conventional medical devices.

Drawing on feedback from more than 12,000 patients, clinicians, and AI developers, it offers 44 recommendations to update the framework to allow AI to become an integral part of healthcare delivery in the UK.

"We have heard clearly about the opportunities of AI to speed up and improve care, and to release more of clinicians' time for the essentially human aspects of their work with patients," said MHRA chief executive Lawrence Tallon, who noted that the independent commission has set out "in more detail than ever before" how AI should be regulated in healthcare settings.

"We will only realise those opportunities if we have a modern, dynamic regulatory framework that accelerates safe adoption, protects patients, and commands public and professional confidence."

Among the recommendations are staged authorisations for healthcare AIs – analogous to L plates for learner drivers – which would allow the technologies to be tested in controlled environments before being rolled out more widely by the NHS. New models should also be rolled out with tight controls and close supervision to prove their real-world safety before they are fully authorised.

There should also be a shift from one-off approvals to continuous, lifecycle monitoring of AI, with rapid pathways to allow them to be withdrawn if their efficacy diminishes over time or they become unsafe, according to the report, which was authored by regulatory science expert Prof Alastair Denniston of the University of Birmingham and Prof Henrietta Hughes, Patient Safety Commissioner for England.

Liability laws and systems need to be upgraded so developers can be penalised if their AIs cause problems, with tougher penalties levied, and the reporting system for adverse events needs to be updated to take AI technologies into account.

Patients must have the right to know when and how AI is actively being used in their care, and information on models must be available to the public via a searchable database.

Overall, AIs should augment and not replace human oversight, and final clinical judgements must always be made by healthcare professionals, concludes the report.

"Any new innovation – including AI – needs to earn its place and earn our trust," said Denniston. In compiling the report, he added, "one message came through clearly: people are open to AI improving their care, but only if it is safe, overseen by humans, and if they know when it is being used. These recommendations are designed to make that trust possible."

The Health Services Safety Investigations Body (HSSIB), which is an arm's-length body of the Department of Health and Social Care tasked with investigating patient safety concerns across the NHS in England, welcomed the new document, saying it reflects the need to balance innovation with patient safety, public confidence, and effective oversight.

HSSIB recently launched an investigation into ambient voice technology (AVT), which is rapidly being rolled out by the NHS for tasks like summarising clinical conversations and drafting letters, after identifying potential safety issues and concerns that reporting and monitoring systems are not geared up to detect errors, hallucinations, or data privacy issues with the AIs.

"As this work progresses, we will carefully consider the Commission's recommendations and the wider conversations taking place across the healthcare system about the safe use of AI," commented HHSIB's interim chief executive, Dr Rosie Benneyworth.

"By improving our understanding of both the opportunities and the risks associated with these technologies, we can help support their safe adoption in ways that benefit patients, staff, and the healthcare system."