AI in the Operating Room: Liability Issues for Device Makers
Published in Law360 on April 16, 2024. © Copyright 2024, Portfolio Media, Inc., publisher of Law360. Reprinted here with permission.
Operating rooms brim with medical devices. [1] From simple monitors for vital signs to intraoperative imaging device networks, the volume of information provided during procedures to surgical staffs is immense. [2]
And the work to incorporate artificial intelligence into these devices to capture, organize and utilize surgical data in the operating room in real time has begun. [3] While technology may not yet be able to perform advanced surgery entirely by autonomous robots, surgical procedures incorporating medical devices with AI software will soon be here. [4]
As advancement in technology moves forward, doctors who plan to use these devices in surgeries have already begun seeking legal protection from any liability for doing so — as evidenced by statements made at the American Medical Association's meeting at the end of March on doctors' lobbying priorities. [5] AMA President Jesse Ehrenfeld expressed concerns, mentioning that the organization is "seeing lawsuits already." [6]
As healthcare providers actively lobby for the shifting of liability from themselves to medical products with AI software, there are several potential liability issues that may arise for the manufacturers of these products. This article examines potential product liability issues that such medical devices may face, and design considerations manufacturers should consider when incorporating AI software into their products.
Current Medical Devices With AI Software in Development
On a very basic level, any system that makes decisions autonomously — as opposed to being human-operated — is AI. [7] The incorporation of AI software in medical devices during surgical procedures can take several different forms, and be applied to a variety of functions.
One type with several different applications is called contextual artificial intelligence for computer-assisted intervention, or CAI4CAI. [8] During any given procedure, surgeries have visual displays providing data about the patient, the status of the surgical procedure itself and potential warnings based on the device's ability to identify concerning data.
When CAI4CAI is incorporated into these devices, it becomes like a heads-up display for a car, processing data from numerous sensors and providing focused and practical information to surgical staff. [9] CAI4CAI integrates the procedure steps, specific patient information and instantaneous sensory data, like vital signs, in the operating room. [10]
CAI4CAI then aggregates and analyzes this large and constantly updated stream of information to provide two things to the clinical surgery team. [11] First, a smart checklist is updated in real time, via cameras and other sensors, providing the main steps of the procedure for the surgeon and sub-checklists showing the progress of other team members. This ensures that the surgical team is aware of upcoming actions, and tools or medications that may be needed. [12]
Second, CAI4CAI works as a fail-safe to help prevent mistakes or quickly identify signs that an adverse event is approaching. [13] As the surgery progresses, the display can produce alerts and warnings if the data it receives demonstrates there is a concern, whether it results from the surgery itself or from patient-specific factors. [14] By analyzing the surgical data in real time, the CAI4CAI is designed to serve as an extra layer of protection for patient safety. [15]
A second type of medical device that utilizes AI software is a monitor that operates as a live risk decision tree. [16] Before the surgery, the device monitors display the risks of each step of a surgery, based on the type of surgery and the patient's individual risk factors. [17]
During the surgery, the decision tree progresses in real time to remind the surgical team of the highest risks for each step in the procedure, while considering data from the surgery. [18] Like CAI4CAI, these monitors use a number of sensors and cameras to update the relevant risks based on surgery progress. [19]
Along with risk warnings, the display recommends actions for the surgical staff if problems arise. [20] Regardless of the medical device used, surgical teams will likely be more prepared for, and hopefully be able to avoid, worst-case scenarios, while minimizing the team's cognitive load. [21]
Potential Product Liability Issues for Medical Devices With AI Software
Medical devices with AI software provide attractive benefits to physicians who have dozens of issues to focus on at a time, and to hospital systems who seek safer and more efficient procedures.
With these benefits in mind, companies that develop medical devices with AI software must be aware of the potential liability issues they may face. This potential is particularly acute in medical devices with AI software designed to take on, at least partially, monitoring and warning roles during surgical procedures.
It is well-known that doctors have a duty to properly monitor their patients during procedures. [22] Healthcare providers who put their patients under anesthesia have a duty to ensure the patient's continued safety during the procedure. [23]
As medical devices have become more complex, though, it is more common for medical device representatives to attend and perform an integral role in surgeries. [24] Indeed, this can lead surgeons to rely on industry representatives and their devices to spot warning signs provided by the medical device, and notify the surgical team accordingly. [25]
And when an adverse event happens during a procedure, product manufacturers can be potentially liable for the alleged professional negligence of the industry representative assigned to be the technical expert on the device. These claims rely on a classic negligence theory of liability, where the patient can bring claims against the doctor and the industry representative alike for negligence and medical malpractice.
However, when a medical device with AI software monitors vital signs or suggests real-time recommendations during a procedure — instead of a human healthcare provider or industry representative — any potential claims that may be made in litigation may now shift from the human decision makers to the product. [26]
In fact, the debate on legal liability when an adverse event involving a medical device incorporating AI software occurs has already begun, with the AMA's recent request to Congress to legislate protection from medical malpractice claims for healthcare providers in these instances. [27] The organization noted that such lawsuits are already being filed, and asked Congress to help shield healthcare providers from legal liability. [28]
Medical devices manufacturers should be aware of this shift, and the potential for product liability issues to arise, when designing and seeking approval to market medical devices with AI software.
Design Defect Claims
Adverse events can happen in the operating room during any surgery without the negligence of any member of the surgical staff or the failure of any medical device product used during procedures.
But when adverse events inevitably occur, inquiries could lead to quasi-design defect claims by the patient or the patient's family members. For example, a physician could expect a device to provide a warning or recommend a different course of action during surgery prior to an adverse event.
To bring a claim against a product for defective design, a patient must show that the medical device was designed in a defective manner, rendering it unreasonably dangerous to the user, and that this defective and unreasonably dangerous condition of the medical device proximately caused the plaintiff's injuries.
In such circumstances, the plaintiff usually must provide detailed allegations as to a defect in the medical device's design, and show that an alternative design would not have caused the alleged unreasonably dangerous condition. [29]
To proactively prepare for this shift in potential liability from a human healthcare provider to a medical device with AI software, medical device manufacturers should consider incorporating the following into devices that use AI:
- Acknowledgement that all AI technologies face the common problem of errors in data processing; [30]
- Provisions of nonliability in contracts and agreements based on errors in data processing;
- Algorithms that are grounded in information from credible and authoritative sources;
- Confidence ratings on conclusions and recommendations, so healthcare providers can weigh such confidence in determining whether to follow the device's recommendations; and,
- Citations to increase trust in the accuracy of the recommendations. [31]
Recognition of potential design defect claims regarding medical devices with AI software, and proactive consideration while designing such devices, will help ensure that the devices are safe and effective for use, and will aid in shielding manufacturers from potential liability.
Failure to Warn Claims
Another type of potential liability medical device manufacturers should consider when incorporating AI software into their products is failure to warn claims. These claims may arise when the patient claims that their medical provider was not provided with proper warnings from the manufacturer of the risks involved in the use of a medical device.
In the context of medical devices used during surgical procedures, one example may involve a doctor who was not aware a medical device would not work in specific situations, or that the device had a limited set of recommendations it could make. In this scenario, the device manufacturer could potentially be subject to a failure to warn claim, based on the manufacturer's purported failure to warn the healthcare provider about limitations of the medical device.
The adequacy of a warning is typically a question of fact directed to a jury. [32] So medical device manufacturers should work to provide fulsome instructions and warnings in the instruction manuals for medical devices with AI software — including clear information regarding limitations and designed functionality of the AI software in the medical device.
These fulsome instructions should be combined with training sessions on the medical device, so that the manufacturer can ensure healthcare providers can use and understand the device accurately.
Conclusion
Medical devices with AI software represent a tremendous opportunity for improving healthcare outcomes in the operating room. With this potential comes the risk that manufacturers may face a new generation of product liability claims.
While the emergence of medical devices with AI software is still in its infancy, the time for medical device manufacturers to think strategically about the future liability risks is now.
[1] Sebastian Bodenstedt, et al., Artificial Intelligence Assisted Surgery: Potential Challenges. National Library of Medicine, PMID: 33447600 (Nov. 4, 2020). Available at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7768095/.
[2] Id.
[3] Id.
[4] Id.
[5] Daniel Payne, Who pays when AI steers your doctor wrong?, Politico (March 24, 2024). Available at https://www.politico.com/news/2024/03/24/who-pays-when-your-doctors-ai-goes-rogue-00148447.
[6] Id.
[7] One definition, among many, comes from The National AI Initiative Act of 2020: "The term 'artificial intelligence' means a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments. Artificial intelligence systems use machine and human-based inputs to perceive real and virtual environments, abstract such perceptions into models through analysis in an automated manner, and use model inference to formulate options for information or action." § 15 U.S.C. 9401(3).
[8] Tom Vercauteren, et al., CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer-Assisted Interventions, 108 Proceedings of the IEEE 1 (Oct. 23, 2019). Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6952279/.
[9] See id.
[10] George Avrunin, et al., Toward Improving Surgical Outcomes by Incorporating Cognitive Load Measurement into Process-Driven Guidance, National Library of Medicine, PMID: 30140792 (May 2018). Available at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6103223/.
[11] Id.
[12] Id.
[13] Jim McCartney, AI Is Poised to "Revolutionize" Surgery, American College of Surgeons (June 7, 2023). Available at https://www.facs.org/for-medical-professionals/news-publications/news-and-articles/bulletin/2023/june-2023-volume-108-issue-6/ai-is-poised-to-revolutionize-surgery/.
[14] Id.
[15] Id.
[16] Heather Conboy, et al., Digital Cognitive Aids to Support Adaptation of Surgical Processes to COVID-19 Protective Policies, National Library of Medicine, PMID: 34723287, (Oct. 7, 2020). Available at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8555746/.
[17] Id.
[18] Id.
[19] Id.
[20] Id.
[21] Id.
[22] See, e.g., Jeffrey Hatef, Miki Katzir, et al., Damned if you monitor, damned if you don't: medical malpractice and intraoperative neuromonitoring for spinal surgery, Journal of Neurosurgery, Vol. 49 (2020). Available at https://doi.org/10.3171/2020.8.FOCUS20580.
[23] Id.
[24] Kristen Schleiter, Liability of Industry Representatives in the OR, AMA Journal of Ethics (Feb. 2010). Available at https://journalofethics.ama-assn.org/article/liability-industry-representatives-or/2010-02.
[25] AMA Opinion 10.6 Industry Representatives in Clinical Settings. Code of Medical Ethics. Available at https://www.ama-assn.org/delivering-care/ethics/industry-representatives-clinical-settings.
[26] James Beck, The Diagnostic Artificial Intelligence Speedbump Nobody's Mentioning, Drug and Device Law Blog (Nov. 8, 2018). Available at https://www.druganddevicelawblog.com/2018/11/the-diagnostic-artificial-intelligence-speedbump-nobodys-mentioning.html.
[27] "Liability protection from healthcare malpractices caused by artificial intelligence (AI) is reportedly doctors' latest bid to Congress, as per the American Medical Association." Aldohn Domingo, Healthcare Professionals Convene on Who to Blame for AI-Assisted Doctor, Tech Times (March 25, 2024). Available at https://www.techtimes.com/articles/302902/20240325/healthcare-professionals-convene-who-blame-ai-assisted-doctor-malpractices.htm. See also Daniel Payne, Who pays when AI steers your doctor wrong?, Politico (March 24, 2024). Available at https://www.politico.com/news/2024/03/24/who-pays-when-your-doctors-ai-goes-rogue-00148447.
[28] Id.
[29] See, e.g., Fearrington v. Boston Sci. Corp. , 410 F. Supp. 3d 794, 803-804 (S.D. Tex. 2019).
[30] Hugo Francisco de Souza, Navigating the minefield of AI in healthcare: Balancing innovation with accuracy, News Medical Life Sciences (March 25, 2004), available at https://www.news-medical.net/news/20240325/Navigating-the-minefield-of-AI-in-healthcare-Balancing-innovation-with-accuracy.aspx.
[31] Id.
[32] Hakim v. Safariland LLC , 79 F.4th 861, 869 (7th Cir. 2023).