AI IN HEALTHCARE

• Summary: On February 6, 2025, the National Science Foundation published a request for information (RFI) on the development of an artificial intelligence (AI) plan, as directed by President Trump’s executive order entitled, “Removing Barriers to American Leadership in AI.” The plan will be required to define the priority policy actions needed to sustain and enhance America’s AI dominance and ensure that unnecessary requirements do not hamper private sector innovation. The RFI seeks input on priority actions to be included in the plan. Reponses can address any relevant AI policy topic and are encouraged to include concrete policy options.

• Summary: The Food and Drug Administration (FDA) issued draft guidance providing lifecycle management and marketing submissions recommendations for AI-enabled devices, including software functions, consistent with its long promoted total product life cycle (TPLC) approach to medical device oversight. The draft guidance recommends the following information be included in submissions for AI-enabled devices:
  • Description of the device and its user interface to support FDA’s understanding of intended use, expected clinical workflow, use environment, features and design;
  • A comprehensive risk assessment to ensure safety and efficacy;
  • A clear explanation of data management practices, including training sets and methodology for mitigating AI bias;
  • Information regarding the model design, including its biases and limitations;
  • A performance validation to ensure predictable and reliable function;
  • A post-market performance monitoring plan to identify and respond to changes;
  • Elements and explanations of cybersecurity risk management and testing; and
  • A public submission summary to enable transparency to the public via a “Model Card.”

[For more information regarding this draft Guidance, McDermott Will & Emery has published the following On The Subject.]

• Summary: The Food and Drug Administration (FDA) issued draft guidance providing recommendations to sponsors on the use of AI to produce information or data intended to support applications or submissions for drugs and biological products. The draft guidance outlines a high-level seven-step risk-based process for sponsors to establish and assess the credibility of an AI model’s output. The draft guidance does not provide extensive details on each step, rather stating that the agency envisions interactive feedback and communication with interested parties would shape the framework. Accordingly, FDA strongly encourages sponsors to engage with the agency early in the drug development process to set expectations and identify potential challenges.

• Summary: On January 13, 2025, California Attorney General (AG) Rob Bonta issued two legal advisories: one for businesses generally (General Advisory) and one specific to healthcare entities (Health Advisory). These advisories identify existing California laws that already apply to artificial intelligence (AI). The advisories demonstrate the AG’s willingness to push the limits of applying existing law to AI and emphasize the need for businesses to follow several key principles:
  • Use AI responsibly, ethically, and safely.
  • Understand how AI systems are trained, what data they use, how they generate outputs, and the risks to individuals, the environment, competition, and the public.
  • Be transparent about when AI is used and when data is used to train AI.
  • Test, validate, and audit AI systems.
• Read our On the Subject for the full breakdown of these advisories.

• Summary: The Trump White House issued an Executive Order (EO), “Removing Barriers to American Leadership in Artificial Intelligence,” revoking former President Biden’s Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence, issued in 2023. The AI Action Plan, as directed by the EO, focuses on several key points:
  • Sustaining and Enhancing AI Dominance: The plan aims to maintain and boost the United States’ leadership in AI to promote human flourishing, economic competitiveness, and national security.
  • Eliminating Ideological Bias: It emphasizes the need for AI systems to be free from ideological bias and engineered social agendas
  • Revising Policies: Departments and agencies are instructed to revise or rescind policies, directives, regulations, and orders that hinder AI innovation.
  • Developing an AI Action Plan: Within 180 days, an AI Action Plan will be developed to outline specific actions and strategies to achieve these goals. The following parties were identified to lead development of the action plan: Assistant to the President for Science and Technology (APST), the Special Advisor for AI and Crypto, and the Assistant to the President for National Security Affairs (APNSA), in coordination with the Assistant to the President for Economic Policy, the Assistant to the President for Domestic Policy, the Director of the Office of Management and Budget (OMB Director), and the heads of such executive departments and agencies as the APST and APNSA deem relevant.

• Summary: Following President Biden’s Executive Order (EO) on the Safe, Secure and Trustworthy Development of AI, the Department of Commerce released new guidance and software to help improve the safety, security and trustworthiness of AI systems, including the following:
  • The Department’s National Institute of Standards and Technology (NIST) released two new items:
    1. 1. NIST’s AI Safety Institute has released the initial public draft of its guidelines on Managing Misuse Risk for Dual-Use Foundation Models (NIST AI 800-1), which outlines voluntary best practices for how foundation model developers can protect their systems from being misused to cause deliberate harm to individuals, public safety and national security. The guidelines offer seven key approaches for mitigating risks that models will be misused, along with recommendations for how to implement them and how to be transparent about their implementation. Comments are being accepted until Sept. 9, 2024, at 11:59 p.m to NISTAI800-1@nist.gov.
       2. A testing platform designed to help AI system users and developers measure how certain types of attacks can degrade the performance of an AI system. The open-source software is available for free download.
  • NIST also released three finalized guidance documents that were first released in April for public comment:
    1. 1. The AI RMF Generative AI Profile (NIST AI 600-1), which can help organizations identify unique risks posed by generative AI and proposes actions for generative AI risk management that best aligns with their goals and priorities. This is intended to be a companion resource for users of NIST’s AI RMF.
       2. The Secure Software Development Practices for Generative AI and Dual-Use Foundation Models (NIST Special Publication (SP) 800-218A), designed to be used alongside the Secure Software Development Framework (SP 800-218) to address the training and use of AI systems.
       3. A Plan for Global Engagement on AI Standards (NIST AI 100-5), designed to drive the worldwide development and implementation of AI-related consensus standards, cooperation and coordination, and information sharing.
  • The Department’s U.S. Patent and Trademark Office (USPTO) issued a guidance update on patent subject matter eligibility to address innovation in critical and emerging technologies, including AI.

• Summary: The Coalition for Health AI, an alliance of major health systems and tech companies, has released a blueprint to build trust in artificial intelligence’s use in healthcare. The blueprint focuses on framing risks, measuring impacts, allocating risk resources and strong governance.

• Summary: The Connected Health Initiative’s Health AI Task Force released recommended principles to guide policymakers in taking action on AI. The principles cover areas such as research, quality assurance and oversight, access and affordability, bias, education, collaboration and interoperability, workforce issues, ethics, and privacy and security.