Chat M.D.

Abstract Large language models (LLMs) such as Claude and ChatGPT are the most powerful artificial intelligence (AI) systems ever created, and they are being used to diagnose and treat patients. But LLMs have been shown to be unreliable, unpredictable, and unsafe on occasion. New AI guidelines recommend hundreds of standards, such as ‘transparency’, ‘trustworthiness’, and ‘safety’. But there is deep uncertainty whether these are sufficient. The literature focuses mostly on which standards best suit AI models, not on how to transmute standards into law. This article does that by considering AI guidelines as a starting point, then evaluating whether existing frameworks for ensuring quality in medicine might form the basis for AI governance. Along the way, the article identifies emerging areas of consensus, lingering questions, and lessons from other areas of law. This article suggests that we should not only treat LLMs like nascent medical professionals who must meet minimum standards of competence and responsibility, but also subject them to measurable, product-like standards of safety and performance. When paired with reimbursement incentives and legal liability, LLMs will be treated on par with others who diagnose and treat US patients. This article offers a path to genuine oversight of LLMs in medicine.