Use of large language models to identify pseudo‐information: Implications for health information

Abstract Background Open‐access scientific research is an essential source of health‐related information and self‐education. Artificial intelligence‐based large language models (LMMs) may be used to identify erroneous health information. Objective To investigate to what extent LMMs can be used to identify pseudo‐information. Methods Four common LMM applications (ChatGPT‐4o, Claude 3.5 Sonnet, Gemini and Copilot) were used to investigate their capability to indicate erroneous information provided in an open‐access article. Results Initially, ChatGPT‐4o and Claude were able to mark the provided article as an unreliable information source, identifying most of the inaccuracy problems. The assessments provided by Gemini and Copilot were inaccurate, as several critical aspects were not identified or were misinterpreted. During the validation phase, the initially accurate assessment of ChatGPT‐4o was not reproducible, and only Claude was able to detect several critical issues in this phase. The verdicts of Copilot and Gemini remained largely unaltered. Discussion Large heterogeneity exists between LMMs in identifying inaccurate pseudo‐information. Replication in LMM output may constitute a significant hurdle in their application. Conclusion The accuracy of LMMs needs to be further improved until they can be reliably used by patients for health‐related online information and as assistant tools for health information and library services workers without restriction.