Summary: The researchers evaluated the performance of ChatGPT, a large language-based AI, on the United States Medical Licensing Exam (USMLE). The USMLE is a set of three standardized tests required for medical licensure in the US. The results showed that ChatGPT performed at or near the passing threshold (60%) on all three exams without specialized training or reinforcement. The AI also demonstrated a high level of concordance and insight in its explanations, suggesting that it may have potential to assist with medical education and potentially, clinical decision-making in the future. This marks a notable milestone in AI maturation and increased confidence in its trust and explainability.
(By ChatGPT, reviewed and edited)
Commentary from one of the authors on Twitter (thread – click for more)
our work showing that ChatGPT passed the US Medical Licensing Exam (USMLE) is published @PLOSDigiHealth!
there's tons of chatter about LLMs in healthcare.
AI is not going to replace doctors anytime soon, but here are the key takeaways from this work:
— Morgan Cheatham (@morgancheatham) February 13, 2023