Scoping Review of RCTs: AI Interventions Show Positive but Varied Impact in Clinical Practice

Background: The rapid expansion of artificial intelligence (AI) in health care has stimulated a growing number of randomized controlled trials (RCTs) intended to validate AI’s clinical utility. However, many AI models previously tested in retrospective or simulated settings lack real-world evidence. Investigating the breadth and depth of these RCTs is key to understanding the current status of AI in clinical practice.

Objective: This scoping review aimed to identify, classify, and evaluate RCTs that integrate modern AI (non-linear computational models, including deep learning) into patient management. The primary goal was to assess geographic distribution, trial design, outcomes measured (diagnostic performance, care management, patient behavior, clinical decision making), and the overall success rate of AI-based interventions.

Methods: The authors systematically searched PubMed, SCOPUS, CENTRAL, and the International Clinical Trials Registry Platform from January 2018 to November 2023. Included studies featured an AI intervention integrated into clinical workflows, with patient outcomes influenced by clinician–AI interactions or standalone AI systems. Exclusions encompassed linear-risk models and non-English publications. After screening 10,484 records, 86 RCTs were ultimately included. Simple descriptive statistics summarized trial characteristics, endpoints, and results.

Results: Most RCTs (63%) were single-center studies with a median sample size of 359 patients. Gastroenterology (43%) and Radiology (13%) were leading specialties, often focusing on deep learning algorithms for endoscopic or imaging tasks. The USA led in overall trial volume, followed by China, with 81% of all trials reporting positive primary outcomes (improvements or non-inferiority). Diagnostic yield or performance metrics predominated (54%), though some studies evaluated patient-centered endpoints such as adherence or symptom reduction. Despite these promising findings, 60% of trials measuring operational time showed mixed effects—some reported reduced procedural times (p<0.05), while others noted significant increases (p<0.05).

Conclusions: AI-driven interventions generally improved diagnostic measures and care processes, demonstrating potential for augmenting clinical decision making. Nevertheless, the prevalence of single-center designs limits the generalizability of outcomes. Publication bias remains a concern, given that negative or null results may be underreported. More extensive multicenter RCTs, greater demographic transparency, and standardized reporting are critical to fully determine AI’s clinical relevance.

Implications for Practice: AI tools might enhance screening, detection rates, and therapeutic monitoring in areas like gastroenterology, radiology, and cardiology. Clinicians should remain mindful of possible workflow inefficiencies and biases. Thorough validation and robust implementation strategies are essential before widespread adoption can be justified.

Study Strengths and Limitations: Strengths include a timely review capturing diverse RCTs up to late 2023 and strict inclusion criteria requiring true AI integration into patient care. Limitations include the English-only search and reliance on published results, potentially omitting unpublished or negative trials.

Future Research: Further investigations should prioritize multicenter, large-scale RCTs with meaningful clinical endpoints—quality of life, survival, and long-term safety. Enhanced adherence to reporting standards (CONSORT-AI) and recruitment of ethnically diverse populations are necessary steps to advance the field.

Reference: Han R, Acosta JN, Shakeri Z, Ioannidis JPA, Topol EJ, Rajpurkar P, et al. “Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review.” The Lancet Digital Health. 2024;6(5). DOI: http://doi.org/10.1016/S2589-7500(24)00047-5