Accurate and timely diagnosis underpins effective patient management because it enables earlier treatment, can improve outcomes and reduces unnecessary investigations and costs [1,2]. Diagnostic error nevertheless remains common: missed, incorrect or delayed diagnoses are reported in 10–15% of cases, autopsy series reveal diagnostic discrepancies in about 20%, and 60–80% of radiological errors are perceptual and associated with fatigue, interruptions or satisfaction of search, while communication failures also contribute substantially [3]. Variability in interpretation between professionals and time-consuming manual procedures further limit conventional diagnostic workflows, particularly for severe and rapidly progressing conditions [4]. Artificial intelligence (AI) has been proposed as a complement to human interpretation. AI systems can perform repetitive, high-volume image-analysis tasks consistently, whereas clinicians remain better suited to integrating clinical context, managing ambiguity and adapting to unfamiliar situations [5]. In radiology, machine learning (ML) and, in particular, deep learning (DL) with convolutional neural networks (CNNs) have been applied to computed tomography (CT), magnetic resonance imaging (MRI), radiography, mammography and positron emission tomography to detect and characterize lesions such as tumors and fractures [6,7]; the long-established digital imaging infrastructure of radiology has facilitated this development [4]. In pathology, whole-slide imaging allows AI models to identify cellular and tissue patterns associated with disease [8,9]. Digital pathology has, however, been adopted more slowly because of practical and financial constraints, the continued need to produce and store glass slides, additional staffing and equipment requirements, and image files that are at least ten times larger than radiological images [4,10]. Computer-aided diagnosis (CAD) was introduced to support rather than replace the reader, for example in mammography and in the detection of pulmonary nodules on CT [11], whereas conventional interpretation remains affected by inter-observer variability [12]. Contemporary DL models learn image features directly through stacked convolution and pooling layers [6,7,13], and newer architectures, including residual networks, Inception-type networks, U-Net and vision transformers (ViT), have extended these capabilities [14,15]. Recent examples include a hybrid ViT–deformable CNN model for lung and colon cancer histopathology [16], a CNN framework for multi-view chest radiographs [17] and a fine-tuned ResNet34 model for brain tumor classification [18]. Applications have also been described in neuroradiology and musculoskeletal radiology [19] and in liver, skin and kidney pathology [20–22]. A meta-analysis of AI in digital pathology reported high summary sensitivity and specificity but also substantial heterogeneity and a high prevalence of risk-of-bias concerns [23]. High reported performance should not be equated with clinical benefit. In a meta-analysis of 83 studies, generative AI models achieved an overall diagnostic accuracy of 52.1%, with no significant difference from physicians overall (p = 0.10) but significantly lower accuracy than expert physicians (p = 0.007) [24]. Methodological reviews of AI diagnostic research have repeatedly identified retrospective designs, selection bias, homogeneous development datasets with a risk of overfitting, limited external validation and incomplete reporting [23,25], and the risks of over-reliance on AI in diagnostic practice have been highlighted [26]. Independent validation frameworks have been proposed to screen AI models before clinical use [27], and AI-specific reporting extensions, prospective registration and external validation have been recommended [25]. Previous reviews have also differed in their approach to synthesis: a review of externally validated CNN models for emergency head CT did not pool results because of heterogeneity [28], whereas a review of AI in digital pathology applied bivariate pooling [23]. It therefore remains unclear how reported discrimination varies with validation strategy across the rapidly growing literature on diagnostic AI in imaging and pathology, and how much of this literature provides the data required for formal diagnostic test accuracy (DTA) synthesis. The primary objective of this systematic review was to describe the reported discriminative performance, expressed as the area under the receiver operating characteristic curve (AUC), of AI models used for diagnosis in medical imaging and pathology and published between January 2021 and February 2026, stratified by validation strategy, model architecture and imaging modality. The secondary objective was to estimate, in an exploratory DTA meta-analysis, the summary sensitivity and specificity of AI models evaluated on external validation sets with extractable 2×2 data. External validation was therefore a stratification variable for the primary objective and an eligibility criterion for the secondary objective.

  Protocol and Registration This systematic review is reported in accordance with the Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) statement [29]; the completed checklist is provided as Supplementary File 2. The review was not registered in PROSPERO or in any other registry, and the review protocol was not published. The eligibility criteria, outcomes and methods are reported in full in this article and its Supplementary Material.   Review Question The review question was structured according to the population, index test, reference standard and target condition framework. The population comprised patients, or specimens obtained from patients, who underwent diagnostic imaging or pathological examination, without restriction by age, sex, disease severity or clinical setting. The index test was an AI model (conventional ML, DL including CNN and transformer-based architectures, or a combination) applied to a diagnostic classification task. The reference standard was that used in each primary study, such as histopathology, expert or consensus reading, clinical follow-up or a confirmed clinical diagnosis. The target condition was the disease or abnormality defined by each primary study. The primary outcome was the AUC; the secondary outcomes were sensitivity and specificity derived from 2×2 tables for an external validation set and the summary receiver operating characteristic (SROC) curve.   Eligibility Criteria Peer-reviewed primary studies were eligible if they were published in English between January 2021 and February 2026, had freely accessible full text, evaluated a diagnostic AI model in the population defined above and reported the AUC of the model. The start date of 2021 was chosen to capture recent deep learning architectures. Reports were excluded when diagnostic accuracy could not be determined, when the application was not related to imaging or pathology, when the study design was inappropriate, or when the AI model was unavailable or had been withdrawn. Studies included in the review were eligible for the DTA meta-analysis if they evaluated the model on an external validation set (Section 2.4) for a binary target condition and reported sufficient information to construct a 2×2 table of true-positive (TP), false-positive (FP), false-negative (FN) and true-negative (TN) results against the reference standard for that set. English was selected because most AI diagnostic studies are published in English and to avoid translation errors. Only articles with freely accessible full text were included, because the full text was required to extract the AUC and the validation design. Open-access status is not related to diagnostic validity, and this criterion may have introduced availability and publication bias, because studies published in subscription journals may differ in design, setting and reported performance; its potential impact is considered in Section 4.6.   Operational Definition of Validation Strategy External validation was defined as evaluation of a trained model on data that were not used for model development (training, tuning or internal testing) and that originated from a different institution or site, a different time period, or an independent dataset. Internal validation comprised split-sample, cross-validation or bootstrap evaluation within the development dataset. Each study was classified into one of four categories according to the validation reported in its methods: internal validation, external validation, internal and external validation, or evaluation within a prospective randomized controlled trial (RCT). The same definition is applied throughout the Results, tables and Supplementary Material.   Information Sources and Search Strategy PubMed (MEDLINE), Embase, the Cochrane Library, IEEE Xplore and Web of Science were searched for records published from January 2021 to February 2026. The search strings combined free-text terms for AI (artificial intelligence, machine learning, deep learning, neural network*, convolutional neural network, CNN), for imaging and pathology (radiology, medical imaging, imaging, medical image, pathology, histopathology, cytology), for diagnostic accuracy (diagnosis, diagnostic accuracy, sensitivity, specificity) and for the diagnostic procedure (pathological examination, diagnostic imaging), linked with Boolean operators. Controlled vocabulary (MeSH or Emtree) was not used. The terms differed between databases: the Embase and Web of Science strings did not include the pathology, histopathology, cytology, sensitivity or specificity terms, and the IEEE Xplore string used the All Metadata field. Database limits were applied as follows. In PubMed, the limits were publication within the last five years, free full text, English language, humans, MEDLINE, and the article types clinical study, clinical trial, observational study and randomized controlled trial. In the Cochrane Library, the limits were the last five years, trials, English language and humans. Embase, IEEE Xplore and Web of Science were limited to 2021–2026, and the Web of Science search was restricted to six journals (BMC Cancer, Cancer Medicine, EBioMedicine, Insights into Imaging, Diagnostic and Interventional Radiology and JMIR Medical Informatics). The complete search strings, limits and numbers of retrieved records for each database are reported in Supplementary Table S1.     Table 1: Reported AUC Category by Validation Strategy (n = 170 Studies; One Record Per Study) Validation strategy n AUC ≥0.95 AUC 0.90–0.94 AUC 0.80–0.89 AUC <0.80 Mean±SD Median (IQR) Internal validation 78 47 (60.3) 11 (14.1) 14 (17.9) 6 (7.7) 0.929±0.077 0.960 (0.893–0.982) External validation 72 28 (38.9) 19 (26.4) 20 (27.8) 5 (6.9) 0.912±0.075 0.920 (0.860–0.980) Internal and external validation 15 9 (60.0) 3 (20.0) 2 (13.3) 1 (6.7) 0.945±0.055 0.970 (0.925–0.982) Prospective RCT 4 1 (25.0) 0 (0.0) 0 (0.0) 3 (75.0) 0.798±0.126 0.760 (0.735–0.823) Not recorded 1 1 (100.0) 0 (0.0) 0 (0.0) 0 (0.0) 0.983 – Total 170 86 (50.6) 33 (19.4) 36 (21.2) 15 (8.8) 0.921±0.078 0.950 (0.880–0.980) Values are n (row %) unless stated otherwise. AUC categories are study-defined descriptive groupings, not clinical thresholds. AUC, area under the receiver operating characteristic curve; IQR, interquartile range; RCT, randomized controlled trial; SD, standard deviation   Study Selection Before screening, duplicate records, records marked as ineligible by automation tools and records that could not be accessed were removed. Titles and abstracts were then screened with AUC as the principal screening outcome; some records were excluded with an AI-based screening tool and the remainder by the reviewers. Potentially eligible reports were assessed in full text against the eligibility criteria.   Data Extraction Two reviewers independently extracted data using a standardized Microsoft Excel form: first author, year of publication, study design, target condition, imaging or pathology modality, index test characteristics (AI architecture and validation strategy), reference standard, flow and timing, AUC and, where available, sensitivity, specificity and 2×2 data. AUC was extracted for the external validation set when one was reported and otherwise for the internal validation set. Disagreements were resolved by discussion between the two reviewers until consensus was reached. The F-score was initially included in the extraction form but was removed because it was inconsistently reported. When key metrics were missing, the corresponding authors were contacted by e-mail; studies without a reply after two attempts were retained in the review with the data available. Data were managed in Microsoft Excel (Power Query and PivotTables). During the revision, the data of the nine studies considered for the meta-analysis were re-extracted from the full texts. TP, FP, FN and TN were taken from reported confusion matrices or calculated from the reported sensitivity, specificity and numbers of units with and without the target condition, and each calculated table was checked against the sensitivity, specificity, accuracy and confidence intervals reported in the study. When a study reported more than one external validation set, the largest set with extractable 2×2 data was used, and the AUC recorded for the study was taken from the same set. This re-extraction corrected the validation category of five studies, the AUC of three studies and the architecture or modality label of seven studies; the corrected values are used throughout this article.     Table 2: QUADAS-2 Risk of Bias and Concerns Regarding Applicability (n = 170 studies) Domain Low Unclear High Risk of bias Patient selection 48 (28.2) 122 (71.8) 0 (0.0) Index test 162 (95.3) 8 (4.7) 0 (0.0) Reference standard 169 (99.4) 1 (0.6) 0 (0.0) Flow and timing 60 (35.3) 110 (64.7) 0 (0.0) Concerns regarding applicability Patient selection 0 (0.0) 170 (100.0) 0 (0.0) Index test Not assessed Not assessed Not assessed Reference standard Not assessed Not assessed Not assessed Values are n (%). Per-study judgments are reported in Supplementary Table S4 and in the traffic-light plot (Supplementary Figure S2). QUADAS-2, Quality Assessment of Diagnostic Accuracy Studies-2   Unit of Analysis The unit of analysis was the study. Each included study contributed a single record, consisting of one AI model evaluated on one validation set; no publication contributed more than one model, dataset, validation set or outcome, so no participant or image set was counted more than once. When a publication reported several validation sets, the result for the external validation set was extracted. In the meta-analysis, each study contributed one 2×2 table. The unit of observation within studies (patients, lesions, images, slides or smears) differed between studies and is reported in Table 3.     Table 3: Characteristics of the Seven Studies Included in the Diagnostic Test Accuracy Meta-Analysis Study Target condition Modality Index test External validation set; units Reference standard Shi et al. [39] Nasopharyngeal carcinoma (malignant vs non-malignant) Endoscopic images Swin Transformer (STND) Prospective, 10 hospitals; 1,885 images from 945 patients Biopsy (carcinoma, hyperplasia) Yang et al. [40] Adhesive capsulitis MRI (PD and T2) Radiomics with LightGBM Second hospital; 57 patients Clinical diagnostic criteria Valesano et al. [41] STIC or STIL WSI Attention-based MIL (UNI encoder) Independent cohort, development institution; 43 cases Pathology with p53 and Ki-67 IHC Liu H et al. [42] Breast cancer ABVS ResNet50, three-plane fusion Third hospital; 119 lesions Histopathology (biopsy or surgery) Lee et al. [43] Myocardial infarction ECG images Dedicated ECG AI (STEMI output) Public dataset from another country; 928 ECG images Dataset labels (clinical diagnosis) Xu et al. [44] DDH (Graf type III or IV) Ultrasound (videos) HSNN (YOLOv11m and ResNet50) Prospective, 4 hospitals; 151 infants Graf classification, expert consensus Wang J et al. [45] Thyroid nodules (TBSRTC III+ vs benign) FNA cytology WSI ThyroPower ensemble Prospective temporal set; 537 smears from 469 patients TBSRTC diagnosis by senior cytopathologists Units of analysis are those of the external validation set used for the 2×2 table. ABVS, automated breast volume scanner; DDH, developmental dysplasia of the hip; ECG, electrocardiogram; FNA, fine-needle aspiration; HSNN, HipSonoNeuNet; IHC, immunohistochemistry; MIL, multiple-instance learning; MRI, magnetic resonance imaging; PD, proton density; STEMI, ST-elevation myocardial infarction; STIC, serous tubal intraepithelial carcinoma; STIL, serous tubal intraepithelial lesion; TBSRTC, The Bethesda System for Reporting Thyroid Cytopathology; WSI, whole-slide image   Outcome Definitions and AUC Categories For descriptive purposes, AUC values were grouped into four study-defined categories: <0.80, 0.80–0.89, 0.90–0.94 and ≥0.95. These cut-offs are not validated clinical thresholds; they were chosen to describe the distribution of the reported values, most of which exceeded 0.80, and only the highest category corresponds to the value (≥0.90) that a commonly cited interpretive scale describes as excellent discrimination [30]. The AUC summarizes discrimination across all thresholds and does not describe calibration, performance at the operating threshold used in practice, clinical utility or performance relative to clinicians.   Risk of Bias and Applicability Both reviewers independently assessed each included study with the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool [31] in four domains (patient selection, index test, reference standard, and flow and timing), each judged as low, high or unclear risk of bias on the basis of the signaling questions. For the reference standard domain, these questions address whether the reference standard is likely to classify the target condition correctly and whether its results were interpreted without knowledge of the index test. Concerns regarding applicability were judged separately from risk of bias and were recorded for the patient-selection domain. Inter-reviewer agreement on the QUADAS-2 judgments was κ = 0.70, indicating substantial agreement [32]; disagreements were resolved by discussion.   Statistical Analysis Systematic review. Reported AUC values were summarized as frequencies and percentages per category and as mean (standard deviation, SD) and median (interquartile range, IQR) by validation strategy, architecture group and modality. No formal statistical comparisons were made between validation strategies, architectures or modalities, because each group combined different target conditions, datasets, reference standards and study designs; differences between groups are therefore reported descriptively. Study-reported AUC values were not pooled, because they were obtained for different tasks, thresholds and datasets and were not accompanied by consistently reported measures of precision. Meta-analysis. Sensitivity and specificity were pooled jointly with the bivariate random-effects model of Reitsma et al. [33], which models logit-transformed sensitivity and false-positive rate (FPR, 1 − specificity) with their between-study variances and correlation, fitted by restricted maximum likelihood. A continuity correction of 0.5 was added to all cells of all studies because two studies contained a zero cell. The SROC curve was derived from the bivariate model parameters with the Rutter–Gatsonis parameterization [34] and is presented with the 95% confidence region for the summary point and the 95% prediction region; the area under the SROC curve (SROC AUC) is reported only as a descriptive summary of the fitted curve and was not obtained by pooling study-level AUC values. Heterogeneity was quantified by the between-study standard deviations (τ) of logit sensitivity and logit FPR, their correlation and the prediction region; univariate I² values are reported descriptively for the forest plots only. Forest plots show study-level estimates with exact binomial 95% confidence intervals (CIs). The pooled estimate combines studies of different target conditions, modalities, reference standards and architectures and was therefore calculated as an exploratory summary of the accuracy of externally validated AI models in this set of studies, not as an estimate for any specific clinical application. Two sensitivity analyses were performed: restriction to studies whose external validation set came from institutions not involved in model development, and leave-one-out analysis. Subgroup analyses and meta-regression were not performed because no target condition was represented by more than one study, and Deeks’ funnel plot asymmetry test was not performed because fewer than ten studies were included [35]. The meta-analysis was performed in R 4.3.3 with the metafor package 4.4.0 (rma.mv function); the code is provided as Supplementary File 4. Descriptive statistics were calculated in Python 3.12 with pandas 3.0 and cross-checked with the spreadsheet formulas in the study-level data file (Supplementary File 3). Figures were drawn with matplotlib 3.10.

  Study Selection The searches identified 1,540 records: 691 in PubMed, 606 in Embase, 209 in the Cochrane Library, 23 in IEEE Xplore and 11 in Web of Science. Before screening, 37 records were removed (11 duplicates, 10 records marked as ineligible by automation tools and 16 records that could not be accessed). Of the 1,503 records screened, 736 were excluded (229 with the AI-based screening tool and 507 by the reviewers). Of the 767 reports sought for retrieval, 2 could not be retrieved, and 765 were assessed in full text. After exclusion of 595 reports (diagnostic accuracy not determined, n = 413; not related to imaging or pathology, n = 145; inappropriate study design, n = 34; AI model unavailable or withdrawn, n = 3), 170 studies were included in the review (Figure 1). Of the nine studies considered for the meta-analysis, seven provided 2×2 data from an external validation set and were included; the remaining 163 studies contributed to the descriptive synthesis only (Section 3.6).   Figure 1: PRISMA-DTA flow Diagram of Study Selection. AI, Artificial Intelligence; DTA, Diagnostic Test Accuracy; PRISMA-DTA, Preferred Reporting Items for a Systematic Review and Meta-aAalysis of Diagnostic Test Accuracy Studies   Characteristics of the Included Studies The 170 studies were published between 2021 and 2026 (2021, n = 3; 2022, n = 17; 2023, n = 20; 2024, n = 45; 2025,n = 81; 2026, n = 4). The most frequent modalities were CT (n = 35), MRI (n = 21), radiography (n = 19), ultrasound (n = 16) and whole-slide imaging (n = 12). Eleven records were based primarily on non-image data or had no recorded modality (electronic health record [EHR] reports, natural language processing, plasma lipidomics, clinical metadata or other non-imaging data), including multimodal models that fused EHR data with radiographs [36]; these records are retained in the descriptive synthesis and are identified in Supplementary Table S2. Internal validation was reported in 78 studies (45.9%), external validation in 72 (42.4%), internal and external validation in 15 (8.8%) and evaluation within a prospective RCT in 4 (2.4%); the validation strategy was not recorded for 1 study (0.6%). Thus, 87 studies (51.2%) reported any external validation. The characteristics of all included studies are listed in Supplementary Table S2.   Distribution of Reported AUC Values by Validation Strategy The median reported AUC was 0.950 (IQR 0.880–0.980; range 0.680–1.000). AUC was ≥0.95 in 86 studies (50.6%), 0.90–0.94 in 33 (19.4%), 0.80–0.89 in 36 (21.2%) and <0.80 in 15 (8.8%); overall, 119 studies (70.0%) reported an AUC ≥0.90. These values are study-level descriptive summaries of reported results obtained for different tasks, datasets and validation designs; they are not independent estimates of diagnostic accuracy and were not pooled. Validation strategy and AUC category are different variables and are cross-tabulated in Table 1. Of the 86 studies with an AUC ≥0.95, 47 reported internal validation, 28 external validations, 9 internal and external validation, 1 an RCT and 1 did not record the validation strategy; thus 37 of these 86 studies (43.0%) reported any external validation. An AUC ≥0.95 was reported by 60.3% of studies with   Reported AUC Values by AI Architecture The median reported AUC was 0.959 (IQR 0.900–0.985) for CNN-based DL models (n = 65), 0.970 (0.915–0.982) for other or unspecified DL models (n = 35), 0.950 (0.920–0.990) for transformer-based or hybrid CNN–transformer models (n = 9), 0.965 (0.960–0.980) for combined DL and ML approaches (n = 9) and 0.900 (0.840–0.953) for conventional ML models, including radiomics (n = 45). Two studies evaluated GPT-4 (AUC 0.770 and 1.000), and the two conventional CAD systems, both evaluated within RCTs, reported AUCs of 0.69 and 0.77. DL-based models therefore reported higher median AUC values than conventional ML models. Because the groups differed in target conditions, datasets and validation strategies and no formal comparative analysis was performed, this difference does not demonstrate the superiority of any architecture. The results for each architecture, as extracted, and the grouping used are reported in Supplementary Table S3.   Risk of Bias and Applicability QUADAS-2 judgments for all 170 studies are summarized in Table 2 and reported per study in Supplementary Table S4 and Supplementary Figures S1 and S2. No study was judged to be at high risk of bias in any domain. Risk of bias was unclear for patient selection in 122 studies (71.8%), mainly because of retrospective designs, the use of publicly available datasets and the absence of information on consecutive or random sampling. Risk of bias for flow and timing was unclear in 110 studies (64.7%) because of multiple cohorts or datasets and incomplete reporting of exclusions and of the interval between the index test and the reference standard. Risk of bias for the index test was low in 162 studies (95.3%) and unclear in 8 (4.7%). Risk of bias for the reference standard was low in 169 studies (99.4%) and unclear in 1 (0.6%). Reference standards were heterogeneous and included histopathology, radiologist consensus, clinical diagnosis and EHR-confirmed outcomes. Reference standards based on clinical diagnosis, EHR records or reader consensus may misclassify the target condition, so the predominance of low-risk judgments in this domain should be interpreted with caution and is considered a limitation (Section 4.6). Concerns regarding applicability of patient selection were judged unclear in all 170 studies.   Diagnostic Test Accuracy Meta-Analysis Nine studies had been considered for the meta-analysis. After re-extraction from the full texts, two were excluded: in one, the test set was a random split of the single-centre development dataset (internal validation) [37], and in the other, the test set was a random split of public datasets and the task was multiclass classification of blood-cell types without a binary target condition [38]. Seven studies were included, evaluating AI models for nasopharyngeal carcinoma on endoscopic images [39], adhesive capsulitis on MRI [40], serous tubal intraepithelial lesions on whole-slide images [41], breast cancer on automated breast volume scanner images [42], myocardial infarction on electrocardiogram images [43], developmental dysplasia of the hip on ultrasound [44] and thyroid nodules on fine-needle aspiration slides [45] (Table 3). The external validation sets comprised 3,720 units of analysis (images, patients, lesions, cases or smears); five came from institutions not involved in model development, one was an independent cohort from the development institution and one was a prospective temporal validation set. Risk of bias was low for patient selection in six of the seven studies and unclear in one, and risk of bias for flow and timing was unclear in all seven. Study-level sensitivity ranged from 0.897 to 1.000 and specificity from 0.659 to 0.975 (Table 4, Figure 2). The summary sensitivity was 92.1% (95% CI 88.5–94.6) and the summary specificity was 90.1% (95% CI 81.3–95.0). The between-study standard deviation was 0.376 for logit sensitivity and 0.931 for logit FPR, indicating greater heterogeneity in specificity than in sensitivity, and the correlation between logit sensitivity and logit FPR was −0.700, giving no indication of a threshold effect. The 95% prediction region extended from about 80.2% to 97.1% for sensitivity and from 43.9% to 99.1% for specificity (Figure 3), so the accuracy of a new externally validated model could differ substantially from the summary point. The SROC AUC was 0.957. Univariate I² was 50.5% for sensitivity and 94.1% for specificity. Restricting the analysis to the five studies validated at institutions not involved in model development gave a summary sensitivity of 92.5% (95% CI 88.1–95.3) and specificity of 88.0% (74.6–94.8); omitting one study at a time gave summary sensitivities of 91.2–92.8% and specificities of 87.8–92.1% (Supplementary Table S8). In two studies, the operating threshold was selected in the validation data with the Youden index [40,43].     Table 4: 2×2 Data, Study-Level Accuracy and Summary Estimates of the Diagnostic Test Accuracy Meta-Analysis Study TP FP FN TN Sensitivity (95% CI) Specificity (95% CI) Shi et al. [39] 612 169 56 1048 0.916 (0.893–0.936) 0.861 (0.840–0.880) Yang et al. [40] 15 14 1 27 0.938 (0.698–0.998) 0.659 (0.494–0.799) Valesano et al. [41] 3 1 0 39 1.000 (0.292–1.000) 0.975 (0.868–0.999) Liu H et al. [42] 53 11 4 51 0.930 (0.830–0.981) 0.823 (0.705–0.908) Lee et al. [43] 231 20 8 669 0.967 (0.935–0.985) 0.971 (0.956–0.982) Xu et al. 2025 6 12 0 133 1.000 (0.541–1.000) 0.917 (0.860–0.957) Wang J et al. [44] 235 16 27 259 0.897 (0.854–0.931) 0.942 (0.907–0.966) Summary estimate (7 studies) - - - - 0.921 (0.885–0.946) 0.901 (0.813–0.950) External institutions only (5 studies) - - - - 0.925 (0.881–0.953) 0.880 (0.746–0.948) Leave-one-out range - - - - 0.912–0.928 0.878–0.921 Study-level 95% CIs are exact (Clopper–Pearson). Summary estimates are from the bivariate random-effects model (restricted maximum likelihood) with a continuity correction of 0.5 added to all cells. τ, logit sensitivity 0.376; τ, logit FPR 0.931; correlation −0.700; SROC AUC 0.957. CI, confidence interval; FN, false negative; FP, false positive; FPR, false-positive rate; SROC, summary receiver operating characteristic; TN, true negative; TP, true positive     Figure 2: Paired Forest Plots of Sensitivity (Left) and Specificity (Right) with Exact 95% Confidence Intervals for the Seven Studies in the Meta-Analysis. Studies are Identified by First Author, Year and Target Condition, followed by TP/FP/FN/TN; The Diamonds Show the Summary Estimates from the Bivariate Random-Effects Model. CI, Confidence Interval; FN, False Negative; FP, False Positive; TN, True Negative; TP, True Positive     Figure 3: Summary Receiver Operating Characteristic (SROC) Curve from the Bivariate Random-Effects Model, with the Study Estimates (Circle Size Proportional to Study Size), the Summary Point, the 95% Confidence Region and the 95% Prediction Region. Axes are Truncated. AUC, Area under the Curve   Reported AUC Values by Imaging Modality The distribution of AUC categories by modality is shown in Figure 4 and in Supplementary Tables S6 and S7. CT contributed the largest number of studies (AUC ≥0.95, n = 12; 0.90–0.94, n = 5; 0.80–0.89, n = 12; <0.80, n = 6), followed by MRI (9, 7, 5 and 0), radiography (11, 3, 3 and 2) and ultrasound (5, 6, 4 and 1). Among the 86 studies with an AUC ≥0.95, the most frequent modalities were CT (n = 12), radiography (n = 11), MRI (n = 9) and whole-slide imaging (n = 8); the remaining 46 studies were distributed across 20 other categories, each containing one to five studies. This distribution largely reflects the number of studies available for each modality, and the data do not support conclusions about modality-specific performance.   Figure 4: Number of Included Studies by Modality and Reported AUC Category (N = 170). Modalities with Fewer than Five Studies are Combined as Other Imaging or Pathology Modalities; Records Based Primarily on Non-Image Data or Without a Recorded Modality are Shown Separately. AUC Categories are Study-Defined Descriptive Groupings. AUC, Area under the Receiver Operating Characteristic Curve; Ct, Computed Tomography; Mri, Magnetic Resonance Imaging; Wsi, Whole-Slide Imaging

  Principal Findings This review identified 170 studies published between 2021 and 2026 that reported the AUC of AI models for diagnosis in medical imaging and pathology. Reported AUC values were high, with 70.0% of studies reporting an AUC ≥0.90. These values were, however, obtained for heterogeneous target conditions and datasets, most studies were retrospective, and risk of bias was unclear for patient selection in 71.8% and for flow and timing in 64.7% of studies. Only seven studies provided 2×2 data from an external validation set, and these addressed clinically unrelated target conditions. The findings therefore describe reported discrimination in selected datasets rather than the performance expected in clinical practice. Similar concerns about the quality of evidence and the scarcity of external validation were raised in an earlier systematic comparison of DL models with health-care professionals [46].   Validation Strategy and Generalizability Studies with only internal validation reported an AUC ≥0.95 more often than studies with external validation (60.3% vs 38.9%) and had a higher mean AUC (0.929 vs 0.912), and the four studies conducted within RCTs reported the lowest values (mean 0.798). The highest mean AUC was observed in the small group of 15 studies that reported both internal and external validation (0.945). These differences are descriptive and are not adjusted for task or dataset, but their direction is consistent with previous evidence: in a systematic review of DL algorithms for radiological diagnosis, 81% of algorithms showed some decline in performance on external datasets, 49% at least a modest decline and 24% a substantial decline [47]. Internal validation can overestimate performance through overfitting and data leakage, whereas external data introduce differences in populations, equipment and acquisition protocols that more closely reflect real-world use [47]. Generalizability is also difficult to assess because many DL studies provide limited demographic and clinical information about their development datasets and because the decision processes of DL models are not fully transparent [47]. With only four RCTs, no conclusion can be drawn about prospective performance.   Interpretation of the Pooled Estimate and Clinical Heterogeneity The seven studies in the meta-analysis addressed unrelated target conditions (nasopharyngeal carcinoma, adhesive capsulitis, serous tubal intraepithelial lesions, breast cancer, myocardial infarction, developmental dysplasia of the hip and thyroid nodules) with different modalities, reference standards, units of analysis and architectures. The summary sensitivity of 92.1% and specificity of 90.1% therefore describe the average accuracy of this particular set of externally validated models and cannot be applied to any specific condition, test or setting. The prediction region, which covered specificities from about 43.9% to 99.1%, is more informative than the summary point: the specificity of a new externally validated model could be considerably lower. The large between-study variance of logit FPR and the high univariate I² for specificity reflect this heterogeneity, whereas sensitivity was more consistent. The summary estimates changed little in the sensitivity analyses, but with seven studies and one study per target condition, the analysis cannot identify sources of heterogeneity. A systematic review of externally validated CNN models for emergency head CT likewise considered meta-analysis inappropriate because of heterogeneity; scan-level sensitivity exceeded 90% in five of six studies, whereas specificity ranged from 58.0% to 97.7% [28], and a review of AI in digital pathology reported high summary accuracy together with substantial heterogeneity and frequent risk-of-bias concerns [23]. Two further issues may make the summary estimates optimistic: in two studies the operating threshold was selected in the validation data, and studies that report complete 2×2 data may differ systematically from those that report only AUC. The pooled estimate should therefore be regarded as exploratory.   AI Architectures and Imaging Modalities DL models, particularly CNN-based and transformer-based architectures, reported higher median AUC values than conventional ML models. CNNs learn hierarchical image features directly from the data instead of relying on handcrafted features, which may partly explain this pattern [6,7,13]. Examples in the recent literature include AlexNet-based classification of femoral bone tumors on MRI [48], a multimodal system for eye emergency triage [49], hybrid ResNet-50–ViT models for acute lymphoblastic leukemia [50], a combined support vector machine and VGGNet-16 model for lung cancer [51], U-Net-based segmentation of spinal MRI and knee CT [52,53], YOLO-based fracture detection on CT [54], radiomics and hybrid radiomics–DL pipelines [55,56], and DL and ensemble models for lung cancer, chronic kidney disease and endometrial cancer on CT and for lumbar and brain disorders on MRI [57–62]. However, the architecture groups in this review differed in target conditions, datasets and validation strategies, and no formal comparison was performed, so higher reported AUC values do not establish the superiority of DL over ML. Radiomics-based models have shown promising results, for example in hepatocellular carcinoma, but heterogeneity in methods and outcomes limits the transferability of such models between studies [63], and similar concerns regarding heterogeneity and applicability have been raised for CNN models in emergency head CT [28]. CT, MRI, radiography, ultrasound and whole-slide imaging accounted for most studies in every AUC category, which mainly reflects the number of studies per modality. Recent examples include CNN-based tumor classification on MRI [64,65], DenseNet-based and hybrid DL models for breast cancer [66,67], ensemble segmentation of chest radiographs [68], ultrasound applications in breast, thyroid, fetal and colorectal imaging [69–73] and fundus-based detection of retinal and ocular diseases [74–76]. In a direct comparison, a CNN model and pediatric radiologists did not differ significantly in the ultrasound diagnosis of biliary atresia [77]. The present data do not allow modality-specific conclusions; the quality of the dataset, annotation standards and validation rigor are likely to be more important determinants of reported performance than the modality itself [78].   Clinical Implementation, Ethical and Regulatory Considerations High discrimination in selected datasets does not establish clinical effectiveness, cost-effectiveness or readiness for routine implementation; these require prospective evaluation of patient outcomes, workflow effects and costs. In current practice, AI is used mainly as a supportive tool that reduces diagnostic workload and prioritizes high-risk cases [5,79]. Clinicians’ adoption decisions depend on transparency and on the evidence available for a given application [80], and over-reliance on AI may affect the training of young pathologists [26]. Ethical and legal issues include algorithmic bias, data privacy and security, patient consent and professional liability [81], and regulatory perspectives, including those of the United States Food and Drug Administration and the European Medicines Agency, emphasize independent evaluation and transparency in data collection and use [82,83]. Prospective validation studies [84], integration with picture archiving and communication systems and radiology information systems, user training and continuous monitoring of performance are prerequisites for safe clinical use [79].   Strengths and Limitations The strengths of this review include a search of five databases, independent data extraction and risk-of-bias assessment by two reviewers, an explicit operational definition of external validation, and stratification of the reported AUC values by validation strategy. The review has several limitations. First, the search did not use controlled vocabulary; the PubMed publication-type and free-full-text limits, the restriction of the Web of Science search to six journals and the omission of pathology terms from the Embase and Web of Science strings reduced sensitivity, so relevant studies were probably missed. Second, the restriction to open-access English-language articles may have introduced availability, publication and language bias, and the direction of this bias is uncertain. Third, the review was not prospectively registered, which increases the risk of post hoc decisions. Fourth, inclusion was based on reported AUC, the AUC categories were defined by the authors, and each study contributed one model; if the best-performing model was preferentially reported or extracted, the AUC distribution will be optimistic. Fifth, most studies were retrospective, risk of bias was frequently unclear for patient selection and for flow and timing, the low-risk judgments for the reference standard may be optimistic given the heterogeneity of reference standards, and applicability was assessed only for patient selection. Sixth, only seven studies were eligible for the meta-analysis; they addressed clinically unrelated conditions and used different units of analysis, including image-level data from patients who contributed several images, one external set came from the development institution and one was a temporal validation set, and in two studies the threshold was selected in the validation data, so the summary estimates are exploratory; the 2×2 data were re-extracted during revision, which corrected errors in the submitted analysis. Seventh, 11 records were based primarily on non-image data or had no recorded modality. Eighth, an AI-based screening tool contributed to the exclusion of 229 records at title and abstract screening, which may have led to the loss of eligible studies. Finally, studies published before 2021 were outside the date range.

Most recent studies of AI models for diagnosis in medical imaging and pathology report high AUC values, but this evidence is dominated by retrospective studies with unclear risk of bias. Reported AUC values were lower in studies with external validation only and in the few prospective RCTs than in studies with internal validation. Only seven of 170 studies provided 2×2 data from an external validation set; in these studies of clinically unrelated conditions, the exploratory summary sensitivity was 92.1% and specificity 90.1%, with a wide prediction region for specificity. High reported performance does not establish clinical effectiveness, generalizability, cost-effectiveness or readiness for routine implementation. Future studies should report complete 2×2 data at prespecified thresholds, evaluate models on external and prospectively collected data from consecutive patients, compare AI with clinicians on identical cohorts and follow AI-specific reporting guidance.   Supplementary Materials The following supporting information is provided. Supplementary File 1: Table S1: Search strategies, limits and numbers of records retrieved; Table S2: Characteristics of the 170 included studies; Table S3: Reported AUC by AI architecture; Table S4: QUADAS-2 risk of bias and applicability for each included study; Figure S1: Summary of QUADAS-2 risk of bias judgments; Figure S2: QUADAS-2 traffic-light plot; Table S5: 2×2 data of the studies considered for the meta-analysis with their sources in the primary publications; Table S6: Imaging modality by reported AUC category and AI architecture; Table S7: Imaging modality by target condition or task and reported AUC category; Table S8: Sensitivity analyses of the meta-analysis. Supplementary File 2: PRISMA-DTA checklist. Supplementary File 3: Study-level data (Microsoft Excel workbook). Supplementary File 4: R code for the meta-analysis.   Author Contributions Conceptualization, D.M.; methodology, D.M. and V.Y.; software, S.M.; validation, V.Y. and O.A.; formal analysis, D.M. and S.M.; investigation, D.M., H.F., S.M., V.Y. and O.A.; data curation, H.F. and S.M.; writing – original draft preparation, D.M.; writing – review and editing, D.M., H.F., S.M., V.Y. and O.A.; visualization, S.M.; supervision, D.M. and O.A.; project administration, D.M. All authors have read and approved the final version of the manuscript. Funding This research received no external funding.   Institutional Review Board Statement Not applicable. This systematic review analyzed data from previously published studies and did not involve human participants or identifiable personal information.   Informed Consent Statement Not applicable, because the review analyzed only data from previously published studies.   Data Availability Statement The study-level data extracted for this review, the QUADAS-2 judgments and the 2×2 data used in the meta-analysis are provided in Table 4 and Supplementary Files 1 and 3, and the analysis code is provided as Supplementary File 4. Acknowledgments The authors thank the investigators of the included studies whose published data made this review possible.   Conflicts of Interest The authors declare no conflicts of interest.