Endo-Histo Foundational Fusion Model: A Novel Artificial Intelligence Approach for Predicting Histologic Remission and Early Response to Therapy in a Phase 2 Ulcerative Colitis Clinical Trial

Aims Artificial Intelligence (AI)- enabled endoscopy and histology offer accurate, objective, and rapid assessment of disease activity in Ulcerative Colitis (UC). Emerging multi-source AI models integrating diverse datasets may enhance standardised disease evaluation and outcome prediction. This study aimed to develop a novel AI model fusing endoscopic and histological findings to improve the assessment of disease remission and predict early response to therapy in UC clinical trials.

Methods A novel multimodal AI fusion algorithm was developed by integrating paired endoscopic videos and histological whole-slide images (WSIs) from the phase 2 clinical trial of Mirikizumab in UC (NCT02589665). The endoscopy branch of the model was trained with 291 white-light videos, using a convolutional neural network to select informative frames and the BioMedCLIP foundational model to extract features. The histology branch utilised the CONCH foundational model and was fine-tuned on 291 WSIs to obtain patch-level features. Features from both modalities were aggregated and fused using multi-head self-attention. The model’s ability to assess histological remission and response to therapy at weeks 12 and 52 was evaluated. Histological remission was defined as Geboes≤2B.0, while a response to therapy at different time points was based on histological remission or improvement (Geboes<3.1).

Results The fusion model outperformed single-modality assessments for histological remission, achieving a sensitivity of 89.72% (95% CI: 82.35–94.76), specificity of 89.67% (95% CI: 84.34–93.67), and accuracy of 89.69% (95% CI: 85.61–92.94). It demonstrated remarkable performance in assessing response to therapy at 12 and 52 weeks, with sensitivity of 97.96% (95% CI: 89.15–99.95), specificity of 86.84% (95% CI: 71.91–95.59) and accuracy of 93.10% (95% CI 85.59 – 97.43) for histological remission at week 52. Substantial agreement was observed between the AI fusion model and central readout.

Conclusions This innovative multimodal fusion AI model enhances the assessment of histological remission and accurately predicts response to therapy. By potentially standardising central readouts and enabling automated disease assessment, this novel tool marks a significant advancement towards precision medicine in clinical trials.

Publikationsverlauf

Artikel online veröffentlicht:
27. März 2025