Epithelial ovarian cancer (EOC) has a high case-fatality rate, largely due to diagnosis at advanced stages and lack of effective screening tools. Conventional screening tools, like CA125 and ultrasound, have limited sensitivity and specificity. Advances in deep learning (DL) applied to medical imaging such as CT scans offer a promising avenue for improving early EOC detection.

To build, develop, and validate prediction models of EOC using deep machine learning (DL) to process and analyze abdominal-pelvic CT scans.

We performed a pilot case-control study to predict EOC using artificial intelligence (AI) methodology and abdominal-pelvic CT scans comparing EOC patients (cases, N = 355) to patients with histology-proven benign adnexal masses (controls, N = 213). CT images were converted to 3D NIFTI format and segmented using the ovseg pipeline. Multiple DL architectures were evaluated, including convolutional neural networks (CNNs) and vision transformers (ViTs). Model performance was assessed using area under the operating characteristic curve (AUC), accuracy, and precision. Model interpretability was evaluated using Gradient-weighted Class Activation Mapping (Grad-CAM). Radiomics-based models were constructed using PyRadiomics features and compared with DL models.

Most of women had and advanced FIGO stage, 88%, and high grade histology, 92%. Exploratory analysis of different methods without segmentation (or masking) resulted in poor performances, with AUCs between 0.39 and 0.83. The best performing models were those trained with CNN architecture with masked images, with AUC of 0.92. Validation and testing performances for EOC prediction with PyRadiomics had AUCs of 0.74 and 0.66 respectively.

DL using segmented abdominal-pelvic CT scans, particularly CNN-based architecture, demonstrate strong potential for distinguishing EOC from benign pelvic masses. Further studies are needed to create accurate DL models for early EOC detection.