2042 - Clinical Evaluation of AI-Generated Contours for Pediatric Proton CSI: A Blinded Expert Review

Purpose/Objective(s): Craniospinal irradiation (CSI) requires precise contouring of target structures. Commercial auto-segmentation tools often fail to capture critical structures such as the optic nerves and nerve roots, necessitating time-intensive manual refinement. This study evaluated an AI-driven segmentation model for pediatric proton CSI through a blinded expert review, assessing segmentation accuracy, consistency, and efficiency while exploring AI’s role in improving workflow feasibility and reducing inter-observer variability.

Materials/Methods: Segmentation data from 19 pediatric CSI patients were used to train an AI model to segment the thecal sac and brain, which was then applied to a separate 21-patient cohort. A blinded expert review was conducted, where 4 radiation oncologists evaluated AI-generated and approved, manual contours. Each contour set was reviewed at least twice. Presentation was randomized to minimize bias, and reviewers were blinded to the segmentation method. A structured rubric assessed quality across 8 domains using a 5-point scale. Scores were averaged per patient, and paired t-tests were performed to compare AI vs. manual scores. Reviewers also provided qualitative feedback on segmentation quality.

Results: Among 21 matched pairs, AI-generated contours demonstrated significantly higher scores than manual contours in spinal canal coverage (4.7 ± 0.4 vs. 4.1 ± 1.0, p=0.0075), nerve roots coverage (4.7 ± 0.4 vs. 4.0 ± 1.0, p = 0.0021), smoothness (4.6 ± 0.5 vs. 3.7 ± 1.3, p = 0.024), & overall acceptability (4.1 ± 0.4 vs. 3.5 ± 1.1, p = 0.015). AI also achieved numerically higher scores in other scoring categories, though these differences were not statistically significant. Reviewer feedback highlighted AI contours as smoother, requiring fewer refinements, while manual contours exhibited greater inter-observer variability, specifically in spinal canal and nerve roots. The model reduced delineation time to 4 mins per case compared to 23 mins for manual contouring on average, a 5 fold difference.

Conclusion: This blinded review indicates that AI-driven segmentation improves consistency and accuracy in key regions while significantly reducing contouring time. AI-assisted contouring minimizes inter-observer variability, enhances standardization, and workflow efficiency in radiotherapy planning while maintaining necessary physician oversight. These findings support broader clinical adoption of AI to optimize segmentation accuracy and efficiency in pediatric proton CSI. Future research should explore AI’s integration into radiotherapy workflows and its potential for automating treatment planning.