356 - Feasibility Study of AI-Assisted Radiation Therapy Contour QA for Prostate Cancer Multi-Center Clinical Trial
Presenter(s)
L. Wang1, H. Geng2, A. O. Sartor3, Z. Chen4, R. L. Ruo5, J. B. Yu6, S. C. Morgan7, K. E. Hoffman8, P. L. Nguyen9, and Y. Xiao10; 1University of Pennsylvania, Philadelphia, PA, 2Department of Radiation Oncology, University of Pennsylvania, Philadelphia, PA, 3Mayo Clinic, Rochester, MN, 4Department of Therapeutic Radiology, Yale University School of Medicine, New Haven, CT, 5McGill University Health Centre, Montreal, QC, Canada, 6Department of Radiation Oncology and Applied Sciences, Dartmouth Geisel School of Medicine, Lebanon, NH, 7The Ottawa Hospital Cancer Centre, Ottawa, ON, Canada, 8Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, 9Department of Radiation Oncology, Brigham and Women’s Hospital/Dana-Farber Cancer Institute, Boston, MA, 10Department of Radiation Oncology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
Purpose/Objective(s): Quality assurance (QA) of radiotherapy contours is a critical component in ensuring the reliability and reproducibility of clinical trial outcomes. The absence of well-defined, quantifiable review criteria renders the evaluation process for contours relatively subjective. In this study, we investigated the potential of Artificial Intelligence (AI) assisted contour QA, aiming to establish an objective procedure to identify all submitted contours that exhibit potential quality concerns.
Materials/Methods: Total of 109 patient datasets submitted to NRG-GU009 were reviewed by four principal investigators (PI) and categorized into three scores: Score 1 – per protocol, Score 2 – variation acceptable, Score 3 – deviation unacceptable.
AI-generated contours were compared against submitted contours using four contour evaluation indices. Expert-assigned score 3 were considered true positives for the AI workflow to identify. Sensitivity analysis was conducted to assess AI’s ability to accurately detect these true positive cases. Physician feedback was then incorporated to the AI workflow, focusing on key factors such as the overlap of CTVs with critical structures, maximum Hausdorff distance for identify over-contoured or under-contoured critical structures such as rectum and sigmoid. Statistical thresholds were established to quantify potential violations. Sensitivity metrics were compared for before and after updating the workflow to evaluate improvements in identifying true positive cases.Results: Incorporating PI feedback into the workflow has led to the development of more precise contour QA criteria. Key updates include specific thresholds for overlapping clinical target volumes (CTVs), with a 1cc threshold for both the rectum and sigmoid. Additional criteria were established to address over- and under-contoured structures, including a Hausdorff Distance limit of 3 cm for over-contoured rectum.
A refined set of quantitative thresholds incorporating these new criteria successfully identified all unacceptable contour errors in prostate CTVs as flagged by the PIs. Additionally, the AI system detected three under-contoured and four over-contoured prostate CTVs that may require further human review. Furthermore, the AI identified 70% of nodal CTV volumes that deviated from contouring guidelines. One contour with a severity score of 3 was identified by the PI due to overlap with sigmoid tissue. Applying the same overlap criteria, the AI detected five nodal CTV volumes that were not initially identified by the PI.Conclusion: Standardizing contour QA criteria has the potential to enhance QA efficiency while minimizing inter-reviewer variability in multi-center clinical trials. Future efforts will aim to validate these findings across larger datasets to further optimize the role of AI in contour QA workflows.
Funding: This project was supported by grants U10CA180868 (NRG Oncology Operations), U24CA180803 (IROC), CTEP from the National Cancer Institute (NCI).