2002 - AI Agent Empowered Interactive Platform for Patient-Specific Dynamic Contour and Plan Modification and Quality Assurance

Purpose/Objective(s): Automated scripts and workflows, such as MIM workflows, have been implemented in clinical settings to streamline the planning process, improving efficiency and consistency. Standardized MIM workflows are widely used in our clinic for contour generation, quality assurance, plan preparation, and adaptation. However, these predefined workflows are inherently rigid, designed for general use, and require significant effort to modify for non-standard cases, making them impractical for personalized MR-guided online adaptive radiotherapy (MR-oART). To overcome this limitation, we integrated a Large Language Model (LLM) agent into the commercially available software environment, enabling users to perform dynamic, interactive modifications and verifications through natural language commands.

Materials/Methods: An AI-powered system (Google Gemini 2.0 and LangChain) was integrated into MIM to allow users to perform customized contour and plan manipulation and QA operations through natural language queries. A dynamic context window (CW), populated from a curated knowledge base of code snippets, SDK manuals, instructions, and DICOM RT dictionaries was used to provide the LLM with essential information. An orchestrating program implemented a multi-step contextualization pipeline with 2 LLM calls; 1) identifying relevant keyword from input queries; 2) sending detailed CW to the LLM for code generation to implement the query. The agent's functionality was built and expanded by queries generated by four medical physicists and augmented by an LLM.

Results: Developed package successfully replicated standard MIM workflows for plan adaptation and contour checks while offering significantly greater adaptability for customized parameters. Approach of providing relevant CW eliminated hallucinations. Errors occurred only in cases of insufficient context, but once updated, system consistently produced accurate results. Extensive range of modification and QA actions were implemented, offering unbound combinability and modifiability via natural language. See Table 1.

Conclusion: This novel LLM-powered tool introduces a new paradigm for verification and adjustments in radiation oncology, enabling adaptive, user-driven modifications that were previously impractical. By integrating interactive AI agent, our approach bridges the gap between predefined automation and flexible, on-the-spot adaptability—reducing workflow rigidity, enhancing efficiency, and expanding the potential of oART.