Intelligent Inverse Treatment Planning via Deep Reinforcement Learning, a Proof-of-Principle Study in High Dose-rate Brachytherapy for Cervical Cancer

Abstract

Inverse treatment planning in radiation therapy is formulated as optimization problems. The objective function and constraints consist of multiple terms designed for different clinical and practical considerations. Weighting factors of these terms are needed to define the optimization problem. While a treatment planning system can solve the optimization problem with given weights, adjusting the weights for high plan quality is performed by human. The weight tuning task is labor intensive, time consuming, and it critically affects the final plan quality. An automatic weight-tuning approach is strongly desired. The weight tuning procedure is essentially a decision making problem. Motivated by the tremendous success in deep learning for decision making with human-level intelligence, we propose a novel framework to tune the weights in a human-like manner. Using treatment planning in high-dose-rate brachytherapy as an example, we develop a weight tuning policy network (WTPN) that observes dose volume histograms of a plan and outputs an action to adjust organ weights, similar to the behaviors of a human planner. We train the WTPN via end-to-end deep reinforcement learning. Experience replay is performed with the epsilon greedy algorithm. Then we apply the trained WTPN to guide treatment planning of testing patient cases. The trained WTPN successfully learns the treatment planning goals to guide the weight tuning process. On average, the quality score of plans generated under the WTPN's guidance is improved by ~8.5% compared to the initial plan with arbitrary weights, and by 10.7% compared to the plans generated by human planners. To our knowledge, this is the first tool to adjust weights for the treatment planning in a human-like fashion based on learnt intelligence. The study demonstrates potential feasibility to develop intelligent treatment planning system via deep reinforcement learning.

References in corpus (11)

• Continuous control with deep reinforcement learning
• Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)
• Understanding deep learning requires rethinking generalization
• Three-Dimensional Radiotherapy Dose Prediction on Head and Neck Cancer Patients with a Hierarchically Densely Connected U-net Deep Learning Architecture
• Maximum Entropy Deep Inverse Reinforcement Learning
• Generating Synthesized Computed Tomography (CT) from Cone-Beam Computed Tomography (CBCT) using CycleGAN for Adaptive Radiation Therapy
• Fully Automated Organ Segmentation in Male Pelvic CT Images
• Super-Resolution 1H Magnetic Resonance Spectroscopic Imaging utilizing Deep Learning
• Accurate Real Time Localization Tracking in A Clinical Environment using Bluetooth Low Energy and Deep Learning
• A feasibility study for predicting optimal radiation therapy dose distributions of prostate cancer patients from patient anatomy using deep learning
• A Perspective on Deep Imaging

Cited by in corpus (9)

• Improving Efficiency of Training a Virtual Treatment Planner Network via Knowledge-guided Deep Reinforcement Learning for Intelligent Automatic Treatment Planning of Radiotherapy
• A preliminary study on a multi-resolution-level inverse planning algorithm for Gamma Knife radiosurgery
• Automatic Inverse Treatment Planning for Gamma Knife Radiosurgery via Deep Reinforcement Learning
• A hyperparameter-tuning approach to automated inverse planning
• The use of deep learning in interventional radiotherapy (brachytherapy): a review with a focus on open source and open data
• A Retrospective Study on the Investigation of Potential Clinical Benefits of Online Adaptive Proton Therapy for Head and Neck Cancer
• Direct optimization of dose-volume histogram metrics in radiation therapy treatment planning
• Patient level simulation and reinforcement learning to discover novel strategies for treating ovarian cancer
• An institutional study on plan quality and variation of manual forward planning for Gamma Knife radiosurgery for vestibular schwannoma