Commissioning and Safe Use in the Clinic

Finally, commissioning any auto-segmentation method should go through multiple phases starting from identifying the appropriate method to ongoing evaluation and periodic testing. A typical commissioning procedure could use a two-phase approach, where the first phase involves training/ validation and identification of a suitable method and the second phase involves testing and verification for use in the clinic [30]. However, the algorithm development and training phase itself does not need to be done in the commissioning institution as long as the vendor providing the software can provide relevant details for evaluating the algorithm with the internal datasets.

Training and Validation Phase

The training/validation phase should consider evaluating a reasonable number of different methods, employ multiple metrics, including dosimetric metrics [35], and if possible evaluate the methods with open-source datasets to identify the best method and the general conditions in which the method works.

Testing and Verification Phase

In the testing/verification phase, the method should be evaluated with an internally curated dataset. In this phase, prior to clinical implementation, it is useful to have a multidisciplinary team such as the algorithm developers, physicists, radiation oncologists, etc. who can review the method’s performance under variable scenarios that are reasonably encountered in the clinic, and understand its performance and limitations.

In Memorial Sloan Kettering Cancer Center, a similar training/testing schedule as proposed in Vandewinckele et al. [30] for clinical commissioning was employed. In addition to the training/ validation and the pre-commissioning testing, a feedback-oriented testing and developmental cycle for clinical commissioning of the deep learning segmentation methods was used. A team of multidisciplinary experts including computer scientists and developers involved in the algorithm development, physicists, a radiation oncologist (for the specific disease site), and an anatomy contourist meets on a weekly basis to visually review cases from the internal clinical datasets. Cases for review are selected from the internally curated dataset, as well as new or abnormal cases identified by the radiation oncologist. Any problems encountered with the algorithm are discussed, and if necessary the algorithms are improved or retrained and reviewed the following week. This feedback-oriented development cycle continues until the team is satisfied with the algorithm’s performance and the algorithm is applied to a set of new incoming clinical cases for the following month and verified in a blinded experiment by the anatomist, physicists, and radiation oncologist. If no issues are encountered in the new cases, the auto-segmentation method is commissioned for clinical use. The autosegmentation methods are periodically reviewed (every three months) and improved as necessary.

Any improvements are further reviewed before recommissioning of the improved method. This method has worked for the institution for two different deep learning auto-segmentation methods currently in use in the clinic for prostate and head and neck cancer treatments.


Some of the issues related to clinical commissioning and data curation of auto-segmentation methods for radiation therapy treatments were reviewed. These issues are explored further in the next two chapters. Some guidelines were presented, and suggestions were proposed that could be implemented for these aforementioned two issues and help with better and more efficient commissioning mechanisms in different institutions.

< Prev   CONTENTS   Source   Next >