Prog Med Phys.  2020 Sep;31(S1):S39.

Real-time Patient Tracking and Reposition in Radiotherapy with TOF-based Depth Camera and CNN-based Auto Calibration

• Jeon C
• Jeong S
• Han Y

Abstract

Purpose
Immobilizing and monitoring a patient during radiation treatment are important elements of accurate radiation therapy. Although patient’s movement is restricted mostly using immobilization devices, patient’s stationariness needs to be observed by therapists. RGB cameras are used in clinic but visual measurement cannot verify patient’s small movement from reference position quantitatively. Therefore, a system of real-time patient position tracking with time-of-flightbased depth cameras was developed. Quantitative monitoring of patients was executed by comparing acquired volumetric data with reference position data to achieve accurate radiation therapy.
Methods
Three Azure Kinect (Kinect V4, Microsoft, USA) were used. For extrinsic calibration of cameras, convolutional neural networks (CNN) based auto calibration process was executed. The auto calibration program gets RGB images of each depth cameras, and calculates each camera’s calibration parameters automatically by object’s movement feature in RGB image. For visualization of tracking patients, Azure Kinect software development kit was used to construct 3D point cloud. Firstly, depth camera stream buffer data at a specific time (camera-calculated reference position data) were stored. Secondly, in order to compute patient’s position difference from the reference position, continuous buffer data were compared with stored reference position data and colorization of each pixel of 3D point cloud was differentiated from reference pixel color if moved position differs larger than 3 mm. The developed process was verified and accuracy was measured by comparing with point data measured with an electronic laser distance measuring equipment (Disto D810, Leica Geosystems, Switzerland) and with CT position data after matching the camera coordinates and CT coordinates for a human body phantom.
Results
The visualization was successfully constructed, and the patient movements were easily recognizable. A series of processes described above were successfully performed. Position discrepancies at several body points were calculated by laser distance measuring equipment and the difference from depth camera was under 3 mm. Comparison results of CT data will be presented.
Conclusions
With proposed method, accurate real-time patient tracking and position correction in radiation treatment time can be achieved.

Keyword

Real-time patient tracking; Patient Setup; Depth Camera