CONSTRAINED DEEP REINFORCEMENT LEARNING FOR ROBOT-ASSISTED SURGICAL TRAINING WITH SOFT TISSUE DEFORMATION
ZENG WEI
ZENG WEI
Citations
Altmetric:
Alternative Title
Abstract
In this thesis, we address the challenges in laparoscopic surgery training by leveraging Deep Reinforcement Learning (DRL). We develop a simulation environment using the SOFA framework and SofaPython3 plugin, allowing seamless integration with Python-based machine learning algorithms. Our Gym environment interfaces with standard RL algorithms for training and evaluation in a simulated surgical environment. We present an uncertainty-aware Model-Based Greedy Search (MBGS) algorithm for constrained RL, promoting safety during DRL training. The laparoscopic surgery task is specified as a target-reaching task, guiding the robot to move the end effector to a given target point in/around the liver while stabilizing environment parameters. The ensemble of uncertainty-aware networks predicts parameters, evaluating multiple actions to select the optimal one. Experimental results show effective parameter stabilization and rapid learning. The algorithm's efficiency and compatibility with other off-policy algorithms make it a promising tool for high-safety scenarios.
Keywords
Laparoscopic surgery training, constrained deep reinforcement learning, soft-tissue simulation, SOFA framework, model-based greedy search
Source Title
Publisher
Series/Report No.
Collections
Rights
Date
2023-05-17
DOI
Type
Thesis