DEEP REINFORCEMENT LEARNING AND ITS APPLICATION FOR FLEXIBLE NEEDLE INSERTION

Abstract

Flexible needle insertion with bevel tips is becoming a preferred method for approaching targets in the human body in the least invasive manner. However, to successfully implement needle insertion, surgeons require prolonged training processes and long-term experience to develop essential handling skills. This thesis presents a new path planning approach with Deep Reinforcement Learning (DRL) to implement automatic needle insertion using a surgical robot. In this thesis, Deep Q-Network (DQN) algorithm and Deep Deterministic Policy Gradient (DDPG) algorithm are utilized to learn the control policy for flexible needle steering with needle-tissue interaction. Our proposed algorithms are effective and time efficient. This study shown that deep reinforcement learning can be used to develop accurate path planning algorithm in a complex environment.