ICBME 2019 Special Symposium in Surgical Robotics

Themes

  • Computer-Assisted Surgery
  • Flexible Robotics and navigation in Surgery
  • Artificial Intelligence in Robotic Surgery

Date: Dec 11, 2019, EA

Speakers and Topics

1600 – 1615 Robotic Intervention Utilizing Bioengineering Based Therapeutic Methods
Ichiro Sakuma University of Tokyo
1615 – 1630 Augmented Reality for Orthopeadic Surgery
Jaesung Hong Daegu Gyeongbuk Institute of Science and Technology
1630 – 1645
Medical Robot Link Architecture Connected to Smart Cyber Operating Theater (SCOT)
Ken Masamune Tokyo Women’s Medical University
1645 – 1700
Transluminal Robotics with Delicate Continuum and Context Awareness
Hongliang Ren National University of Singapore
1700 – 1715
Robot-Assisted Interventions under Intra-Operative MRI-Based Guidance
Ka-Wai Kwok University of Hong Kong
1715 – 1730
Biomimetic Wrinkled MXene Pressure Sensors towards Collision-Aware Robots
Catherine Cai National University of Singapore


 

SAKUMA, Ichiro, University of Tokyo

  • Robotic Intervention Utilizing Bioengineering Based Therapeutic Methods
  • Biography
    Ichiro Sakuma received B.S., M.S., and Ph.D. degrees in precision machinery engineering from the University of Tokyo, Tokyo, Japan, in 1982, 1984, and 1989, respectively. He was Research Associate (1987), Associate Professor (1991) at the Department of Applied Electronic Engineering, Tokyo Denki University, Saitama, Japan. He was research instructor at Department of Surgery, Baylor College of Medicine, Houston, Texas from 1990 to 1991. He was Associate Professor at Department of Precision Engineering (1998), Associate Professor (1999) and Professor (2001), Graduate School of Frontier Sciences, the University of Tokyo. He is currently a Professor at Department of Bioengineering, Department of Precision Engineering, Director of Medical Device Development and Regulation Research Center, and Vice dean, School of Engineering, the University of Tokyo. He is the immediate past president of Japanese Society for Medical and Biological Engineering (JSMBE) (2014-2016). He is also Deputy Director for Medical Devices, Center for Product Evaluation, Pharmaceuticals and Medical Devices Agency (PMDA)His research interests includes 1) Computer Aided Surgery, 2) Medical Robotics and medical devise for minimally invasive therapy, 3) Analysis of cardiac arrhythmia phenomena and control of arrhythmia, and 4) Regulatory sciences for medical device development.He received various academic awards including, The Japan Society of Computer Aided Surgery, Best Paper Award (2006), Robotic Society of Japan, Best Paper Award (2010, 2015). In 2014, his group’s research was selected in 2014 as one of the most exciting peer-reviewed optics research to have emerged over the past 12 months by Photonics and Optics News (OSA)

 

Ken Masamune, Tokyo Women’s Medical University

  • Medical Robot Link Architecture Connected to Smart Cyber Operating Theater (SCOT)
  • Abstract
    Nowadays, several medical devices/systems including imaging machine, anesthesia, navigation system, biomonitoring devices, surgical bed, medical robots, et al., are installed in the operation room, however, it is unpleasant situation that all devices are performed in stand-alone mode, without time-synchronization, and it is difficult to combine/analyze some set of information from devices to make surgeon’s decision during surgery. To improve this situation, we’ve been developing an integrated operating room named “Smart Cyber Operating Theater (SCOT) with middleware ORiN system. In this presentation, we introduce a current SCOT project and the design concept of new open platform architecture for the integration of master/slave robotic devices and information guided robot especially for oral and maxillofacial surgery. This design will accelerate the development of any types of robotic interfaces/end effectors with fast validation.
  • Biography
    Ken Masamune received the Ph.D. degree in precision machinery engineering from the University of Tokyo, Japan, in 1999. From 1995 to 1999, he was a Research Associate in the Department of Precision Machinery Engineering, the University of Tokyo. From 2000 to 2004, he was an Assistant Professor in the Department of Biotechnology, Tokyo Denki University, Tokyo. Since 2005, he has been an Associate Professor in the Department of Mechanoinformatics, Graduate School of Information Science and Technology, the University of Tokyo. His current research interests include computer-aided surgery, especially medical robotics and visualization devices and systems for surgery.

 

Jaesung Hong, Daegu Gyeongbuk Institute of Science and Technology

  • AUGMENTED REALITY FOR SURGICAL NAVIGATION
  • Abstract
    In these days, augmented reality (AR) has become a key technology for surgical navigation. Using the AR technology, the shape of invisible organs are overlapped to the visible endoscopic or microscopic images. Therefore the surgeon can avoid damaging the healthy tissue, and reduce the incision area. In the AR-based surgery, optical tracker and camera are generally used. Optical tracker can measure the position and pose of multiple markers, and the relationship between the camera and target organs of patient can be measured in real-time by tracking of the markers which are mounted on the camera body and the patient. In the AR display, finding the relationship between the optical marker mounted on the camera body and the center of camera (camera registration) is particulary important. This relationship strongly affects the overall accuracy of AR display. In this talk, the latest AR technologies applied for the surgical navigation are introduced.
  • Biography
    Jaesung Hong is an associate professor and the Department Chair of Robotics Engineering at the Daegu Gyeongbuk Institute of Science and Technology (DGIST), South Korea. His research area is medical imaging and medical robotics for minimally invasive surgery.
    At the University of Tokyo, he has developed the world first US-guided needle insertion robot tracking a movable and deformable organ. This was reported in Physics in Medicine and Biololgy in 2004, and has been frequently cited (> 160). While he worked at Kyushu University Hospital in Japan, he developed various customized surgical navigation systems, which were clinically applied in approximately 120 surgeries. These included percutaneous ablation therapies for liver tumors, cochlear implant surgeries, neurosurgeries for gliomas, and dental implant surgeries.
    After moving to DGIST which is a research-oriented special university supported by Korean government, he developed a single port surgery robot and its master device for high force transimisson and large workspace as well as a portable, AR-based surgical navigation system, which has been tested in tibia tumor resections and orthognathic surgeries in collaboration with major Korean hospitals, including the Seoul National University Hospital of Bundang, Samsung Seoul Hospital, etc. He is one of a small number of specialists who is familiar with both engineering and clinical medicine.
    Until 2016, Prof. Hong has published approximately 42 journal papers including 30 SCI/SCIE papers with impact factors. Six of them are top 10% ranked journal papers. He also submitted or registered 15 domestic and 7 international patents. He has received 9 best paper/presentation awards, in addition to obtaining 10 research funds amounting to approximately 4.5M USD (including planned budgets).

 

Ka-Wai Kwok, University of Hong Kong

  • Robot-Assisted Interventions under Intra-Operative MRI-Based Guidance
  • Abstract
    Advanced surgical robotics has attracted significant research interest in supporting image guidance, even magnetic resonance imaging (MRI) for effective navigation of surgical instruments. In situ effective guidance of access routes to the target anatomy, rendered based on imaging data, can enable a distinct awareness of the position of robotic instrument tip relative to the target anatomy in various types of minimally invasive interventions. Therefore, such MRI-guided robots will rely on real-time processing the co-registration of surgical plan with the imaging data captured during the intervention, as well as computing the relative configuration between the instrument and the anatomy of surgical interest.
    This talk will present a compact robotic system capable to operate inside the bore of MRI scanner, as well as its solutions to technical challenges of providing a safe, effective catheter-based surgical manipulation. The proposed image processing system demonstrates its clinical potential of enhanced surgical safety by imposing visual feedback on tele-operated robotic instruments even under large-scale and rapid tissue deformations in soft tissue surgeries, such as cardiac electrophysiology and stereotactic neurosurgery. The ultimate research objective is to enable the operator to perform safe, precise and effective control of robotics instruments with the aid of pre- and intra-operative MRI models. The present work will be timely to bridge the current technical gap between MRI and surgical robotic control.
  • Biography
    Dr. Ka-Wai Kwok is currently assistant professor in Department of Mechanical Engineering, The University of Hong Kong, who completed his PhD training in The Hamlyn Centre for Robotic Surgery, Imperial College London in 2011, where he continued research on surgical robotics as a postdoctoral fellow. After then, Dr. Kwok obtained the Croucher Foundation Fellowship 2013-14, which supported his research jointly hosted by The University of Georgia, and Brigham and Women’s Hospital – Harvard Medical School. His research interests focus on surgical robotics, intra-operative medical image processing, and their uses of high-performance computing techniques. To date, he has been involved in various designs of surgical robotic devices and interfaces for endoscopy, laparoscopy, stereotactic and intra-cardiac catheter interventions. His work has also been recognized by several awards from IEEE international conferences, including ICRA’14, IROS’13 and FCCM’11. He also became the recipient of Early Career Awards 2015/16 offered by Research Grants Council (RGC) of Hong Kong.

 

Hongliang Ren, National University of Singapore, Singapore

  • Transluminal Robotics with Delicate Continuum and Context Awareness
  • Biography
    Dr. Hongliang Ren is currently an assistant professor and leading a research group on medical mechatronics in the Biomedical Engineering Department of National University of Singapore (NUS). He is an affiliated Principal Investigator for the Singapore Institute of Neurotechnology (SINAPSE) and Advanced Robotics Center at National University of Singapore. Dr. Ren received his PhD in Electronic Engineering (Specialized in Biomedical Engineering) from The Chinese University of Hong Kong (CUHK) in 2008. After his graduation, he worked as a Research Fellow in the Laboratory for Computational Sensing and Robotics (LCSR) and the Engineering Center for Computer-Integrated Surgical Systems and Technology (ERC-CISST), Department of Biomedical Engineering and Department of Computer Science, The Johns Hopkins University, Baltimore, MD, USA, from 2008 to 2010. In 2010, he joined the Pediatric Cardiac Biorobotics Lab, Department of Cardiovascular Surgery, Children’s Hospital Boston & Harvard Medical School, USA, for investigating the beating heart robotic surgery system. Prior to joining NUS, he also worked in 2012 on a collaborative computer integrated surgery project, at the Surgical Innovation Institute of Children’s National Medical Center, USA. His main areas of interest include Biomedical Mechatronics, Computer-Integrated Surgery, and Dynamic Positioning in Medicine.

 

Catherine Cai, National University of Singapore, Singapore

  • Biomimetic Wrinkled MXene Pressure Sensors towards Collision-Aware Robots
  • Abstract
    The use of surgical robots in the field of minimally invasive neurosurgical procedures can offer several benefits and advantages. However, the lack of force sensing hinders and limits their use in such procedures. Equipping surgical robots with pressure sensors can enhance robot-environment interaction by enabling collision awareness and enhance human-robot interactions by providing surgeons the necessary force feedback for safe tissue manipulation. With the emergence of soft robotics in biomedical applications, the attached pressure sensors are required to be flexible and stretchable in order to comply with the mechanically dynamic robotic movements and deformations. Inspired by the multi-dimensional wrinkles of Shar-Pei dog’s skin, we have fabricated a flexible and stretchable piezoresistive pressure sensor consisting of MXene electrodes with biomimetic topographies. This pressure sensor is found to be more sensitive in low-pressure regimes (0.934 kPa-1, <236 Pa), and less sensitive in the higher pressure regimes (0.188 kPa-1, <2070 Pa).

ICBME 2016 Special Symposium in surgical navigation and robotics

Themes

  • Computer-Assisted Surgery
  • Flexible Robotics and navigation in Surgery
  • Artificial Intelligence in Robotic Surgery

Date: Dec 7, 2016 U-Town

Gallery

https://goo.gl/photos/vCNqYfWmD26UF41j7
   
   

Speakers and Topics


sakuma

Keynote speaker: SAKUMA, Ichiro, University of Tokyo

  • COMPUTER AIDED SURGERY FOR ASSISTING MINIMALLY INVASIVE THERAPIES
  • Abstract
    Minimally invasive therapy such as endoscopic surgery and catheter based intervention are being spread in many surgical intervention fields. Thus engineering assistance is important to realize safe and effective minimally invasive therapy. Computer Assisted Surgical guidance such as surgical navigation is one of key technologies. It is expected that application of robotic technology to minimally invasive surgery will provide the following functions:
    (1) Precise manipulation of biological tissues and surgical instruments in narrow and confined surgical field.
    (2) Precise and accurate localization of therapeutic devices using various pre and intra-operative medical information.In the first mode of application, more compact system is required. It can be realized by introduction of novel mechanical design and application off a new mechanism as well as new materials. At the same time integration with various energy devices are also required. Intra-operative guidance utilizing various pre and intra operative information is necessary in the second mode of application. Image guided robotic system for RF ablation, laser ablation, intensity modified radiation therapy, and high intensity focused ultrasound. In this type of robot, various preand intraoperative information including functional information is used to navigate the therapeutic devices to the target lesion. Intra-operative identification of pathological state of the target tissue and evaluation of outcome after therapeutic intervention are also important.Factors limiting the application of surgical navigation systems and medical robotics include limited usability requiring additional procedures for preparation, and high costs. Recent progress of computer vision technologies will solve part of these issues.For wider spread of these technologies in clinical environment, further improvement of usability, cost reduction, and accumulation of clinical evidences demonstrating efficacy from both clinical and economical point of view are required.
  • Biography
    Ichiro Sakuma received B.S., M.S., and Ph.D. degrees in precision machinery engineering from the University of Tokyo, Tokyo, Japan, in 1982, 1984, and 1989, respectively. He was Research Associate (1987), Associate Professor (1991) at the Department of Applied Electronic Engineering, Tokyo Denki University, Saitama, Japan. He was research instructor at Department of Surgery, Baylor College of Medicine, Houston, Texas from 1990 to 1991. He was Associate Professor at Department of Precision Engineering (1998), Associate Professor (1999) and Professor (2001), Graduate School of Frontier Sciences, the University of Tokyo. He is currently a Professor at Department of Bioengineering, Department of Precision Engineering, Director of Medical Device Development and Regulation Research Center, and Vice dean, School of Engineering, the University of Tokyo. He is the immediate past president of Japanese Society for Medical and Biological Engineering (JSMBE) (2014-2016). He is also Deputy Director for Medical Devices, Center for Product Evaluation, Pharmaceuticals and Medical Devices Agency (PMDA)His research interests includes 1) Computer Aided Surgery, 2) Medical Robotics and medical devise for minimally invasive therapy, 3) Analysis of cardiac arrhythmia phenomena and control of arrhythmia, and 4) Regulatory sciences for medical device development.He received various academic awards including, The Japan Society of Computer Aided Surgery, Best Paper Award (2006), Robotic Society of Japan, Best Paper Award (2010, 2015). In 2014, his group’s research was selected in 2014 as one of the most exciting peer-reviewed optics research to have emerged over the past 12 months by Photonics and Optics News (OSA)

masamuneken

Ken Masamune, Tokyo Women’s Medical University

  • OPEN PLATFORM OF MEDICAL ROBOTS/ DEVICES WITH SMART CYBER OPERATING THEATER (SCOT): DESIGN CONCEPT AND PROTOTYPE ROBOT DEVELOPMENTS
  • Abstract
    Nowadays, several medical devices/systems including imaging machine, anesthesia, navigation system, biomonitoring devices, surgical bed, medical robots, et al., are installed in the operation room, however, it is unpleasant situation that all devices are performed in stand-alone mode, without time-synchronization, and it is difficult to combine/analyze some set of information from devices to make surgeon’s decision during surgery. To improve this situation, we’ve been developing an integrated operating room named “Smart Cyber Operating Theater (SCOT) with middleware ORiN system. In this presentation, we introduce a current SCOT project and the design concept of new open platform architecture for the integration of master/slave robotic devices and information guided robot especially for oral and maxillofacial surgery. This design will accelerate the development of any types of robotic interfaces/end effectors with fast validation.
  • Biography
    Ken Masamune received the Ph.D. degree in precision machinery engineering from the University of Tokyo, Japan, in 1999. From 1995 to 1999, he was a Research Associate in the Department of Precision Machinery Engineering, the University of Tokyo. From 2000 to 2004, he was an Assistant Professor in the Department of Biotechnology, Tokyo Denki University, Tokyo. Since 2005, he has been an Associate Professor in the Department of Mechanoinformatics, Graduate School of Information Science and Technology, the University of Tokyo. His current research interests include computer-aided surgery, especially medical robotics and visualization devices and systems for surgery.

jshongdgist

Jaesung Hong, Daegu Gyeongbuk Institute of Science and Technology

  • AUGMENTED REALITY FOR SURGICAL NAVIGATION
  • Abstract
    In these days, augmented reality (AR) has become a key technology for surgical navigation. Using the AR technology, the shape of invisible organs are overlapped to the visible endoscopic or microscopic images. Therefore the surgeon can avoid damaging the healthy tissue, and reduce the incision area. In the AR-based surgery, optical tracker and camera are generally used. Optical tracker can measure the position and pose of multiple markers, and the relationship between the camera and target organs of patient can be measured in real-time by tracking of the markers which are mounted on the camera body and the patient. In the AR display, finding the relationship between the optical marker mounted on the camera body and the center of camera (camera registration) is particulary important. This relationship strongly affects the overall accuracy of AR display. In this talk, the latest AR technologies applied for the surgical navigation are introduced.
  • Biography
    Jaesung Hong is an associate professor and the Department Chair of Robotics Engineering at the Daegu Gyeongbuk Institute of Science and Technology (DGIST), South Korea. His research area is medical imaging and medical robotics for minimally invasive surgery.
    At the University of Tokyo, he has developed the world first US-guided needle insertion robot tracking a movable and deformable organ. This was reported in Physics in Medicine and Biololgy in 2004, and has been frequently cited (> 160). While he worked at Kyushu University Hospital in Japan, he developed various customized surgical navigation systems, which were clinically applied in approximately 120 surgeries. These included percutaneous ablation therapies for liver tumors, cochlear implant surgeries, neurosurgeries for gliomas, and dental implant surgeries.
    After moving to DGIST which is a research-oriented special university supported by Korean government, he developed a single port surgery robot and its master device for high force transimisson and large workspace as well as a portable, AR-based surgical navigation system, which has been tested in tibia tumor resections and orthognathic surgeries in collaboration with major Korean hospitals, including the Seoul National University Hospital of Bundang, Samsung Seoul Hospital, etc. He is one of a small number of specialists who is familiar with both engineering and clinical medicine.
    Until 2016, Prof. Hong has published approximately 42 journal papers including 30 SCI/SCIE papers with impact factors. Six of them are top 10% ranked journal papers. He also submitted or registered 15 domestic and 7 international patents. He has received 9 best paper/presentation awards, in addition to obtaining 10 research funds amounting to approximately 4.5M USD (including planned budgets).

KwokKaWai

Ka-Wai Kwok, University of Hong Kong

  • MR-COMPATIBLE ROBOTIC SYSTEMS: TOWARDS THE INTRAOPERATIVE MRI-GUIDED INTERVENTIONS
  • Abstract
    Advanced surgical robotics has attracted significant research interest in supporting image guidance, even magnetic resonance imaging (MRI) for effective navigation of surgical instruments. In situ effective guidance of access routes to the target anatomy, rendered based on imaging data, can enable a distinct awareness of the position of robotic instrument tip relative to the target anatomy in various types of minimally invasive interventions. Therefore, such MRI-guided robots will rely on real-time processing the co-registration of surgical plan with the imaging data captured during the intervention, as well as computing the relative configuration between the instrument and the anatomy of surgical interest.
    This talk will present a compact robotic system capable to operate inside the bore of MRI scanner, as well as its solutions to technical challenges of providing a safe, effective catheter-based surgical manipulation. The proposed image processing system demonstrates its clinical potential of enhanced surgical safety by imposing visual feedback on tele-operated robotic instruments even under large-scale and rapid tissue deformations in soft tissue surgeries, such as cardiac electrophysiology and stereotactic neurosurgery. The ultimate research objective is to enable the operator to perform safe, precise and effective control of robotics instruments with the aid of pre- and intra-operative MRI models. The present work will be timely to bridge the current technical gap between MRI and surgical robotic control.
  • Biography
    Dr. Ka-Wai Kwok is currently assistant professor in Department of Mechanical Engineering, The University of Hong Kong, who completed his PhD training in The Hamlyn Centre for Robotic Surgery, Imperial College London in 2011, where he continued research on surgical robotics as a postdoctoral fellow. After then, Dr. Kwok obtained the Croucher Foundation Fellowship 2013-14, which supported his research jointly hosted by The University of Georgia, and Brigham and Women’s Hospital – Harvard Medical School. His research interests focus on surgical robotics, intra-operative medical image processing, and their uses of high-performance computing techniques. To date, he has been involved in various designs of surgical robotic devices and interfaces for endoscopy, laparoscopy, stereotactic and intra-cardiac catheter interventions. His work has also been recognized by several awards from IEEE international conferences, including ICRA’14, IROS’13 and FCCM’11. He also became the recipient of Early Career Awards 2015/16 offered by Research Grants Council (RGC) of Hong Kong.

RenHongliang-BME-NUS

Hongliang Ren, National University of Singapore, Singapore

  • TOWARDS MAGNETIC ACTUATED MICROROBOTIC NEEDLESS INJECTION
  • Abstract
    The feasibility of a needleless magnetic-actuated device for the purpose of intravitreal injections is investigated using three different design prototypes.
    A needleless device could potentially significantly reduce patient anxiety levels and occurrences of needle stick injuries to both healthcare workers and patients Moreover, a magnetic-actuated device allows for control of the current supplied over time to the device and the corresponding depth of penetration of the drug.
    Substitutes for the sclera and vitreous region were used in the experiments where a blue dye was injected using the two separate devices to identify if these devices were able to eject the liquid with enough force needed to penetrate the sclera and deliver the liquid to within the vitreous region and whether there was a relationship between the current supplied to the devices and the depth of delivery The solenoid prototype injector was not able to eject the liquid at a force required to penetrate the sclera although, because the vitreous region was a lot softer, a follow through current was predicted to be able to deliver the bulk of the liquid to the middle portion of the vitreous substitute used in this experiment. Moreover, the addition of a controller to the system was able to produce a two part force to the liquid, the initial peak force meant to penetrate the sclera and a follow through force to deliver the drug to the vitreous region only.
  • Biography
    Dr. Hongliang Ren is currently an assistant professor and leading a research group on medical mechatronics in the Biomedical Engineering Department of National University of Singapore (NUS). He is an affiliated Principal Investigator for the Singapore Institute of Neurotechnology (SINAPSE) and Advanced Robotics Center at National University of Singapore. Dr. Ren received his PhD in Electronic Engineering (Specialized in Biomedical Engineering) from The Chinese University of Hong Kong (CUHK) in 2008. After his graduation, he worked as a Research Fellow in the Laboratory for Computational Sensing and Robotics (LCSR) and the Engineering Center for Computer-Integrated Surgical Systems and Technology (ERC-CISST), Department of Biomedical Engineering and Department of Computer Science, The Johns Hopkins University, Baltimore, MD, USA, from 2008 to 2010. In 2010, he joined the Pediatric Cardiac Biorobotics Lab, Department of Cardiovascular Surgery, Children’s Hospital Boston & Harvard Medical School, USA, for investigating the beating heart robotic surgery system. Prior to joining NUS, he also worked in 2012 on a collaborative computer integrated surgery project, at the Surgical Innovation Institute of Children’s National Medical Center, USA. His main areas of interest include Biomedical Mechatronics, Computer-Integrated Surgery, and Dynamic Positioning in Medicine.

A compact continuum tubular robotic system for transnasal procedures

Video

[kad_youtube url=”https://youtu.be/E_OXL-4kxAY” ]

Project Goals

Nasopharynx cancer, or nasopharyngeal carcinoma (NPC), is a tumor that originates in the nasopharynx, the uppermost region of the pharynx where the nasal passage and the throat join. It is a common disease occurring to ethnic Chinese people living in or emigrating from southern China; it is also the eighth most frequently occurred cancer among Singaporean men. Traditional posterior nasopharyngeal biopsy using a flexible nasal endoscope has the risks of abrasion and injury to the nasal mucosa and thus causing trauma to the patient. Therefore, the goal of this project is to develop a compact continuum tubular robotic system to achieve collision free nasopharyngeal biopsy.

illustration

Fig.1  Illustration of the proposed CTR for nasopharyngeal biopsy.

Approaches

We developed a compact CTR which is 35 cm in total length, 10 cm in diameter, 2.15 kg in weight, and easy to be integrated with a robotic arm to perform more complicated operations.

2

Fig.2 The proposed continuum tubular robot

3

Fig.3 Compact and light weight CTR integrated with a positioning arm for
better conducting surgery

We also developed a 3D printed biopsy needle to equip our robot for transnasal biopsy procedure.
5

Fig.4  3D printed biopsy needle for transnasal biopsy

The workspace of the robot was analyzed to determine optimized tube parameters.

workspace

Fig.5 Workspace comparison for 3-DOF CTR with three initial configurations.
Top: all the outstretched part of the inner tube exposes; Middle: the outstretched part of the inner tube is partially covered by the outer tube; Bottom: the outstretched part of the inner tube is totally covered by the outer tube.

Further more, by using an electromagnetic tracking system, we are able to build a navigation system with shape reconstruction for the tubes.

7

Fig.6  Shape reconstruction using 3-order Bézier curve fitting

8

Fig.7 Sensing by EM tracker

10

Fig.8 Navigation interface

Results

Three groups of experiments were carried out. The first group is to tele-operate the robot to follow a linear path and a circular path. We found that the path following accuracy was about 2 mm.

11

Fig.9 Tele-operating the robot to follow a linear path and a circular path

12

Fig.10 Accuracy of the robot following the predefined paths

The second group is to validate the shape reconstruction algorithm. The accuracy of the results is about 1 mm.

13

Fig.11 Reconstruction setup

13

Fig.12 Reconstruction error

In the last group of experiments, the robot was tested in a biopsy procedure on a cadaver. The feasibility of the proposed robotic system was validated.

14

Fig.13  Cadaver experiment setup

15

Fig.14 Cadaver experiment process

People Involved

Research Fellow: Liao Wu
PhD Student: Keyu Wu
FYP Student: Li Ting Lynette Teo
Intern Student: Jan Feiling and Xin Liu
Project Investigator: Hongliang Ren

Publications

[1] Liao Wu, Shuang Song, Keyu Wu, Chwee Ming Lim, Hongliang Ren. Development of a compact continuum tubular robotic system for nasopharyngeal biopsy. Medical & Biological Engineering & Computing. 2016.
[2] Keyu Wu, Liao Wu, Hongliang Ren. Motion planning of continuum tubular robots based on features extracted from statistical atlas. In: Proceedings of 2015 IEEE International Conference on Intelligent Robots and Systems (IROS 2015).
[3] Keyu Wu, Liao Wu, Chwee Ming Lim, Hongliang Ren. Model-free image guidance for intelligent tubular robots with pre-clinical feasibility study: towards minimally invasive trans-orifice surgery. In: Proceedings of 2015 IEEE International Conference on Information and Automation (ICIA 2015). ( best paper finalist)
[4] Benedict Tan, Liao Wu, Hongliang Ren. Prototype development of a handheld tubular curvilinear robot for minimally invasive surgery. In: The 11th Asian Conference on Computer Aided Surgery (ACCAS 2015).
[5] Keyu Wu†, Liao Wu†, Hongliang Ren. An image based targeting method to guide a curvilinear concentric tube robot. In: Proceedings of 2014 IEEE International Conference on Robotics and Biomimetics (ROBIO 2014). Bali, Indonesia, 2014: 386-391 († equally contributed author).

Tracking Magnetic Particles under Ultrasound Imaging using Contrast-Enhancing Microbubbles

Abstract

Magnetic microbubbles which can be controlled by an external magnetic field have been explored as a method for precise and efficient drug delivery. In this paper, a technique for the fabrication of microbubble encapsulated magnetic spheres is presented. The resultant magnetic spheres were subsequently imaged using ultrasound and the encapsulated microbubbles proved to appear as bright spots and resulted in enhanced ultrasound image contrast, as compared to the solid magnetic spheres which appeared dull. A tracking algorithm was then developed for the tracking of the magnetic microbubbles based on optical flow tracking. Further development of the magnetic microbubbles and tracking algorithm can lead to future use of the tracking algorithm in the case of in vivo injection of the magnetic microbubbles.

Publications

1. Loh Kai Ting, Ren Hongliang and Li Jun, Tracking Magnetic Particles under Ultrasound Imaging using Contrast-Enhancing Microbubbles, The 11th Asian Conference on Computer Aided Surgery, 2015.

Poster in BME Showcase 2015

KT Poster Final Printed

Planning and Navigation for Percutaneous Ablations

Project Goals

Two challenges are mostly clinical concerns in tumor ablation — the size of the tumor and accessibility to the probes. Multiple overlapping ablations need to be planned to cover irregular and oversize tumors through a series of single probe ablations. In the meantime, the planned ablations should be accessible by the needle-based probe and should avoid critical healthy tissue. Manual treatment planning and execution is dependent on the operator’s experience and relies on a trial and error approach, which is error-prone and time-consuming without the assistance of planning and navigation. To address these challenges, we focus on an automated planning and navigation system for percutaneous radio-frequency ablations, particularly for liver tumor ablation. The planning system incorporates clinical constraints on ablations and trajectories using a multiple objective optimization formulation.

Collaborators:

Bioengineering Initiative, Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Medical Center, Washington, DC
Imaging Science and Information Systems (ISIS) Center, Department of Radiology, Georgetown University Hospital
Surgical Planning Laboratory, Department of Radiology, Brigham and Women’s Hospital and Harvard Medical School

Approaches

Towards semi-automatic treatment planning for image-guided surgical interventions, we develop a systematic approach to the needle-based ablation placement task, including pre-operative planning algorithms and intra-operative tracking and navigation approaches.

The overall system concept is shown in Fig. 1, with the key components including treatment optimization, treatment evaluation, and surgical navigation. Specifically, the planning workstation implements patient-specific modeling through segmentation, margin addition, optimization, and plan evaluation as illustrated in Fig. 2.
Semi-automatic segmentation based on geodesic active contour method is used to identify the key structures including: the tumor; structures that should not be traversed such as the ribs, liver vasculature, and adjacent critical anatomical structures, collectively referred to as a no-fly-zone; and surgeon preferred entry points. Additional margins are created for tumor tissue, ablation margin, and critical tissue, which includes safety margins that should be avoided. This is realized by applying a binary image morphological operator, dilation, to the segmented tumor and critical structures. The margin creation process can be described by the following morphological dilation operation.

The flowchart in Fig. 2 describes the optimal planning and the evaluation modules. A semiautomatic treatment planning module for optimized probe placement is developed to guide the RFA ablation probe. For a given irregular liver tumor, the solution of a mathematical optimization problem provides 1) optimized probe trajectories, 2) location of multiple overlapping ablations in order to cover the tumor, and 3) a tumor-free margin, while avoiding the no-fly zone. Hence, the treatment planning is a multiple-objective optimization problem guided by these five clinical considerations:

  • Minimize the number of ablations. Fewer ablations mean shorter treatment times and less chance for complications.
  • Limit the number of probe insertions. This reduces the perforations to the liver capsule decreasing the chances of intraperitoneal haemorrhage.
  • Probe trajectory constraints. The model includes physical constraints imposed by ribs, vessels, and other organs which restrict possible trajectories.
  • Irregular shaped tumor coverage. The optimization uses segmented tumor data from patients and does not pre-suppose a particular tumor shape. This makes this planning method more general.
  • Minimize unnecessary damage to healthy tissue while fully covering the tumor and margin.

The optimization module uses integer programming techniques to model and solve the planning problem. Considering a voxelized tumor region, the possible choices for trajectories and ablations are represented by binary decision variables and the clinical constraints are modeled algebraically using linear inequalities. Aiming at optimizing multiple measures of RFA planning performance simultaneously, we present a decomposition approach that solves this decision problem by repeatedly solving two integer programming models. Initially, a set of entry points is specified by the clinician and each entry point is tested for feasibility in avoiding direct puncture of critical structures to the tumor. Then, for each feasible entry point we define the following two optimization models: the Minimal Trajectories Integer Program (MTIP) to find a minimal number of trajectories necessary to cover the tumor, and the Minimal Ablations Integer Program (MAIP) to find a minimal number of ablations along the selected trajectories necessary to cover the tumor. In each of these integer programs we employ a weighted formulation to reduce healthy tissue damage, while keeping as main objective the minimization of the number of trajectories and ablations that are needed to guarantee coverage of the tumor and safety margin.

Results and Remarks

The planning module yielded 100% coverage over the large tumor using multiple ablations and can generate multiple feasible plans with evaluation parameters for physicians to choose. Both numerical evaluation and visual evaluation can be performed to determine the execution plan from those candidates. The number of trajectories and ablations are reduced to a minimum at the same time. In our previous approach for planning ablations for lung tumors, we only generated an “optimal” solution, which removed the specific perspective of the interventionalist. We now provide the physician with multiple feasible plans which satisfy to some degree the optimization requirements. This is a cooperative approach to planning in which the computational burden is automated, and the clinician selects from a small set of plans which satisfy the clinical criteria such as maximum number of trajectories, maximum number of ablations, overlap of ablation spheres, etc. This approach yields comprehensive and clinically feasible planning results. Given the requirement of 100% coverage on the tumor, the over-ablation rate is found relevant to the size and shape of the tumor, the size of ablation probe and the maximum number of ablations.


The navigation module based on electromagnetic tracking system is susceptible to interference from the CT scanner. In earlier phantom studies on the CT table directly, the fiducial registration error was up to 10 mm, which is too large for accurate targeting. Once we moved the phantom to a metal-free environment the fiducial registration error could be decreased to 1 mm and yield accurate targeting performance. For this reason, in our animal study the swine was moved to a table in the CT room away from the CT gantry, where we were able to obtain a registration error from 3.6 to 3.9 mm for several trials. This makes the postoperative CT evaluation difficult for each ablation, as the animal cannot be moved back to CT and moved out for performing the subsequent planned ablations without potentially changing its position relative to the V-trough. The final targeting error is difficult to evaluate as the planned trajectory cannot be mapped to the postoperative image coordinate system. Instead, we measure the distance from the probe to the tumor margin region surface in 3D-Slicer and found the distance from the probe to the closest tumor surface was approximately 5 mm. For the future study, a pre-operative image to post-operative image registration method can be developed to overcome this limit in ablation evaluation.
According to the planning results and evaluation results on the second ablation, we show the feasibility of semiautomatic planning and navigation procedures overseen by the radiologist. The presented ablation planning and navigation approach provides a comprehensive solution for treating large tumors using RFA, while keeping the physician in the loop. The planning system uses a patient specific model and an optimization approach to produce potential plans which satisfy multiple clinical criteria to certain degrees. The clinicians then select the plan which they judge to be most appropriate. The navigation system provides the precise guidance required to carry out the plan, which currently is all but impossible to do using the standard free hand technique
To summarize, a new treatment planning and navigation system was developed for liver tumor ablations, particularly for multiple overlapping radiofrequency ablations. The treatment planning is composed of needle-like probe trajectory planning and overlapping ablation planning. Multiple-objective optimization for probe insertions incorporates both clinical and technical constraints. Additional validation is required prior to introducing our system into a clinical trial. Systematic evaluations were presented to check the candidate plans by both statistical measures and visualization. The presented semiautomatic planning and guidance method can be applied to tumor ablation in other organs using the proposed techniques. In its current form the system in combination with a phantom can also be used as a training aid for interventional radiologists.

People Involved

Hongliang Ren
Enrique Campos-Nanez
Ziv Yaniv
Filip Banovac
Hernan Abeledo
Nobuhiko Hata
Kevin Cleary

Publications

[1] Ren, H.; Campos-Nanez, E.; Yaniv, Z.; Banovac, F.; Hata, N. & Cleary, K. Treatment Planning and Image Guidance for Radiofrequency Ablations of Large Tumors IEEE Transactions on Information Technology in Biomedicine (IEEE Journal of Biomedical and Health Informatics), 2013

Videos:

3D Ultrasound Tracking and Servoing of Tubular Surgical Robots

Collaborators:

[Pediatric Cardiac Bioengineering Lab of Children’s Hospital Boston, Harvard Medical School, USA]
[Philips Research]

Abstract

Ultrasound imaging is a useful modality for guiding minimally invasive interventions due to its portability and safety. In cardiac surgery, for example, real-time 3D ultrasound imaging is being investigated for guiding repairs of complex defects inside the beating heart. Substantial difficulty can arise, however, when surgical instruments and tissue structures are imaged simultaneously to achieve precise manipulations. This research project includes: (1) the development of echogenic instrument coatings, (2) the design of passive instrument markers, and (3) the development of algorithms for instrument tracking and servoing. For example, a family of passive markers has been developed by which the position and orientation of a surgical instrument can be determined from a single 3D ultrasound volume using simple image processing. Marker-based estimates of instrument pose can be used in augmented reality displays or for image-based servoing.
For example, a family of passive markers has been developed by which the position and orientation of a surgical instrument can be determined from a single 3D ultrasound volume using simple image processing. Marker-based estimates of instrument pose can be used in augmented reality displays or for image-based servoing. The design principles for marker shapes ensure imaging system and measurement uniqueness constraints are met. Error analysis is used to guide marker design and to establish a lower bound on measurement uncertaintanty. Experimental evaluation of marker designs and tracking algorithms demonstrate a tracking accuracy of 0.7 mm in position and 0.075 rad in orientation.
Another example is to investigate the problem of automatic curve pattern detection from 3D ultrasound images, because many surgical instruments are curved along the distal end during operation, such as continuum tube robot, and catheter insertion etc. We propose a two-stage approach to decompose the six parameter constant-curvature curve estimation problem into a two stage parameter estimation problems: 3D spatial plane detection and 2D circular pattern detection. The algorithm includes an image-preprocessing pipeline, including thresholding, denoising, connected component analysis and skeletonization, for automatically extracting the curved robot from ultrasound volumetric images. The proposed method can also be used for spatial circular or arc pattern recognition from other volumetric images such as CT and MRI.
Additional related information at [Pediatric Cardiac Bioengineering Lab of Children’s Hospital Boston, Harvard Medical School]

Surgical Tracking System for Laparoscopic Surgery

Collaborators:
ERC-CISST, LCSR Lab of Johns Hopkins University, USA
Fraunhofer Germany (FhG)

Laparoscopic surgery poses a challenging problem for a real-time intra-body navigation system: how to keep tracking the surgical instruments inside the human body intra-operatively. This project aims to develop surgical tracking technology that is accurate, robust against environmental disturbances, and does not require line-of-sight. The current approach is to combine electromagnetic and inertial sensing. Sensor fusion methods are proposed for a hybrid tracking system that incorporates a miniature inertial measurement unit and an electromagnetic navigation system, in order to obtain continuous position and orientation information, even in the presence of metal objects.
Additional information at [SMARTS Lab of Johns Hopkins University]