Motion Planning of Flexible Manipulators by Learning from Human Expert Demonstrations

Abstract

Motion Planning of Multiple-segment flexible soft, and continuum Flexible Manipulators by Learning from Human Expert Demonstrations

Multiple-segment flexible and soft robotic actuators exhibit compliance but suffer from the difficulty of path planning due to their redundant degrees of freedom, although they are promising in complex tasks such as crossing body cavities to grasp objects. We propose a learning from demonstration method to plan the motion paths of flexible manipulators, by statistics machine-learning algorithms. To encode demonstrated trajectories and estimate suitable paths for the manipulators to reproduce the task, models are built based on Gaussian Mixture Model and Gaussian Mixture Regression respectively. The forward and inverse kinematic models of soft robotic arm are derived for the motion control. A flexible and soft robotic manipulator verifies the learned paths by successfully completing a representative task of navigating through a narrow keyhole.


 

Demo video at:

 

Publications

  • H. Wang; J. Chen; H. Y. Lau & H. Ren Motion Planning of IPMC Flexible Manipulators by Learning from Human Expert Demonstrations ICRA2016, IEEE International Conference on Robotics and Automation, 2016
  • J. Chen; H. REN & A. Lau Learning Reaching Movement Primitives from Human Demonstrations with Gaussian Mixture Regression and Stabilized Dynamical Systems International Conference on Control Science and Systems Engineering ICCSSE 2016, 2016
  • J. Chen; W. Xu; A. Lau & H. REN Towards Transferring Skills to Flexible Surgical Robots with Programming by Demonstration and Reinforcement Learning The Eighth International Conference on Advanced Computational Intelligence (ICACI2016), 2016
  • J. Chen; W. Xu; H. Ren & H. Y. Lau Automate Adaptive Robot Reaching Movement Based on Learning from Human Demonstrations with Dynamical Systems ROBIO2016, 2016

Statistical Humerus Implants and Associated Intramedullary Robotics

Project Goals

The sizes of current off-the-shelf humerus implants are unable to accommodate Asian patients since they are mainly produced for American and European populations according to locally collected data. By creating statistical humerus atlases based on Asian data, gender-specific and region-specific humeral implants can be developed by considering the characteristics of the statistical atlas constructed in order to improve stability of the fixation and avoid related complications. Besides, it is envisioned that the statistical atlas can serve as a critical reference for development and evaluation of robots in surgical procedures. Particularly, for the surgical and interventional procedures in the confined and rotated intramedullary space, the curvature and shape statistics of internal humerus canal is of great significance for the dedicatedly design of snake-like curvilinear tubular robot.

In this project, an efficient way has been demonstrated to construct statistical atlas by adopting an efficient alignment algorithm with improved efficiency and good accuracy. The constructed humerus atlas is then regarded as the reference for design of various humerus implants and development of snake-like concentric tube robots.

Approach

Statistical Atlas Construction: A three-step algorithm is adopted in statistical atlas construction, including segmentation, alignment and principal component analysis (PCA). Segmentation is to extract the desired surface mesh information of the humeri from the raw CT data and alignment is to align all the samples. The final step is to perform the principal component analysis of the shapes and represent the statistical model using principal components.

Creation and application of a statistical humerus atlas

Creation and application of a statistical humerus atlas

Centerline Extraction for Intramedullary Robot Design: The Laplacian-Based Contraction Method is adopted to extract the centerline of the humerus atlas. The purpose is to explore the intramedullary structure of the humerus since the curvature and shape statistics of internal humerus canal is significant for the design of snake-like curvilinear tubular robot.

Centerline Extraction Process

Centerline Extraction Process

Curvature Analysis for Design of Humerus Implants: The maximum principal curvature is depicted in the below Figure. The curvature analysis is to study the statistical surface curvature of the humerus atlas, in order to assist the design of humerus implants such as proximal and distal humerus locking plates, both used in orthopaedic trauma fixation.

Principal curvature of the statistical humerus atlas

Principal curvature of the statistical humerus atlas

Current Results

By adopting the novel atlas construction algorithm, the statistical humerus atlas is constructed as shown in the below figure, where the shape variation is along the first three principal components (PCs) and each row is generated by varying the shape with -3 to +3 standard deviations.

The variation along the first principal component is shown here (click to view the animation). From -3std to +3std, the length of the humerus model is increased while the width is decreased.

Shape variation along the 1st principal component

Shape variation along the 1st principal component

Moreover, by analyzing the characteristics of the humerus atlas, the intramedullary continuum robot design and the proximal humerus locking plate are depicted in the following figures.

Design of intramedullary continuum robot based on the statistical humerus atlas

Design of intramedullary continuum robot based on the statistical humerus atlas

Proximal humerus locking plate

Proximal humerus locking plate

People Involved

Staff: Keyu WU
Advisor: Hongliang REN
Clinicians: Keng Lin Wong, Zubin Jimmy Daruwalla, Diarmuid Murphy, National University Hospital

Publications

  • Wu K, Wong FKL, Ng SJK, Quek ST, Zhou B, Murphy D, Daruwalla ZJ and Ren H (2015), “Statistical atlas-based morphological variation analysis of the asian humerus: Towards consistent allometric implant positioning”, International Journal of Computer Assisted Radiology and Surgery. Vol. 10(3), pp. 317-327. Springer Berlin Heidelberg
  • Wu K, Daruwalla ZJ, Wong FKL, Murphyand D and Ren H (2015), “Development and Selection of Asian-specific Humeral Implants based on Statistical Atlas: Towards Planning Minimally Invasive Surgery”, International Journal of Computer Assisted Radiology and Surgery. Vol. 10(8), pp. 1333-1345. Springer Berlin Heidelberg.
  • Wu, K.; Wong, F. K. L.; Daruwalla, Z. J.; Murphy, D. & Ren, H. Statistical Humerus Atlas for Optimal Design of Asian-Specific Humerus Implants and Associated Intramedullary Robotics, ROBIO 2013, IEEE International Conference on Robotics and Biomimetics, 2013; (Best Paper Finalist award)

Supporting materials

Supplementary information for

Statistical Atlas Based Morphologic Variation Analysis of the Asian Humerus: Towards Consistent Allometric Implant Positioning

K. Wu, K. L. Wong, S. J. K. Ng, S. T. Quek, B. Zhou, D. P. Murphy, Z. J. Daruwalla, H. Ren*

 

All data are published in .nii format, which can be opened by ITK-SNAP (http://www.itksnap.org/pmwiki/pmwiki.php) and 3D Slicer (http://www.slicer.org/).

Table 1. Information of the subjects.

Study Number

Male/Female

Age

Race

Weight (kg)

Height (cm)

Exercise habit:

S = sedentary,

A = active

Left/Right:

L = left,

R = right

5260823

F

61

C

67.9

143

S

L

5190438

M

16

C

55.6

160

A

R

5186672

M

63

C

67.5

160

A

L

5172526

M

50

C

61.0

170

S

L

5018737

F

77

C

57.9

151

S

R

4636229

M

54

C

42.8

161

S

L

4585165

M

38

C

60.3

171

A

L & R

4481609

F

83

C

60.1

149

S

L

4448930

F

49

I

73.7

150

S

R

4320030

M

69

C

63.7

163

S

L

4111802

M

63

C

57.4

160

S

L

3969853

F

61

C

64.3

157

A

R

3564131

M

19

C

97.5

167

A

L

3495415

M

74

M

57.1

175

S

L

3827904

M

33

M

64.2

170

A

L

3722692

F

73

I

57.3

151

S

L

3768879

M

20

C

65.0

170

A

L

3771777

F

74

C

53.3

165

S

L & R

3159961

M

51

C

60.0

170

A

L

3273883

F

80

C

54.8

155

S

L

3384844

F

71

C

53.1

155

S

R

3353378

F

78

M

40.0

140

S

L

3271006

F

59

M

60.0

151

S

L

3189139

M

47

C

72.0

179

A

R

3132590

M

22

M

79.0

168

A

L

3143094

M

23

C

60.6

162

A

R

3282457

F

90

C

41.2

150

S

L

3296805

M

57

M

61.2

160

S

L

3326396

M

21

C

76.0

175

A

L

3338591

F

61

C

56.0

153

S

L & R

3060290

M

21

M

73.0

160

A

L

5581085

F

49

M

81.9

159

S

L & R

5691045

F

45

C

45.1

161

A

L

5731852

F

87

C

58.2

150

S

R

5733267

F

57

C

56.4

142

S

L & R

5622431

M

64

I

52.9

162

A

R

6007641

M

44

I

74.2

165

A

L

6013682

F

74

C

52.0

153

S

R

5504982

F

24

C

54.0

153

A

L

image001L
Fig. 1. Statistical male humerus atlas (left). The shape variation is along the first three principal components (PCs) and each row is generated by varying the shape with standard deviations ranging from -3√(λ_k ) to +3√(λ_k ).

image004R
Fig. 2. Statistical female humerus atlas (left). The shape variation is along the first three principal components (PCs) and each row is generated by varying the shape with standard deviations ranging from -3√(λ_k ) to +3√(λ_k ).

References

[1] H. Ren, N. V. Vasilyev, and P. E. Dupont, “Detection of curved robots using 3d ultrasound,” in Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on. IEEE, 2011, pp. 2083–2089.
[2] H. Ren and P. E. Dupont, “Artifacts reduction and tubular structure enhancement in 3d ultrasound images,” in International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC, 2011.
[3] G. Chintalapani, L. M. Ellingsen, O. Sadowsky, J. L. Prince, and R. H. Taylor, “Statistical atlases of bone anatomy: construction, iterative improvement and validation,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2007. Springer, 2007, pp. 499–506.
[4] H. Ren and P. E. Dupont, “Tubular enhanced geodesic active contours for continuum robot detection using 3d ultrasound,” in IEEE International Conference on Robotics and Automation, ICRA ’12, 2012.
[5] X. Kang, H. Ren, J. Li, and W.-P. Yau, “Statistical atlas based registration and planning for ablating bone tumors in minimally invasive interventions,” in Robotics and Biomimetics (ROBIO), 2012 IEEE International Conference on. IEEE, 2012, pp. 606–611.
[6] J. Cao, A. Tagliasacchi, M. Olson, H. Zhang, Z. Su, “Point Cloud Skeletons via Laplacian Based Contraction,” in Shape Modeling International Conference (SMI), pp. 21-23, 2010.

Ablation Planning in Computer-Assisted Interventions

Project Goals

Tumor ablation is the removal of tumor tissue and is considered as one type of minimally invasive interventions. It can be performed using techniques like cryoablation, high-intensity focused ultrasound (HIFU), and radiofrequency ablation (RFA). These techniques rely on minimally invasive principles to ablate tumor tissues, without having to directly expose the target regions to the environment. It has been widely noted that the success of a tumor ablation procedure hinges greatly on its pre-operative planning, which is often assisted by computational interventions. The proposed ablation planning system in this paper focuses mainly on the radiofrequency ablation (RFA) of hepatic tumors. This project is to develop computational optimization algorithms to plan optimal ablation delivery. Ablation planning systems are necessary to model the 3D interventional environments, identify feasible needle insertion trajectories and deploy ablating electrodes, while avoiding many critical structures.

Approach

Genetic Algorithm (GA) was used as it can be designed to consider the multi-objective nature of a tumor ablation planning system. The proposed ablation planning system is designed based on the following objectives: to achieve complete tumor coverage; and to minimize the number of ablations, number of needle trajectories and healthy tissue damage. These objectives are taken into account using an optimization method, Genetic Algorithm (GA). GA is capable of generating many solutions within a defined search space, and these solutions can be selected to undergo evolution based on a quantified value given by a fitness function. An exponential weight-criterion fitness function is used to represent the multiple objectives such as the number of ablation spheres, the number of trajectories, the covariance, and the coverage volume.

Current Results

The proposed mathematical protocol to determine the range of ablation spheres required to achieve complete tumor coverage is feasible to be used as a reference in the context of tumor ablation planning. The following figure shows how tumor coverage changed when trajectory optimization was considered: 0% tumor coverage (top), 100% tumor coverage with [ablation radius]=15 and [number of spheres]=3 (orange spheres) (bottom).

Publications

  • Ren, H.; Guo, W.; Ge, S. S. & Lim, W. Coverage Planning in Computer-Assisted Ablation Based On Genetic Optimization Computers in Biology and Medicine, in press, 2014
  • Lim, W. & Ren, H. Cognitive Planning Based on Genetic Algorithm in Computer-Assisted Interventions CIS-RAM 2013, 6th IEEE International Conference on Cybernetics and Intelligent Systems (CIS) and the 6th IEEE International Conference on Robotics, Automation and Mechatronics (RAM), 2013

People Involved

FYP Student: Wan Cheng LIM
Graduate Student: Weian GUO
Advisor: Dr. Hongliang REN

References

[1] C. Baegert, C. Villard, P. Schreck, L. Soler, and A. Gangi, “Trajectory optimization for the planning of percutaneous radiofrequency ablation of hepatic tumors,” Computer Aided Surgery, 12(2): pp. 82-90, March, 2007.
[2] Z. Yaniv, P. Cheng, E. Wilson, T. Popa, D. Lindisch, E. Campos-Nanez, H. Abeledo, V. Watson, and F. Banovac, “Needle-Based Interventions With the Image-guided Surgery Toolkit (IGSTK): From Phantoms to Clinical Trials,” IEEE Trans. on Biomedical Engineering, vol. 57, no. 4, April, 2010.
[3] G. D. Dodd, M. C. Soulen, R. A. Kane, T. Livraghi, W. R. Lees, Y. Yamashita, A. R. Gillams, O. I. Karahan, H. Rhim. “Minimally invasive treatment of malignant hepatic tumors: At the threshold of a major breakthrough,” RadioGraphics, vol. 20, no. 1, January-February, 2000.
[4] C. Rieder, T. Kroger, C. Schumann, and H. K. Hahn, “GPU-Based Real-Time Approximation of the Ablation Zone for Radiofrequency Ablation”, IEEE Trans. On Visualization and Computer Graphics, vol. 17, no. 12, pp. 1812-1821, December, 2011.

Related Poster

Planning and Navigation for Percutaneous Ablations

Project Goals

Two challenges are mostly clinical concerns in tumor ablation — the size of the tumor and accessibility to the probes. Multiple overlapping ablations need to be planned to cover irregular and oversize tumors through a series of single probe ablations. In the meantime, the planned ablations should be accessible by the needle-based probe and should avoid critical healthy tissue. Manual treatment planning and execution is dependent on the operator’s experience and relies on a trial and error approach, which is error-prone and time-consuming without the assistance of planning and navigation. To address these challenges, we focus on an automated planning and navigation system for percutaneous radio-frequency ablations, particularly for liver tumor ablation. The planning system incorporates clinical constraints on ablations and trajectories using a multiple objective optimization formulation.

Collaborators:

Bioengineering Initiative, Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Medical Center, Washington, DC
Imaging Science and Information Systems (ISIS) Center, Department of Radiology, Georgetown University Hospital
Surgical Planning Laboratory, Department of Radiology, Brigham and Women’s Hospital and Harvard Medical School

Approaches

Towards semi-automatic treatment planning for image-guided surgical interventions, we develop a systematic approach to the needle-based ablation placement task, including pre-operative planning algorithms and intra-operative tracking and navigation approaches.

The overall system concept is shown in Fig. 1, with the key components including treatment optimization, treatment evaluation, and surgical navigation. Specifically, the planning workstation implements patient-specific modeling through segmentation, margin addition, optimization, and plan evaluation as illustrated in Fig. 2.
Semi-automatic segmentation based on geodesic active contour method is used to identify the key structures including: the tumor; structures that should not be traversed such as the ribs, liver vasculature, and adjacent critical anatomical structures, collectively referred to as a no-fly-zone; and surgeon preferred entry points. Additional margins are created for tumor tissue, ablation margin, and critical tissue, which includes safety margins that should be avoided. This is realized by applying a binary image morphological operator, dilation, to the segmented tumor and critical structures. The margin creation process can be described by the following morphological dilation operation.

The flowchart in Fig. 2 describes the optimal planning and the evaluation modules. A semiautomatic treatment planning module for optimized probe placement is developed to guide the RFA ablation probe. For a given irregular liver tumor, the solution of a mathematical optimization problem provides 1) optimized probe trajectories, 2) location of multiple overlapping ablations in order to cover the tumor, and 3) a tumor-free margin, while avoiding the no-fly zone. Hence, the treatment planning is a multiple-objective optimization problem guided by these five clinical considerations:

  • Minimize the number of ablations. Fewer ablations mean shorter treatment times and less chance for complications.
  • Limit the number of probe insertions. This reduces the perforations to the liver capsule decreasing the chances of intraperitoneal haemorrhage.
  • Probe trajectory constraints. The model includes physical constraints imposed by ribs, vessels, and other organs which restrict possible trajectories.
  • Irregular shaped tumor coverage. The optimization uses segmented tumor data from patients and does not pre-suppose a particular tumor shape. This makes this planning method more general.
  • Minimize unnecessary damage to healthy tissue while fully covering the tumor and margin.

The optimization module uses integer programming techniques to model and solve the planning problem. Considering a voxelized tumor region, the possible choices for trajectories and ablations are represented by binary decision variables and the clinical constraints are modeled algebraically using linear inequalities. Aiming at optimizing multiple measures of RFA planning performance simultaneously, we present a decomposition approach that solves this decision problem by repeatedly solving two integer programming models. Initially, a set of entry points is specified by the clinician and each entry point is tested for feasibility in avoiding direct puncture of critical structures to the tumor. Then, for each feasible entry point we define the following two optimization models: the Minimal Trajectories Integer Program (MTIP) to find a minimal number of trajectories necessary to cover the tumor, and the Minimal Ablations Integer Program (MAIP) to find a minimal number of ablations along the selected trajectories necessary to cover the tumor. In each of these integer programs we employ a weighted formulation to reduce healthy tissue damage, while keeping as main objective the minimization of the number of trajectories and ablations that are needed to guarantee coverage of the tumor and safety margin.

Results and Remarks

The planning module yielded 100% coverage over the large tumor using multiple ablations and can generate multiple feasible plans with evaluation parameters for physicians to choose. Both numerical evaluation and visual evaluation can be performed to determine the execution plan from those candidates. The number of trajectories and ablations are reduced to a minimum at the same time. In our previous approach for planning ablations for lung tumors, we only generated an “optimal” solution, which removed the specific perspective of the interventionalist. We now provide the physician with multiple feasible plans which satisfy to some degree the optimization requirements. This is a cooperative approach to planning in which the computational burden is automated, and the clinician selects from a small set of plans which satisfy the clinical criteria such as maximum number of trajectories, maximum number of ablations, overlap of ablation spheres, etc. This approach yields comprehensive and clinically feasible planning results. Given the requirement of 100% coverage on the tumor, the over-ablation rate is found relevant to the size and shape of the tumor, the size of ablation probe and the maximum number of ablations.


The navigation module based on electromagnetic tracking system is susceptible to interference from the CT scanner. In earlier phantom studies on the CT table directly, the fiducial registration error was up to 10 mm, which is too large for accurate targeting. Once we moved the phantom to a metal-free environment the fiducial registration error could be decreased to 1 mm and yield accurate targeting performance. For this reason, in our animal study the swine was moved to a table in the CT room away from the CT gantry, where we were able to obtain a registration error from 3.6 to 3.9 mm for several trials. This makes the postoperative CT evaluation difficult for each ablation, as the animal cannot be moved back to CT and moved out for performing the subsequent planned ablations without potentially changing its position relative to the V-trough. The final targeting error is difficult to evaluate as the planned trajectory cannot be mapped to the postoperative image coordinate system. Instead, we measure the distance from the probe to the tumor margin region surface in 3D-Slicer and found the distance from the probe to the closest tumor surface was approximately 5 mm. For the future study, a pre-operative image to post-operative image registration method can be developed to overcome this limit in ablation evaluation.
According to the planning results and evaluation results on the second ablation, we show the feasibility of semiautomatic planning and navigation procedures overseen by the radiologist. The presented ablation planning and navigation approach provides a comprehensive solution for treating large tumors using RFA, while keeping the physician in the loop. The planning system uses a patient specific model and an optimization approach to produce potential plans which satisfy multiple clinical criteria to certain degrees. The clinicians then select the plan which they judge to be most appropriate. The navigation system provides the precise guidance required to carry out the plan, which currently is all but impossible to do using the standard free hand technique
To summarize, a new treatment planning and navigation system was developed for liver tumor ablations, particularly for multiple overlapping radiofrequency ablations. The treatment planning is composed of needle-like probe trajectory planning and overlapping ablation planning. Multiple-objective optimization for probe insertions incorporates both clinical and technical constraints. Additional validation is required prior to introducing our system into a clinical trial. Systematic evaluations were presented to check the candidate plans by both statistical measures and visualization. The presented semiautomatic planning and guidance method can be applied to tumor ablation in other organs using the proposed techniques. In its current form the system in combination with a phantom can also be used as a training aid for interventional radiologists.

People Involved

Hongliang Ren
Enrique Campos-Nanez
Ziv Yaniv
Filip Banovac
Hernan Abeledo
Nobuhiko Hata
Kevin Cleary

Publications

[1] Ren, H.; Campos-Nanez, E.; Yaniv, Z.; Banovac, F.; Hata, N. & Cleary, K. Treatment Planning and Image Guidance for Radiofrequency Ablations of Large Tumors IEEE Transactions on Information Technology in Biomedicine (IEEE Journal of Biomedical and Health Informatics), 2013

Videos:

Biomedical Application of Wireless Heterogeneous Sensor Networks

We study a typical heterogeneous network, a Wireless Biomedical Sensor Network (WBSN), as it consists of various types of biosensors to monitor different physiological parameters. WBSN will help to enhance medical services with its unique advantages in long-term monitoring, easy network deployment, wireless connections, and ambulatory capabilities. The network protocol plays an important role in carrying out the medical and healthcare services. Many unique challenges exist in WBSN design for medical and healthcare services, including extensive optimization problems in network protocol design to deal with power scheduling and radiation absorption concerns. Concerning these issues, we present a systematic solution to the wireless biomedical sensor network in our project named MediMesh. We develop a prototypical test-bed for medical and healthcare applications and evaluate the radiation absorption effects and efficiency. A lightweight network protocol is proposed, taking into consideration of the radiation absorption effects and communication overhead. After data acquisition from the sink stations, a data publishing system based on web service technology is implemented for typical medical and healthcare monitoring services in a hospital or home environment.