Category: Surgical Robotics

Tendon-driven Flexible Manipulator

  By ,

Project goals

This project aims to develop a flexible manipulator for transnasal/transoral surgery. Compared with existing surgical manipulators, the developed one should have better performance in workspace and dexterity, thus better facilitate the surgical operation.

Approaches

A constrained tendon-driven serpentine manipulator (CTSM) is designed as shown in Figure 1. It includes an underactuated tendon-driven flexible section, a constraint and a set of tendons. The tendon-driven flexible section is similar to our previous wire-driven robot arm design. It comprises of several identical vertebras, and an elastic tube. Two successive vertebras form a joint and the joint rotation follows the elastic tube bending. Four tendons pass through all the vertebras. For each tendon, the two ends are attached to the distal vertebra and the motor respectively. These tendons are grouped to two pairs and are orthogonally arranged as shown in Figure. 1 (b). One tendon pair controls the bending about X axis and the other tendon pair controls the bending about Y axis. The manipulator bending is planar. The bending angle and bending direction are controlled by the motion of the four tendons. The constraint can be an elastic tube or rigid tube. The constraint translates along the tendon-driven flexible section. Vertebras in the range of the constraint are confined and vertebras out of the range of the constraint are free of rotation. Thus, the last constrained vertebra serves the base of the bending section.
Fig1CTSMDesign

Figure 1 3D design of the CTSM: (a) the assembled and explosion view of the CTSM; (b) the tendon configuration; (c) the cross section view of the joint.

The bending motion of the manipulator is shown in Figure 2: when the insertion of the constraint is 0, the CTSM bends by the tendons as a traditional TSM. By pushing the constraint forward the backbone is segmented to two parts: the proximal constrained section and the distal free bending section. Compared to the distal free bending section, the proximal constrained section is stiffer and the joints’ rotations are smaller. By pushing and pulling the constraint, the lengths of the two sections are controlled.
Fig2CTSMBendingMotion

Fig. 2 Bending motion illustration: (a) the bending section is not constrained; (b) part of the bending section is constrained; (c) the whole bending section is constrained.

Prototype

A prototype is built as shown in Figure 3. In the prototype, the flexible backbone has 27 vertebras. The vertebras are fabricated by 3D printing, and the material used is plastic. Each joint can rotate up to 7.25°.The total length of the flexible backbone is 104mm, and the diameter is 7.5mm. A silicon rubber tube serves the elastic tube. The outer diameter is 3 mm and inner diameter is 2 mm. Four steel wires with nylon coating are used to control the backbone bending. The diameter of the steel is 0.3 mm. The wires are arranged orthogonally, with opponent wires make a pair. Each wire pair is connected to a drum wheel. The rotation of the drum wheel is controlled by a servo motor. The diameter of the drum wheel is 50 mm. The wires are guided by a Teflon tube, whose outer diameter is 0.9 mm and inner diameter is 0.5 mm. The replaceable constraint is hold by a chuck, which is mounted on the linear actuator. The range of the linear actuator is 100 mm.

Results

By changing the stiffness ratio between the flexible bending section and the overall stiffness λ, the workspace of the CTSM is as shown in Figure 4. In the simulation the length of the CTSM is 100 mm, and the number of vertebrae is 25.
Fig4workspace

Fig. 4 workspace comparison: (a) traditional TSM; (b) CTSM with elastic constraint; (c) CTSM with elastic constraint; (d) CTSM with rigid constraint.

When the CTSM with a rigid constraint is attached to a mobile base, the workspace and dexterity distribution are shown in Figure 5. For the tendon-driven serpentine manipulator (TSM), the dexterity is indexed as the kinematic flexibility. For a traditional TSM, the kinematic flexibility is 1 in most places; the maximum is 2. For the designed CTSM, the kinematic flexibility is enhances all over the workspace and the maximum is 15.
 

Fig5Comparison-wkdb15

Figure 5 Comparison of the dexterity distribution over the workspace: (a) traditional TSM; (b) CTSM with λ=0.

People involved

Staff: Zheng Li
Visiting Students: Gui Fu, Zhengchu Tan, Jan Feiling
PIs: Hongliang Ren and Haoyong Yu

Experiment Videos

– Phantom tests

– CTSM Experiments in ex-vivo hearts and phantoms (2014/11/22)

Publications

1. Zheng Li, Haoyong Yu and Hongliang Ren, “A Novel Constrained Tendon-driven Serpentine Manipulator (CTSM)”, ICRA 2015 (under review)
2. Zheng Li, Haoyong Yu and Hongliang Ren, “A Novel Underactuated Wire-driven Flexible Robotic Arm with Controllable Bending Section Length”, ICRA 2014 Workshop on Advances in Flexible Robots for Surgical Interventions, Hong Kong, May 31-June 7, 2014
3. Zheng Li, Ruxu Du, Haoyong Yu and Hongliang Ren, “Statics Modeling of an Underactuated Wire-driven Flexible Robotic Arm”,IEEE BioRob 2014, Sao Pauo, Brazil, Aug12-15, 2014

Presentation at BIOROB2014

Presentation at ICRA 2014

Poster at ICRA 2014

<!–
Fig3CTSMprototype

Fig. 3 CTSM prototype.

.–>

ETH Image Based Visual Servoing to Guide Flexible Robots

  By , , ,

Video Demo

Eye-To-Hand Image Based Visual Servoing to Guide Flexible Robots

Project goals

Flexible robots including active cannula or cable driven continuum robots are typically suitable for such minimally invasive surgeries because they are able to present various flexible shapes with great dexterity, which strengthens the ability of collision avoidance and enlarges the reachability of operation tools. Using model based control method will lead to artificial singularities and even inverted mapping in many situations because the models are usually developed in free space and cannot perform effectively in constrained environments. Therefore, the goal of this project is control the motion of a tentacle-like curvilinear concentric tube robot by model-less visual servoing.

Approaches

A two-dimensional planar manipulator is constructed by enabling only the three translation inputs of a six DOF concentric tube robot. As shown in Fig. 1, the concentric tube manipulator is controlled using a PID controller and the images captured by an uncalibrated camera are used as visual feedback.
Fig1setup

Fig. 1. The experimental setup includes a concentric tube robot, a camera, a laptop, a marker and a target.

The visual tracking of the concentric tube robot is based on shape detection. The circular marker is attached to the tip of the concentric tube robot and a square target is given for the tip to trace. During the experiments, the coordinates of the marker centroid and target centroid are calculated while the next target position is calculated at the same time as shown in Fig. 2.
Fig2workingmechanism

Fig. 2. Working mechanism of the system. Top: translations of the three tubes. Bottom: marker, final target and the next target position on the image plane.

Fig3overview

Fig. 3. Overview of the control algorithm. The Jacobian matrix is estimated based on the measurements of each incremental movement detected from the camera.

The framework of the controlling the robot is shown in Fig. 3. The initial Jacobian matrix is acquired by running each individual motor separately and measuring the change of tip position of the robot in the image space. Then the optimal control is achieved by solving a typical redundant inverse kinematics. And finally the Jacobian matrix is continuously estimated based on the measured displacements.

Results

To evaluate the proposed model-less algorithm, a simulation was carried out on MATLAB first. The desired and actual trajectory was shown in Fig. 4, from which it could be seen that the robot succeeded in following the reference trajectory and reaching the target position.
Fig4simulationcrt

Fig. 4. Simulation of using the proposed algorithm to control a concentric tube robot.

The proposed algorithm was also implemented on a physical concentric tube robot in free space. It was found the robot was able to reach goal with zero steady state error in all trials as shown in Fig. 5.
Fig5experiments

Fig. 5. The concentric tube robot is able to reach a desired goal using the proposed method. Top: the motion of the robot. Bottom: the reference and actual trajectories of two experiments.

People involved

Staff: Keyu WU, Liao WU
PI: Hongliang REN

Publications

1. Keyu Wu, Liao Wu and Hongliang Ren, “An Image Based Targeting Method to Guide a Tentacle-like Curvilinear Concentric Tube Robot”, ROBIO 2014, IEEE International Conference on Robotics and Biomimetics, 2014.

Nasopharyngeal Carcinoma Surveillance

  By ,

Project Goals:

Nasopharyngeal carcinoma (NPC) is a tumor arising from the epithelial cells that cover the surface and line the nasopharynx. The concern about NPC in our studies is that it is more common in regions in East Asia and Africa, specifically Southeast Asia. Due to the high tendency for NPC to develop into metastatic dissemination, about 30- 60% of locally advanced patients will develop distant metastasis and die of disseminated disease. Thus this implies that apart from early diagnosis, it is also of paramount importance to locally monitor for the recurrence of NPC or the development of distant metastasis. Therefore, there is a need for a patient-operated, in-vivo surveillance system.

Approaches:

The approach for this project is that it has to be a remote surveillance system that is able to monitor the growth of the tumor in the nasopharyngeal region independently by the patient. The design requirements are as follows below:
1. The device must be made of medically approved materials that are mechanically strong enough to withstand 1 to 2 years of constant use. This is because the average time period for NPC surveillance spans to around 2 years.
2. The device must be durable so as to last the entire time period of use.
3. The device must house a camera module, which is rotatable to the minimum of 90 degrees, such that it can accurately pan throughout the entire nasopharynx region.
Additional aims were also realized in the device design as follows:
1. The device must be made in an economically feasible manner, such that it is able to reduce medical costs as much as possible.
2. The device outcome must be similar to any other method of NPC surveillance so that the quality of the monitoring system is not compromised.
3. The device must be patient-administered, meaning that the patient is able to deploy and use the device without the assistance of medical personnel. This is so as to decrease patient dependence on the healthcare system and also an attempt to decrease the burden on clinicians.
4. The device must be as safe and hassle-free to the patient as possible.

Results and Remarks

The first prototypical device comprises of the following components:
1. Camera Module
2. Arm Head and Hook
3. Arm
4. Wheel and Cover
5. Handle
The camera module houses a mini-camera, which will be able to obtain imaging output from within the nasopharyngeal region after its deployment into the nasal cavity. This housing contains two axles on either side, to enable the rotational mechanism. The camera module is then attached to the arm head, which clicks the axle of the camera into place.
The arm head also contains a hook, whose purpose is to ensure the stability of the device once deployed to the site of the vomer bone. The hook will allow the easy positioning of the camera module to the site and also increase the stability of the device during image capture. This decreases the possibilities of blurred imaging outputs.
The rotational mechanism is mainly powered by three components: (i) the camera module and axle, (ii) the arm, and (iii) the wheel. A thin piece of nylon wire is first threaded through the camera module, then threaded down the tunnels in the arm and finally around the wheel. The rotation mechanism works when the wheel is turned in either direction.
The results of the design verification experiments show that the maximum force required to fracture the Veroclear tip and head are 5.1N and 15.6N respectively. The maximum tensile strength of nylon is 262.4 MPa. The arm can bend to a maximum of 7.3 cm with a force of 15.3N before fracture. FEA shows that even under an exaggerated maximum loading of 100N, the device does not fracture.
The device is able to capture images from within the nasopharyngeal region from the mini camera. This was done by inserting the device through the nasal passageway of a phantom skull and the tumor is shown by the piece of BluTack.
The forces obtained for fracture and/or bending of the Veroclear arm are well beyond the acceptable range of forces as set by the acceptance criteria. This shows that the Veroclear arm passes the first stage of verification testing to determine is safety and suitability in this design. However, to add further to the safety of the device, it is aimed that the final device will be made of a much more mechanically strong material, which is also medically approved: 316L Stainless Steel. From the extrapolated calculations and research of 316L Stainless Steel through FEA, it can be concluded that 316L Stainless Steel is also a good, if not better choice of material for the manufacturing of this device due to its excellent mechanical strength and durability.
Video to be uploaded

People Involved

Undergraduate Students: Neerajha Ram, Khor Jing An, Paul Ng, Ong Jun Shu, Anselina Goh
Advisor: Dr. Hongliang Ren

Awards

Awarded the MOST ELEGANT DESIGN INSTRUMENTATION AWARD at the BN3101 Presentations 2013.

3D Ultrasound Tracking and Servoing of Tubular Surgical Robots

  By , ,

Collaborators:

[Pediatric Cardiac Bioengineering Lab of Children’s Hospital Boston, Harvard Medical School, USA]
[Philips Research]

Abstract

Ultrasound imaging is a useful modality for guiding minimally invasive interventions due to its portability and safety. In cardiac surgery, for example, real-time 3D ultrasound imaging is being investigated for guiding repairs of complex defects inside the beating heart. Substantial difficulty can arise, however, when surgical instruments and tissue structures are imaged simultaneously to achieve precise manipulations. This research project includes: (1) the development of echogenic instrument coatings, (2) the design of passive instrument markers, and (3) the development of algorithms for instrument tracking and servoing. For example, a family of passive markers has been developed by which the position and orientation of a surgical instrument can be determined from a single 3D ultrasound volume using simple image processing. Marker-based estimates of instrument pose can be used in augmented reality displays or for image-based servoing.
For example, a family of passive markers has been developed by which the position and orientation of a surgical instrument can be determined from a single 3D ultrasound volume using simple image processing. Marker-based estimates of instrument pose can be used in augmented reality displays or for image-based servoing. The design principles for marker shapes ensure imaging system and measurement uniqueness constraints are met. Error analysis is used to guide marker design and to establish a lower bound on measurement uncertaintanty. Experimental evaluation of marker designs and tracking algorithms demonstrate a tracking accuracy of 0.7 mm in position and 0.075 rad in orientation.
Another example is to investigate the problem of automatic curve pattern detection from 3D ultrasound images, because many surgical instruments are curved along the distal end during operation, such as continuum tube robot, and catheter insertion etc. We propose a two-stage approach to decompose the six parameter constant-curvature curve estimation problem into a two stage parameter estimation problems: 3D spatial plane detection and 2D circular pattern detection. The algorithm includes an image-preprocessing pipeline, including thresholding, denoising, connected component analysis and skeletonization, for automatically extracting the curved robot from ultrasound volumetric images. The proposed method can also be used for spatial circular or arc pattern recognition from other volumetric images such as CT and MRI.
Additional related information at [Pediatric Cardiac Bioengineering Lab of Children’s Hospital Boston, Harvard Medical School]