Ultrasound imaging procedures are deemed as one of the most convenient and least invasive medical diagnostic imaging modalities and have been widely utilized in health care providers, which are expecting semiautomatic or fully-automatic imaging systems to reduce the current clinical workloads. This paper presents a portable and wearable soft robotic system which has been designed with the purpose of replacing the manual operation to cooperatively steer the ultrasound probe. This human-compliant soft robotic system, which is equipped with four separated parallel soft pneumatic actuators and is able to achieve movements in three directions. Vacuum suction force is introduced to attach the robot onto the intended body location. The design and fabrication of this soft robotic system are illustrated. To our knowledge, this is the first body-attached soft robot for compliant ultrasound imaging. The feasibility of the system is demonstrated through proof-of-concept experiments.
Approaches/Results/Video
Developing a wearable soft robotic system (Figure 1), which is capable of mimicking the procedure of probe steering and optimizing the contact force and angle according to the specific conditions, has great significance of reducing the reliance of the ultrasound imaging on the experience of operators and obtaining images with high quality.
People Involved
PhD Student: Xiaoyi Gu
FYP Student: Koon Lin Tan
Project Investigator: Hongliang Ren
Related Publications
Ren, H.; Gu, X. ; Tan, K. L. Human-Compliant Body-Attached Soft Robots Towards Automatic Cooperative Ultrasound Imaging 2016 20th IEEE International Conference on Computer Supported Cooperative Work in Design (CSCWD 2016), IEEE, 2016, –
The part-time research positions are to support a funded project on the study of a computer integrated surgery system. The successful applicants will involve hands-on development for surgical robot prototype or surgical navigation system study based on their backgrounds. The successful applicants will also be exposed to the other interesting research projects in surgical robotic system development.
Position Responsibilities:
Assist in the development of a computer integrated surgery system.
Closely collaborates with the PI and other research assistants throughout the study.
Write experiment reports.
Learn to manage the project.
Qualifications:
Prior clinical knowledge is not required but would be a plus.
Experience with mechanical design or computer programming.
Self-motivated.
Willing to learn and work independently.
Strong problem-solving, interpersonal, and analytic skills. To apply, please email a detailed CV (including education background, courses, experiences, and skills) to:
Dr. REN, Hongliang
Prepare Student Seminars – paper selection
Time-Frequency-Spatial Analysis STFT (HR)
5
Feb9 (CNY)
Holidays
6
Feb16
Neural signals (clinical applications)- EEG, evoked potentials (HR) Lab tutorial for Project I: Neural Signals and Analysis
Recess
Feb22
7
Mar1
Multiple Dimensional Signal Processing (HR) Lab Project II: Application in neural systems
Student Reading Seminars (HR)
8
Mar8
Neuro Diagnostic and Therapeutic Devices by NT
9
Mar15
Brain machine interfaces (NT)
EEG/ECoG
10
Mar22
Neuromorphic Engineering – Brain Inspired Robotics by SK
11 Mar29
Neuroimaging and Image Processing (HR)
Neuroimaging fMRI (HR)
12 Apr5
Advanced Neurosignal Processing / Neurosurgical systems (HR)
13
Apr12 (makeup)
Project Reports (due before final) & presentations (HR, NT)
Course Projects
1. EEG for brain state monitoring
2. EEG/EMG Feature Identification Extension
AIMS & OBJECTIVES
This module teaches students the advanced neuroengineering principles ranging from basic neuroscience introduction to neurosensing technology as well as advanced signal processing techniques. Major topics include: introduction to neurosciences, neural recording methods, neural circuits, amplifiers, telemetry, stimulation, sensors for measuring the electric field and magnetic field of the brain in relation to brain activities, digitization of brain activities, neural signal processing, brain machine interfaces, neurosurgical systems and applications of neural interfaces. The module is designed for students at Master and PhD levels in Engineering, Science and Medicine.
PREREQUISITES
Basic probability
Basic circuits
Linear algebra (matrix/vector)
Matlab or other programming
Recommended Textbooks: Neural Engineering, Edited by Bin He
Seminar papers
TEACHING MODES
The majority of the course will be in lecture-tutorial format. Some advanced topics will be in the formats of seminar and research presentations.
ASSESSMENT
Take Home Tests (5 for 50%)
Labs/Projects Reports + Presentations (20%)
Seminars (10%)
Take Home Final Exam(20%)
When a robot is required to perform specific tasks defined in the world frame, there is a need for finding the coordinate transformation between the kinematic base frame of the robot and the world frame. The kinematic base frame used by the robot controller to define and evaluate the kinematics may deviate from the mechanical base frame constructed based on structural features. Besides, by using kinematic modeling rules such as the product of exponentials (POE) formula, the base frame can be arbitrarily located, and does not have to be related to any feature of the mechanical structure. As a result, the kinematic base frame cannot be measured directly. This paper proposes to find the kinematic base frame by solving a hand-eye calibration problem using 3D position measurements only, which avoids the inconvenience and inaccuracy of measuring orientations and thus significantly facilitates practical operations. A closed-form solution and an iterative solution are explicitly formulated and proved effective by simulations. Comprehensive analyses of the impact of key parameters to the accuracy of the solution are also carried out, providing four guidelines to better conduct practical operations. Finally, experiments on a 7-DOF industrial robot are performed with an optical tracking system to demonstrate the superiority of the proposed method using position data only over the method using full pose data.
Flexible robotic manipulators have been widely used in minimally invasive surgery (MIS) and many other applications requiring closer inspection and operation. Although a variety of manipulators enabled by different mechanism have been developed, few of them can preserve softness, thinness and decent bending capability simultaneously. We develop miniature soft robotic manipulators made of hyper-elastic silicone rubber. Along with the manipulator design, novel fabrication methods are proposed and elaborated. Detailed characterizations are specified to show the bending capability of the manipulator given different air pressure. Specifically, our manipulator, as thin as 6 mm, is able to achieve 360° directional bending, and, when given pressure over 70kPa, it can reach 180° bending angle and around 5mm bending radius easily. Due to its innate compliance and small dimension, this type of robotic manipulator can deliver safe and comfortable interactions with the subjects. More significantly, the novel fabrications in this paper diversify the fabrication methods for soft pneumatic robots and actuators (SPRA) and further scale down their sizes. Continue reading
Light pipe is a tool providing illumination for delicate operations inside the eyeball in an eye surgery. Traditional light pipes with straight shafts have to be reoriented in order to enlarge its illustration range, which implies more risk of damage to the sclera and requires greater operation space for the handle. In order to overcome these limitations, this paper has proposed a novel robotized handheld light pipe based on concentric tubular structure. Tube parameters are selected based on the anatomical dimension of eyeballs. Novel actuation designs are described to meet the requirements of the intraocular procedures. Intuitive control scheme is implemented to facilitate surgeons’ operations. Furthermore, the benefit of the precurved tube compared with that of a straight tube is evaluated through experiments. It has been demonstrated that the proposed design as a light pipe holds the promise of enhancing intraocular surgeries.
The overall objective of this project is to develop and evaluate the key technologies of Transnasal Artificial Tentacles, a new approach involving research on the tentacle-like robotic and sensing technologies. This is aiming to create a compliant, flexible, multiple-sensorized, immersive, intraoperative targeting and guidance system for constrained transnasal endoscopic surgeries in a confined workspace.
Minimally invasive surgery-based nasopharyngeal cancer treatment is promising, but currently, it is not a common treatment choice because of the absence of suitable tools. In this project, a multi-channel concentric tube robot is proposed for the treatment of nasopharyngeal cancer based on natural orifice translumenal endoscopic surgery. The proposed system has three channels, i.e. two manipulation channels and one vision channel, and all the three channels are confined by a 10 mm active sheath. The robot is controlled by human-in-the-loop bimanual teleoperation under active endoscopic guidance. The reduced sheath diameter and the steerable vision channel improve the functionality of the system and distinguish our design from the prior art. The feasibility of the system has been evaluated through a series of simulations and experiments. Results show that the proposed system is capable to conduct cooperative tasks in a confined space and the miniaturized manipulator is suitable for transnasal procedures. Besides, comparisons with other types of flexible surgical robots are discussed to further demonstrate the superiority of the proposed system in the target clinical applications.
Funding
Singapore Millennium Foundation
PI: Jan/11/2015 – Jan/10/2018
PUBLICATIONS
Yu H, Wu L, Wu K, Lim CM and Ren H (2016), “Development of a Multi-Channel Concentric Tube Robotic System With Active Vision for Transnasal Nasopharyngeal Carcinoma Procedures”, IEEE Robotics and Automation Letters., Jan, 2016. Vol. 1(2), pp. 1172-1178.
Out of 72 articles in the same issue, our paper on flexible robotic sensing, entitled “Electromagnetic Positioning for Tip Tracking and Shape Sensing of Flexible Robots”, was featured on the cover of IEEE Sensor Journal, (Volume:15 , Issue: 8, Issue Date: Aug. 2015).
Nasopharynx cancer, or nasopharyngeal carcinoma (NPC), is a tumor that originates in the nasopharynx, the uppermost region of the pharynx where the nasal passage and the throat join. It is a common disease occurring to ethnic Chinese people living in or emigrating from southern China; it is also the eighth most frequently occurred cancer among Singaporean men. Traditional posterior nasopharyngeal biopsy using a flexible nasal endoscope has the risks of abrasion and injury to the nasal mucosa and thus causing trauma to the patient. Therefore, the goal of this project is to develop a compact continuum tubular robotic system to achieve collision free nasopharyngeal biopsy.
Fig.1 Illustration of the proposed CTR for nasopharyngeal biopsy.
Approaches
We developed a compact CTR which is 35 cm in total length, 10 cm in diameter, 2.15 kg in weight, and easy to be integrated with a robotic arm to perform more complicated operations.
Fig.2 The proposed continuum tubular robot
Fig.3 Compact and light weight CTR integrated with a positioning arm for
better conducting surgery
We also developed a 3D printed biopsy needle to equip our robot for transnasal biopsy procedure.
Fig.4 3D printed biopsy needle for transnasal biopsy
The workspace of the robot was analyzed to determine optimized tube parameters.
Fig.5 Workspace comparison for 3-DOF CTR with three initial configurations.
Top: all the outstretched part of the inner tube exposes; Middle: the outstretched part of the inner tube is partially covered by the outer tube; Bottom: the outstretched part of the inner tube is totally covered by the outer tube.
Further more, by using an electromagnetic tracking system, we are able to build a navigation system with shape reconstruction for the tubes.
Fig.6 Shape reconstruction using 3-order Bézier curve fitting
Fig.7 Sensing by EM tracker
Fig.8 Navigation interface
Results
Three groups of experiments were carried out. The first group is to tele-operate the robot to follow a linear path and a circular path. We found that the path following accuracy was about 2 mm.
Fig.9 Tele-operating the robot to follow a linear path and a circular path
Fig.10 Accuracy of the robot following the predefined paths
The second group is to validate the shape reconstruction algorithm. The accuracy of the results is about 1 mm.
Fig.11 Reconstruction setup
Fig.12 Reconstruction error
In the last group of experiments, the robot was tested in a biopsy procedure on a cadaver. The feasibility of the proposed robotic system was validated.
Fig.13 Cadaver experiment setup
Fig.14 Cadaver experiment process
People Involved
Research Fellow: Liao Wu
PhD Student: Keyu Wu
FYP Student: Li Ting Lynette Teo
Intern Student: Jan Feiling and Xin Liu
Project Investigator: Hongliang Ren
Publications
[1] Liao Wu, Shuang Song, Keyu Wu, Chwee Ming Lim, Hongliang Ren. Development of a compact continuum tubular robotic system for nasopharyngeal biopsy. Medical & Biological Engineering & Computing. 2016.
[2] Keyu Wu, Liao Wu, Hongliang Ren. Motion planning of continuum tubular robots based on features extracted from statistical atlas. In: Proceedings of 2015 IEEE International Conference on Intelligent Robots and Systems (IROS 2015).
[3] Keyu Wu, Liao Wu, Chwee Ming Lim, Hongliang Ren. Model-free image guidance for intelligent tubular robots with pre-clinical feasibility study: towards minimally invasive trans-orifice surgery. In: Proceedings of 2015 IEEE International Conference on Information and Automation (ICIA 2015). ( best paper finalist)
[4] Benedict Tan, Liao Wu, Hongliang Ren. Prototype development of a handheld tubular curvilinear robot for minimally invasive surgery. In: The 11th Asian Conference on Computer Aided Surgery (ACCAS 2015).
[5] Keyu Wu†, Liao Wu†, Hongliang Ren. An image based targeting method to guide a curvilinear concentric tube robot. In: Proceedings of 2014 IEEE International Conference on Robotics and Biomimetics (ROBIO 2014). Bali, Indonesia, 2014: 386-391 († equally contributed author).
A constrained tendon-driven serpentine manipulator is developed in our previous work, high level intelligence is expected to make manipulator system working autonomously. Considering the limited and confined working space of surgical operation, this project is aim to develop novel motion planning techniques for our surgical tendon-driven serpentine manipulator, which is expected to assist surgeon operation more accurate and convenience.
Approaches
In the clinical environment, chances are high that the manipulator may bump into neighboring tissue and organs and cause additional damages. For medical manipulator working within human body, optimal and accurate trajectory planning is the key enabler of surgical security because of any additional damage to proper functioning organs is intolerable. Moreover, the surgery usually has many additional disturbances including breathe, physical hand tremor and tiny displacement of the organs. These uncertainties require the planning algorithm to have good robustness to avoid damaging the proper functioning organs. As the most important factor that can lower the risk of additional damages, less sweeping area of manipulator motion results in more disturbances tolerance capability. Especially when energy cost of different planned trajectories are in the same level, the less sweeping area of whole manipulator body becomes more attractive for physician and reduce unpredictable risks in transoral procedure applications. Therefore, we propose a three dimensional neural dynamic planning algorithm which introduces sweeping area as a very important factor in neural stimulation propagation.
The three dimensional neural network is show in Figure 1, each neuron is connected with its adjacent 26 neurons. In our minimum planning model, the start state with highest activity propagates stimulation to whole network through connective weight. On the other hand, the configuration parameters representing obstacle collision hold the lowest value and do not connect with neighboring neurons. As a result, the neural stimulation spread as water ran down from a height place in planning neural dynamic field building, which is shown in Figure 2. Simultaneously, the robot starts from target state and looks for the highest state as climbing mountain. When robot reach the start state configuration parameter, the planning is finished. Then, the final motion sequence will be obtained by reversing the planned trajectory, which is from start state to target state.
Fig.1 Three dimensional neural dynamic model
Fig.2 Neural dynamic field, which stimulation spread from start state to whole map.
Results
At first, we test our planning algorithm in representative simulation scenarios and compare with other famous planning algorithms, such as traditional neural dynamic planning algorithm, RRT and RRT*.
(a) Minimum sweeping area planning algorithm
(b) Neural dynamic algorithm
(c) RRT*
Fig.3 Simple case: a comparison simulation on a simple scenario consisting of two obstacles in the middle of the map is used for first test. The tendon-driven serpentine manipulator is left bended at the beginning and expected to reach the upper right target point. The manipulator is presented by green color (free bending segments) and red color (constrained bending segments). The distal tip trajectory is presented by a dash line. The performance of different planning algorithms in terms of sweeping area and obstacle avoidance ability are shown. Particularly, the area of green parts can be seen as the sweeping area of manipulator approximately
(a) Minimum sweeping area planning algorithm
(b) Neural dynamic algorithm
(c) RRT*
Fig.4 Complex case: a comparison simulation on a complex scenario consisting of four obstacles are used for test. The
tendon-driven serpentine manipulator is left bended at the beginning and expected to reach the target point surrounded by three obstacles in the upper right area. The differences in sweeping area and obstacle avoidance ability are shown obviously among different algorithms guidance.
(a) Minimum sweeping area planning algorithm
(b) Neural dynamic algorithm
(c) RRT*
Fig.5 Tubular case 1: A virtual tubular clinical map in simulation is conducted in advance, where target is at the right side sub-branch. The tendon-driven serpentine manipulator is straight on the bottom of the tubular at the beginning.
(a) Minimum sweeping area planning algorithm
(b) Neural dynamic algorithm
(c) RRT*
Fig.6 Tubular case 2: The same virtual tubular clinical map as Tubular case 1, but target is at the left side sub-branch and start point at the right side sub-branch. The tendon-driven serpentine manipulator is expected to move from right sub-branch to left one as manipulator moving in clinical operation.
Moreover, experiments are conducted in environments built by Lego blocks, where obstacle configurations are similar to simulation cases. In this experiment stage, the tendon-driven serpentine manipulator is expected to execute same motion sequences that are generated from complex case, tubular case 1 and tubular case 2 in simulation studies. The experimental results in phantom test are shown in video 2 (Phantom test on Lego bricks).
Finally, a preliminary transoral trials on cadaver human head is conducted to at Khoo Teck Puat Advanced Surgery Training Centre (ASTC), National University of Hospital, Singapore. The panorama of cadaver transoral experiments can be found in video 3. The operations on compute and corresponding softwares are shown in video 4. The experimental data are ploted by MATLAB, and four algorithms comparisons are shown in video 5.
[1] Yanjie Chen, Zheng Li, Wenjun Xu, Hang Zhong, Yaonan Wang and Hongliang Ren, “Minimum Sweeping Area Motion Planning for Flexible Serpentine Surgical Manipulator with Kinematic Constraints”, IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS 2015), Accepted.
[2] Yanjie Chen, Wenjun Xu, Zheng Li, Shuang Song, Chwee Ming Lim, Yaonan Wang, and Hongliang Ren, “Safety-Enhanced Motion Planning with Minimum Sweeping Area for Flexible Surgical Manipulators using Neural Dynamics”, IEEE Transactions on Cybernetics, Submitted.
Videos
-Support powerpoint.
-Phantom test on Lego bricks.
-The panorama of cadaveric transoral experiments.
-The compute vision and corresponding software during clinical experiments.
