Body-Attached Soft Robot for Ultrasound Imaging

Project Goals

Ultrasound imaging procedures are deemed as one of the most convenient and least invasive medical diagnostic imaging modalities and have been widely utilized in health care providers, which are expecting semiautomatic or fully-automatic imaging systems to reduce the current clinical workloads. This paper presents a portable and wearable soft robotic system which has been designed with the purpose of replacing the manual operation to cooperatively steer the ultrasound probe. This human-compliant soft robotic system, which is equipped with four separated parallel soft pneumatic actuators and is able to achieve movements in three directions. Vacuum suction force is introduced to attach the robot onto the intended body location. The design and fabrication of this soft robotic system are illustrated. To our knowledge, this is the first body-attached soft robot for compliant ultrasound imaging. The feasibility of the system is demonstrated through proof-of-concept experiments.


Developing a wearable soft robotic system (Figure 1), which is capable of mimicking the procedure of probe steering and optimizing the contact force and angle according to the specific conditions, has great significance of reducing the reliance of the ultrasound imaging on the experience of operators and obtaining images with high quality.

People Involved

PhD Student: Xiaoyi Gu
FYP Student: Koon Lin Tan
Project Investigator: Hongliang Ren

Related Publications

Ren, H.; Gu, X. ; Tan, K. L. Human-Compliant Body-Attached Soft Robots Towards Automatic Cooperative Ultrasound Imaging 2016 20th IEEE International Conference on Computer Supported Cooperative Work in Design (CSCWD 2016), IEEE, 2016, –

FYP: Towards Magnetic Actuated Drug Delivery

Project Goals

The objectives of this project are to design and evaluate the performance of an electromagnetic actuated (EMA) drug delivery system and explore the related issues.


The EMA system consists of magneto-responsive microcapsules as drug carriers, a coil system with controlled currents flowing through, as well as a tracking algorithm for close loop feedback control.
The magneto-responsive and thermal sensitive microcapsules are prepared through an encapsulator. The properties can be further utilized for controlled drug release. The encapsulated microbubbles are prepared based on a gas foaming technique for enhancing the ultrasound imaging contrast.
The coil system consists of 2 Helmoholz coil pairs and 2 Maxwell coil pairs are fabricated with printed aluminum skeleton and copper wires. A current control system including 3 DC motor governors and a USB to RS485 converter are added to realize programmable current control. Hence, the magnetic fields generated by the coils are controlled by the signals sent by the computer. Figure 1 shows the principle of actuation over the microcapsules.
Fig. 1: Principle of Magnetic Actuation over the Microcapsules


Figure 2 shows the preliminary set up for actuation over microparticles within the region of interest.
Fig. 2 Setup for Microparticles Actuation
Microcapsules with evenly distributed magnetic stripes have been fabricated. The stripes make the spherical microcapsules asymmetric so that their locomotion control is directed. Alignment and movement of the microcapsules are observed in the EMA system under DC output, while rotation is observed under sinusoidal output current.


Fig. 3 Microcapsules with magnetic CI strips. Scale bar: 200μm.
Fig.4 Magnetic actuation with (A)small cylindrical magnet and (B)magnetic microcapsules

People Involved

Staff: Shen Shen, Song Shuang and Zhu Jingling
PIs: Ren Hongliang and Li Jun

Experiment Videos

Presentations and Publications

1.Shen Shen, Shuang Song, Jingling Zhu, Max Q-H Meng, Jun Li and Hongliang Ren, Preliminary Design towards a Magnetic Actuated Drug Delivery System, 7th IEEE International Conference on Cybernetics and Intelligent Systems and the 7th IEEE International Conference on Robotics, Automation and Mechatronics, 2015.

Poster in BME Showcase 2015


Tracking Magnetic Particles under Ultrasound Imaging using Contrast-Enhancing Microbubbles


Magnetic microbubbles which can be controlled by an external magnetic field have been explored as a method for precise and efficient drug delivery. In this paper, a technique for the fabrication of microbubble encapsulated magnetic spheres is presented. The resultant magnetic spheres were subsequently imaged using ultrasound and the encapsulated microbubbles proved to appear as bright spots and resulted in enhanced ultrasound image contrast, as compared to the solid magnetic spheres which appeared dull. A tracking algorithm was then developed for the tracking of the magnetic microbubbles based on optical flow tracking. Further development of the magnetic microbubbles and tracking algorithm can lead to future use of the tracking algorithm in the case of in vivo injection of the magnetic microbubbles.


1. Loh Kai Ting, Ren Hongliang and Li Jun, Tracking Magnetic Particles under Ultrasound Imaging using Contrast-Enhancing Microbubbles, The 11th Asian Conference on Computer Aided Surgery, 2015.

Poster in BME Showcase 2015

KT Poster Final Printed


This is a local copy of the website:

Home-Based Self-Administered Nasopharynscopy

Nasopharynscope is a valuable tool in diagnosing Nasopharyngeal carcinoma in patients since 84% of patients display ulcerations.

Aim: To provide a home-based, affordable and easy-to-use diagnosing kit for detecting Nasopharyngeal Carcinoma

Key features of 5th Generation

  • Clear viewing with a specially designed camera lens
  • Secure extension and contraction lock
  • Tight fit between nylon strings to ensure good power transmission 
  • Diamond cuts to enhance bending capabilities
  • Optical zoom of up to 5 mm due to shooting mechanism

Bending Capability: >90 degrees

Extension and contraction to evade obstacles

Overall Demonstration of Bending and Zooming capability


  • Outer Diameter: 7 mm
  • Length of extension portion:25 mm
  • Length of bending segment: 25mm
  • Minimum inserted length: 11-15 cm
  • Gear box: 30 by 20 by 30mm
  • Bending angle: 90 degrees bend per side
  • Distal tip mechanism: Optical zoom
  • Material: Polyurethane (Biocompatible)
  • Stent design: Flexibility
  • Flexible guiding tube

Technical Advantages

  • Large bending angle
  • Extending the camera using the spring mechanism to obtain better optical viewing up to 5mm
  • Endoscope is very flexible with the stent design
  • Able to control the bending of the body segments using cable driven mechanism
  • The bending of the endoscope at the entrance can be controlled flexibly by the guiding shaft


From Left: Mr Teo Jing Chun, Dr Ren Hongliang, Mr Un Weiyang, Miss Soh Yan Bing, Mr Ong Jun Hao Edmund
Foreground: Mr Yeow Bok Seng

Nasopharyngeal Carcinoma Surveillance

Project Goals:

Nasopharyngeal carcinoma (NPC) is a tumor arising from the epithelial cells that cover the surface and line the nasopharynx. The concern about NPC in our studies is that it is more common in regions in East Asia and Africa, specifically Southeast Asia. Due to the high tendency for NPC to develop into metastatic dissemination, about 30- 60% of locally advanced patients will develop distant metastasis and die of disseminated disease. Thus this implies that apart from early diagnosis, it is also of paramount importance to locally monitor for the recurrence of NPC or the development of distant metastasis. Therefore, there is a need for a patient-operated, in-vivo surveillance system.


The approach for this project is that it has to be a remote surveillance system that is able to monitor the growth of the tumor in the nasopharyngeal region independently by the patient. The design requirements are as follows below:
1. The device must be made of medically approved materials that are mechanically strong enough to withstand 1 to 2 years of constant use. This is because the average time period for NPC surveillance spans to around 2 years.
2. The device must be durable so as to last the entire time period of use.
3. The device must house a camera module, which is rotatable to the minimum of 90 degrees, such that it can accurately pan throughout the entire nasopharynx region.
Additional aims were also realized in the device design as follows:
1. The device must be made in an economically feasible manner, such that it is able to reduce medical costs as much as possible.
2. The device outcome must be similar to any other method of NPC surveillance so that the quality of the monitoring system is not compromised.
3. The device must be patient-administered, meaning that the patient is able to deploy and use the device without the assistance of medical personnel. This is so as to decrease patient dependence on the healthcare system and also an attempt to decrease the burden on clinicians.
4. The device must be as safe and hassle-free to the patient as possible.

Results and Remarks

The first prototypical device comprises of the following components:
1. Camera Module
2. Arm Head and Hook
3. Arm
4. Wheel and Cover
5. Handle
The camera module houses a mini-camera, which will be able to obtain imaging output from within the nasopharyngeal region after its deployment into the nasal cavity. This housing contains two axles on either side, to enable the rotational mechanism. The camera module is then attached to the arm head, which clicks the axle of the camera into place.
The arm head also contains a hook, whose purpose is to ensure the stability of the device once deployed to the site of the vomer bone. The hook will allow the easy positioning of the camera module to the site and also increase the stability of the device during image capture. This decreases the possibilities of blurred imaging outputs.
The rotational mechanism is mainly powered by three components: (i) the camera module and axle, (ii) the arm, and (iii) the wheel. A thin piece of nylon wire is first threaded through the camera module, then threaded down the tunnels in the arm and finally around the wheel. The rotation mechanism works when the wheel is turned in either direction.
The results of the design verification experiments show that the maximum force required to fracture the Veroclear tip and head are 5.1N and 15.6N respectively. The maximum tensile strength of nylon is 262.4 MPa. The arm can bend to a maximum of 7.3 cm with a force of 15.3N before fracture. FEA shows that even under an exaggerated maximum loading of 100N, the device does not fracture.
The device is able to capture images from within the nasopharyngeal region from the mini camera. This was done by inserting the device through the nasal passageway of a phantom skull and the tumor is shown by the piece of BluTack.
The forces obtained for fracture and/or bending of the Veroclear arm are well beyond the acceptable range of forces as set by the acceptance criteria. This shows that the Veroclear arm passes the first stage of verification testing to determine is safety and suitability in this design. However, to add further to the safety of the device, it is aimed that the final device will be made of a much more mechanically strong material, which is also medically approved: 316L Stainless Steel. From the extrapolated calculations and research of 316L Stainless Steel through FEA, it can be concluded that 316L Stainless Steel is also a good, if not better choice of material for the manufacturing of this device due to its excellent mechanical strength and durability.
Video to be uploaded

People Involved

Undergraduate Students: Neerajha Ram, Khor Jing An, Paul Ng, Ong Jun Shu, Anselina Goh
Advisor: Dr. Hongliang Ren


Awarded the MOST ELEGANT DESIGN INSTRUMENTATION AWARD at the BN3101 Presentations 2013.

FYP: Surgical Tracking With Multiple Microsoft Kinects

FYP Project Goals

The aim of this project is to perform tracking of surgical instruments utilizing the Kinect sensors. With the advances in computing and imaging technologies in the recent years, visual limitations during surgery such as those due to poor depth perception and limited field of view, can be overcome by using computer-assisted systems. 3D models of the patient’s anatomy (obtained during pre-operative planning via Computed Tomography scans or Magnetic Resonance Imaging) can be combined with intraoperative information such as the 3D pose and orientation of surgical instruments. Such a computer-assisted system will reduce surgical mistakes and help identify unnecessary or imperfect surgical movements, effectively increasing the success rate of the surgeries.
For computer-assisted systems to work, accurate spatial information of surgical instruments is required. Most surgical tools are capable of 6 degrees of freedom (6DoF) movement, which includes the translation in the x, y, z- axes as well as the rotation about these axes. The introduction of Microsoft Kinect sensor raises the possibility of an alternative optical tracking system for surgical instruments.
This project’s objective would be the development of an optical tracking system for surgical instruments utilising the capabilities of the Kinect sensor. In this part of the project, the focus will be on marker-based tracking using the Kinect sensor.


  • The setup for the tracking of surgical instruments consists of two Kinects placed side by side with overlapping field of views.
  • The calibration board used to find out the intrinsic camera parameters as well as the relative position of the cameras. This allows us to calculate the fundamental matrix, which is essential for epipolar geometry calculations used in 3D point reconstruction. (a) without external LED illumination (b) with LED illumination. The same board is used for RGB camera calibration.
  • Seeded region growing allows the segmentation of retro-reflective markers from the duller background. The algorithm is implemented through OpenCV.
  • Corner detection algorithm: the cornerSubPix algorithm from OpenCV is used to refine the position of the corners. This results in sub-pixel accuracy of the corner position.

Current Results

  • The RMS error for IRR and checkerboard tracking ranges from 0.37 to 0.68 mm and 0.18 to 0.37 mm respectively over a range of 1.2 m. Checkerboard tracking is found to be more accurate. Error increases with distance from camera.
  • The jitter for the checkerboard tracking system was investigated and it was found to range from 0.071 mm to 0.29 mm over the range of 1.2 m.
  • (dots) Measurement of jitter plotted against the distance from the left camera. (line) the data is fitted to a polynomial of order 2 to analyze how jitter varies with depth.

People Involved

FYP Student: Andy Lim Yong Mong
Research Engineer: Liu Wei
Advisor: Dr. Ren Hongliang

Related Project

Surgical Tracking Based on Stereo Vision and Depth Sensing


[1] Sun, W., Yang, X., Xiao, S., & Hu, W. (2008). Robust Checkerboard Recognition for Efficient Nonplanar Geometry Registration in Projector-camera Systems. Proceedings of the 5th ACM/IEEE International Workshop on Projector camera systems. ACM.
[2] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2 ed., Cambridge: Cambridge University Press, 2003.
[3] Q. He, C. Hu, W. Liu, N. Wei, M. Q.-H. Meng, L. Liu and C. Wang, “Simple 3-D Point Reconstruction Methods With Accuracy Prediction for Multiocular System, “IEEE/ASME Transactions on Mechatronics, vol. 18, no. 1, pp. 366-375, 2013

Ablation Planning in Computer-Assisted Interventions

Project Goals

Tumor ablation is the removal of tumor tissue and is considered as one type of minimally invasive interventions. It can be performed using techniques like cryoablation, high-intensity focused ultrasound (HIFU), and radiofrequency ablation (RFA). These techniques rely on minimally invasive principles to ablate tumor tissues, without having to directly expose the target regions to the environment. It has been widely noted that the success of a tumor ablation procedure hinges greatly on its pre-operative planning, which is often assisted by computational interventions. The proposed ablation planning system in this paper focuses mainly on the radiofrequency ablation (RFA) of hepatic tumors. This project is to develop computational optimization algorithms to plan optimal ablation delivery. Ablation planning systems are necessary to model the 3D interventional environments, identify feasible needle insertion trajectories and deploy ablating electrodes, while avoiding many critical structures.


Genetic Algorithm (GA) was used as it can be designed to consider the multi-objective nature of a tumor ablation planning system. The proposed ablation planning system is designed based on the following objectives: to achieve complete tumor coverage; and to minimize the number of ablations, number of needle trajectories and healthy tissue damage. These objectives are taken into account using an optimization method, Genetic Algorithm (GA). GA is capable of generating many solutions within a defined search space, and these solutions can be selected to undergo evolution based on a quantified value given by a fitness function. An exponential weight-criterion fitness function is used to represent the multiple objectives such as the number of ablation spheres, the number of trajectories, the covariance, and the coverage volume.

Current Results

The proposed mathematical protocol to determine the range of ablation spheres required to achieve complete tumor coverage is feasible to be used as a reference in the context of tumor ablation planning. The following figure shows how tumor coverage changed when trajectory optimization was considered: 0% tumor coverage (top), 100% tumor coverage with [ablation radius]=15 and [number of spheres]=3 (orange spheres) (bottom).


  • Ren, H.; Guo, W.; Ge, S. S. & Lim, W. Coverage Planning in Computer-Assisted Ablation Based On Genetic Optimization Computers in Biology and Medicine, in press, 2014
  • Lim, W. & Ren, H. Cognitive Planning Based on Genetic Algorithm in Computer-Assisted Interventions CIS-RAM 2013, 6th IEEE International Conference on Cybernetics and Intelligent Systems (CIS) and the 6th IEEE International Conference on Robotics, Automation and Mechatronics (RAM), 2013

People Involved

FYP Student: Wan Cheng LIM
Graduate Student: Weian GUO
Advisor: Dr. Hongliang REN


[1] C. Baegert, C. Villard, P. Schreck, L. Soler, and A. Gangi, “Trajectory optimization for the planning of percutaneous radiofrequency ablation of hepatic tumors,” Computer Aided Surgery, 12(2): pp. 82-90, March, 2007.
[2] Z. Yaniv, P. Cheng, E. Wilson, T. Popa, D. Lindisch, E. Campos-Nanez, H. Abeledo, V. Watson, and F. Banovac, “Needle-Based Interventions With the Image-guided Surgery Toolkit (IGSTK): From Phantoms to Clinical Trials,” IEEE Trans. on Biomedical Engineering, vol. 57, no. 4, April, 2010.
[3] G. D. Dodd, M. C. Soulen, R. A. Kane, T. Livraghi, W. R. Lees, Y. Yamashita, A. R. Gillams, O. I. Karahan, H. Rhim. “Minimally invasive treatment of malignant hepatic tumors: At the threshold of a major breakthrough,” RadioGraphics, vol. 20, no. 1, January-February, 2000.
[4] C. Rieder, T. Kroger, C. Schumann, and H. K. Hahn, “GPU-Based Real-Time Approximation of the Ablation Zone for Radiofrequency Ablation”, IEEE Trans. On Visualization and Computer Graphics, vol. 17, no. 12, pp. 1812-1821, December, 2011.

Related Poster