INI: Electromagnetic needleless injector with halbach array towards intravitreal delivery

Electromagnetic needleless injector with halbach array towards intravitreal delivery

IEEE Access 2017

Abstract

The fear of pain and needles deter some patients from seeking intravitreal treatment, which drives our group to develop an needleless device for performing intravitreal injections. A prototype for an electro-magnetically actuated needleless injector, based on Halbach arrays, is described and characterized in a lab setting. The implication of the prototype for needleless ocular drug delivery is investigated. This investigation is intended to eventually improve drug delivery of glaucoma medication enabling safe needleless approaches. We detail the design aspects of the injector and characterized the device with custom-made phantoms. It was observed that, despite delivering the drug bolus to the center, the viscous vitreous phantom indicated vorticities similar to counter rotating vortex pairs, which could damage the retina. The observed peak velocity during the phantom experiments was 6.1mm/sec at the retinal layer, indicating that the delivery bolus can impart shear forces to the retina via the vitreous.

 

Design and Characterization of an Electromagnetic Needleless Injector Based on Halbach Design Towards Intravitreal Delivery

Supporting videos

> 1-minute short video demo

> full video demo

Appendix

The results comparing the measured Fz, the force in the normal direction to the sensor surface, and Mag, the magnitude of the force vector, across the various depths of injection and at the two voltage points.

fig24fzmag15v20v
Fig 24.  Left (15V) Right (20V); Top (Fz) Bottom (Mag)

Comparison between the four depths did not show significant differences:

Changing the distances between the sensor surface and the injection nozzle does not appear to have a distinguishable relationship. Note: each data plot is the average from 3 data samples. This is indicative of the following; regardless of the size of the orbital, the peak force observed is going to be dependent on the injectant properties. We note that more experiments should be done for characterization of distances between 20mm and 12.5mm, we did not test this range due to concerns over damaging the sensitive ATInano17, and hence the initial expectation of measuring 20 to 35mm and anticipating an extrapolate-able relationship. The rational for these distances assumes an orbital diameter of 25mm and the drug is delivered to the center at 12.5mm. The 25mm distance is likely to be near the retinal layer and 20mm would be in between.

Test in phantom chamber:

The dimension of the vitreous phantom chamber is a cylinder with height 17mm and diameter of 25mm. This relates to a volume of approximately 8345uL. This is significantly larger than the vitreous volume in human eyes. Assuming the human vitreous to be 4000uL, the height of our cylinder should be 8mm. In such a scenario, the rotational symmetry in the spherical orbital is lost and hence the flow will be more a keen to a 2D planar flow rather than a 3D flow. Our selection of 17 mm was a compromise between maintaining the 3D flow while allowing an observable 2D slice of the injection profile. In addition, the selection of the 10.9 mm by 17 mm access point is arbitrary. Its purpose was to allow ease of filling and removal of vitreous phantom while maintaining the spherical shape as much as possible. The current dimensions relates to a chord at 11.25mm from the center of the circle, thus maintaining at least 95% of the diameter.

 

Related Publication

Electromagnetic needleless injector with halbach array towards intravitreal delivery (in press) IEEE Access, 2017

Simultaneous Robot-World, Sensor-Tip, and Kinematics Calibration of an Underactuated Robotic Hand with Soft Fingers

IEEE ACCESS. PREPRINT VERSION. ACCEPTED Dec, 2017

Simultaneous Robot-World, Sensor-Tip, and Kinematics Calibration of an Underactuated Robotic Hand with Soft Fingers

Ning Tan, Xiaoyi Gu, and Hongliang Ren Senior Member, IEEE

Abstract—Soft robotics is a research field growing rapidly with primary focuses on the prototype design, development of soft robots and their applications. Due to their highly deformable features, it is difficult to model and control such robots in a very precise compared with conventional rigid structured robots. Hence, the calibration and parameter identification problems of an underactuated robotic hand with soft fingers are important, but have not been investigated intensively. In this paper, we present a comparative study on the calibration of a soft robotic hand. The calibration problem is framed as an AX=YB problem with the partially known matrix A. The identifiability of the parameters is analyzed, and calibration methods based on nonlinear optimization (i.e., L-M method and interior-point method) and evolutionary computation (i.e., differential evolution) are presented. Extensive simulation tests are performed to examine the parameter identification using the three methods in a comparative way. The experiments are conducted on the real soft robotic-hand setup. The fitting, interpolating, and extrapolating errors are presented as well.

Index Terms—Soft robotics, calibration and identification, robotic hand, AX=YB, hand-eye calibration, tendon-driven robot

Force/Torque Sensor for Tele-operated Catheterization Procedures

Abstract

A tele-operated robotic catheterization system can significantly alleviate the surgeons from radiation exposure and fatigue resulted from long standing time with protective suits. Proximal force/torque signals imply the critical information about the contact forces between the catheter and its surrounding structures. This paper presents a compact, cost-effective force and torque sensing device suitable for catheterization procedures to measure the proximal force/torque signals of the input catheter. The device consists of a rotatable and linear retractable mechanism, a laser mouse sensor, and a coil spring. As the stretched, compressed, and twisted values vary due to the sliding joint, the force and torque signals can be computed based on the Hooke’s law. The proposed sensing device has many advantages such as cost-effective, easily miniaturized and customized, and can be extended to the MRI compatible sensors. The experimental results with step response and time-varying loads by comparing to an ATI Nano17 force/torque sensor show that the Root Mean Squared Error (RMSE) for force and torque measurement are 0.042 N and 0.228 mNm respectively.

Video PPT demo

Publications

J. Guo; M. Li; P. Ho & H. Ren Design and Performance Evaluation of a Force/Torque Sensor for Tele-operated Catheterization Procedures IEEE Sensors Journal, 2016, PP, 1-8

Data-driven Learning Intelligent Control for Flexible Surgical Manipulators

Abstract

Automate Surgical Tasks for A Flexible Serpentine Manipulator via Learning Actuation Space Trajectory from Demonstration

Background: Accurate motion control of flexible surgical manipulators is crucial in tissue manipulation tasks. Tendon-driven serpentine manipulator (TSM) is one of the most widely adopted flexible mechanisms in MIS for its enhanced maneuverability in torturous environment. TSM, however, exhibits high nonlinearities and conventional analytical kinematics model is insufficient to achieve high accuracy.
Methods: To account for the system nonlinearities, we applied data driven approach to encode the system inverse kinematics. Three regression methods: Extreme Learning Machine (ELM), Gaussian Mixture Regression (GMR) and K-Nearest Neighbors Regression (KNNR) were implemented to learn a nonlinear mapping from the robot 3D position state to the control inputs.
Results: The performance of the three algorithms were evaluated both in simulation and physical trajectory tracking experiments. KNNR performs the best in the tracking experiments with the lowest RMSE of 2.1275mm.
Conclusions: The proposed inverse kinematics learning methods provide an alternative and efficient way to accurately model the challenging tendon driven flexible manipulator.
Keywords: Tendon-driven serpentine manipulator; surgical robotics; Inverse kinematics; Heuristic Methods

Demo video at:

Publications

  • W. Xu; J. Chen; H. Y. Lau & H. Ren Data-driven Methods towards Learning the Highly Nonlinear Inverse Kinematics of Tendon-driven Surgical Manipulators International Journal of Medical Robotics and Computer Assisted Surgery , 2016, 1-13
  • W. Xu; J. Chen; H. Y. Lau & H. Ren Automate Surgical Tasks for A Flexible Serpentine Manipulator via Learning Actuation Space Trajectory from Demonstration ICRA2016, IEEE International Conference on Robotics and Automation, 2016, –

Body-Attached Soft Robot for Ultrasound Imaging

Project Goals

Ultrasound imaging procedures are deemed as one of the most convenient and least invasive medical diagnostic imaging modalities and have been widely utilized in health care providers, which are expecting semiautomatic or fully-automatic imaging systems to reduce the current clinical workloads. This paper presents a portable and wearable soft robotic system which has been designed with the purpose of replacing the manual operation to cooperatively steer the ultrasound probe. This human-compliant soft robotic system, which is equipped with four separated parallel soft pneumatic actuators and is able to achieve movements in three directions. Vacuum suction force is introduced to attach the robot onto the intended body location. The design and fabrication of this soft robotic system are illustrated. To our knowledge, this is the first body-attached soft robot for compliant ultrasound imaging. The feasibility of the system is demonstrated through proof-of-concept experiments.
softusprobe

Approaches/Results/Video

Developing a wearable soft robotic system (Figure 1), which is capable of mimicking the procedure of probe steering and optimizing the contact force and angle according to the specific conditions, has great significance of reducing the reliance of the ultrasound imaging on the experience of operators and obtaining images with high quality.

People Involved

PhD Student: Xiaoyi Gu
FYP Student: Koon Lin Tan
Project Investigator: Hongliang Ren

Related Publications

Ren, H.; Gu, X. ; Tan, K. L. Human-Compliant Body-Attached Soft Robots Towards Automatic Cooperative Ultrasound Imaging 2016 20th IEEE International Conference on Computer Supported Cooperative Work in Design (CSCWD 2016), IEEE, 2016, –

BIOROB2016 Workshop in Surgical Robotics

Themes

  • Image-Guided Therapy
  • Interventional Devices
  • Human-in-the-loop System
  • Computer-Assisted Surgery
  • Multi-modal sensor fusion
  • Human-Robot Interactions
  • Soft Robotics in Surgery
  • Artificial Intelligence in Robotic Surgery

Date: June 26 (Sunday), 9:30 AM ~ 5:30 PM (BIOROB2016 Program[.pdf])

  • Morning session: 09:30-12:30 (with 30 minutes for each talk and one coffee break)
  • Afternoon Session: 14:30-17:30

no images were found

Venue: NUS CeLS, 28 Medical Dr, Singapore 117456

Speakers and Agenda (Tentative)

  • — 0800-0930 Registration & Coffee
  • 9:30AM, Zheng Li, Chinese University of Hong Kong, Hong Kong
  • 10AM, Chwee Ming Lim, National University Hospital, National University of Singapore, Singapore
  • — 10:30AM-10:45AM, Coffee break
  • 10:45AM, Hongbin Liu, King’s College London, UK
  • 11:15AM, Sebastian Matich, Technische Universität Darmstadt, Germany
  • 12PM, Sarthak Misra, University of Twente, Netherlands
  • — 12:30, Lunch break & SINAPSE Tour
  • 2:30PM, Leonardo De Mattos, Istituto Italiano di Tecnologia, Italy
  • 3PM, Wooram Park, University of Texas at Dallas, USA
  • — 3:30PM-3:45PM, Coffee break
  • 3:45PM, Etsuko Kobayashi, University of Tokyo, Japan
  • 4:15PM, Kevin Cleary, Children’s National Medical Center, USA
  • 4:45PM, Hongliang Ren, National University of Singapore, Singapore
  • — 6PM, Social Event

Topics

lizheng

Zheng Li, Chinese University of Hong Kong, Hong Kong

  • Surgical Robot: from Rigid to Flexible – a mechanical point of view
  • Abstract
    Surgical robots, represented by the da Vinci Surgical System, has been increasingly used in the operating theatres. Benefits of the robot include: increased precision, finer movement, improved dexterity, reduced tremor, better vision, capable of remote-operation, etc. These benefits are attributed to multiple factors, mechanical design, advanced control, imaging, communication, etc. Mechanical design or the structure is fundamental. It defines the reachable workspace as well as dexterity of the end effector. From the structure point of view, current surgical arms could be divided into rigid arms and flexible arms. Rigid arms, like the Endowrist instruments of da Vinci surgical system, contains a rigid slim shaft and a wrist. During the operation the shaft pivots about the trocar to reposition the end effector. The consequence is small internal workspace, large external operation space and possible tear of the incision. Flexible arms, on the other hand, could bend inside the cavity. This increases the workspace and alleviates the need for pivoting. For semi-flexible surgical arms the shaft is still rigid. At the distal end of the shaft is a flexible bending section. They are mostly used in MIS, LESS and SPA surgeries. When the shaft becomes flexible as well, the surgical arm could navigate through natural orifice and performing NOTES. Due to the benefits of flexible surgical arms, much effort has been devoted to developing flexible surgical robots globally. A few flexible surgical robots have received FDA approval. It is believed that in the near future, a lot more would go into the operating theatre. In this presentation, the current status of flexible surgical robots research will be introduced. Also, some of my research on flexible mechanisms will be shared.
  • Biography
    Zheng Li received his B.S and M.S degrees in Mechanical Engineering from Beihang University, China, and his Ph.D. degree in Mechanical and Automation Engineering from the Chinese University of Hong Kong, Hong Kong SAR, in 2007, 2010, and 2013 respectively. Now he is a research assistant professor in the institute of Digestive Disease and Chow Yuk Ho Technology Centre for Innovative Medicine, The Chinese University of Hong Kong. His current research mainly focuses on the topic of medical robots, including flexible surgical robot, active capsule-like robots and bio-inspired robots. He is a member of IEEE and ASME. In the past years, he served as committee member of a number of conferences, such as Robio, ICIA and CCECE, and reviewer of several journals, such as IEEE/ASME T-MECH, Robotics and Biomimetics, Journal of Intelligent and Robotic systems. He is the author of one book, three book chapters, over 30 journal/conference papers, and a number of patents/copyrights. He won the best paper finalist of Robio 2012, CCECE 2015, Conference paper award of CCECE 2015, First prize of 13th Challenge Cup, Champion of 2013 CUHK Professor Charlse K. Kao Student Creativity Awards, etc.

Dr LimChweeMing

Chwee Ming Lim, National University Hospital, Singapore

  • Current status of robotic head and neck surgery in Singapore
  • Abstract
    Robotic assisted head and neck surgery is a new surgical aramentarium in the management of both benign and malignant head and neck disease. Trans-oral robotic surgery (TORS) has been utilized in the resection of early T1/T2 oropharyngeal cancer, as well as removing deep-seated tumors such as parapharyngeal space tumors and supraglottic/hypopharyngeal tumors. Additionally, accessing thyroid nodules or benign tumors in the neck can be made from a remote access incision such as the trans-axillary or retro-auricular (or modified facelift). These new techniques open a new paradigm in the surgical management of head and neck diseases and the current adoption of the technology in an academic tertiary medical centre will be presented.
  • Biography
    Dr Lim graduated from the Faculty of Medicine, National University of Singapore in 1998. He underwent residency in Otolaryngology in Singapore and was awarded the Gold medal for the MRCS (Edinburgh) General Surgery examination in 2003, and subsequently the Gold medal award by the College of Surgeons for the best performing resident in the exit examination organised by the Specialist Training Committee in Otolaryngology in 2009. After completing his specialist training, Dr Lim joined the Department of Otolaryngology – Head and Neck Surgery in NUH as an Associate Consultant. He started his practice and pursued further training in head and neck oncologic surgery at the National University Health System, Singapore. In 2010, he was awarded the Ministry of Health Overseas Training award and pursued a 2-year Head and Neck Oncologic Fellowship at the University of Pittsburgh Medical Center in the United States of America. This fellowship is accredited by the Advanced Training Council of the American Head and Neck Society. During these 2 years, Dr Lim did translational benchwork research in immunotherapy in head and neck cancer, focusing on immunological mechanisms in monoclonal antibody based therapy in cancer and identifying novel immune modifiers in cancer therapy. In his clinical training, Dr Lim was trained in transoral robotic surgery (TORS), minimally invasive video assisted thyroidectomy (MIVAT) and minimally invasive approaches in head and neck surgery, in addition to the major head and neck resections. Upon his return to Singapore in 2012, he was appointed Consultant at the Department of Otolaryngology – Head and Neck Surgery, NUH, and Assistant Professor at the Yong Loo Lin School of Medicine, NUS.

hbliu

Hongbin Liu, King’s College London, UK

  • A catheter robot for semi-autonomous cardiac ablation
  • Abstract
    Catheter ablation is a widely used minimally invasive procedure to treat arrhythmias when medications are unable to restore the normal heart rhythm. However, this is a highly skilled procedure, requiring extensive training, and may take many hours for a single procedure even in the most experienced hands. Robotics technology provides promising solutions to increase the accessibility of this treatment modality, and to assist training. However, existing robotic solutions remain unable to reduce the procedure time notably and the associated high cost precludes their widespread use. To address the above issues, in a new catheter steering robot for ablation procedures are being developed at King’s College London. This talk introduces the design of the catheter robot, the semi-autonomous catheter navigation control, and the contact force estimation based on shape tracking. The initial results show that robot can improve the current catheter ablation procedure in terms of both reducing the procedure time and increasing the ablation accuracy, with cost-effective design features.
  • Biography
    Hongbin Liu is currently a lecturer (Assistant Professor) in the Centre for Robotics Research, Department of Informatics, King’s College London, UK, where he is leading the Robotic Contact Perception Lab. He received his B.S. degree in 2005 from the Northwestern Polytechnique University, Xi’an, China, and received MSc in 2006 and PhD degree in robotics in 2010 both from Kings College London, UK. He is a member of IEEE and Technical Committee Member for IEEE EMBS BioRobotics. His research is focusing on enriching the robot’s perception of during medical interventions, and making use of the augmented perception to enable quicker, safer, procedure. Applications of his research include soft tissue palpation during minimally invasive surgery, interventional cardiology and endoscopy.

SebastianMatich

Sebastian Matich, Technische Universität Darmstadt, Germany

  • A Single Port Robotic System with parallel kinematic arms for rectal surgery
  • Abstract
    Single-Port surgery is an innovative technique and the next step in minimally invasive surgery. Performing complex surgical procedures through only one single incision or a natural orifice is a challenging task that could be simplified by involving surgical telemanipulators. Throughout the last decade, several single-port robots where introduced. While most of these system use cable driven continuum robots to set up the intracorporeal arms the FLEXMIN device uses miniaturized parallel kinematics instead.
    This talk introduces the surgical telemanipulator FLEXMIN and focuses on the design and characterisation of the parallel kinematics. Because the telemanipulator will provide haptic feedback, the results of a first realized sensor concept that uses distally located force sensors to measure the tip force is presented.
    With the manufactured prototype, complex surgical procedures like suturing and knot tying can be performed. Furthermore, it is shown that this approach is capable of applying high payloads exceeding 5 N and generating dynamic movements with speeds of more than 320 mm/s and accelerations beyond 1 G.
    Because of the rigidity, excellent controllability and low latency the manipulator can perform high dynamic movements with the potential to compensate the movements of organs.
  • Biography
    Sebastian Matich received his diploma in Electrical Engineering and Information Technology from Technische Universität Darmstadt, in 2011. He is currently working as research associate at the Institute of Electromechanical Design. His main research topic is the development of small scale parallel kinematic structures for use in surgical robotics. Additional research topics are precision engineering, small drives and sensor design.

Sarthak

Sarthak Misra, University of Twente, Netherlands

  • Steering Flexible Instruments using Fiber Bragg Grating Sensors
  • Abstract
    Flexible minimally invasive surgical instruments can be used to target difficult-to-reach locations within the human body. Accurately steering these instruments requires information about the three-dimensional shape of the instrument. In this talk, we use an array of Fiber Bragg Grating (FBG) sensors to reconstruct the shape of flexible instruments, and to robotically control the instrument. FBG sensors have several advantages over existing imaging modalities, which make them well-suited for use in a clinical environment. First, an array of FBG sensors is integrated on a Nitinol wire (1 mm diameter). A bevel tip is made on the tip of the shape sensing wire, such that a flexible needle is realized which bends during insertion into tissue. This needle with FBG sensors is inserted into biological tissue, and the resulting deflected shape is reconstructed and compared with camera images. Next, the needle is used as a stylet in a novel needle design with a tendon-driven actuated-tip. Kinematic models of the needle are combined with the needle pose reconstructed from FBG sensors to steer the needle in soft-tissue simulants. FBG-based shape sensing has also been used for the control of tendon-driven continuum manipulators. We control the position of the tip of a single-segment manipulator. This is followed by a case where we use FBG sensors to control the shape of a planar manipulator with two segments. In this case, the reconstructed shape was used for both obstacle avoidance while steering the manipulator tip along a pre-defined path. Finally, we demonstrate that continuum manipulators along with FBG sensors can be used for intrinsic force sensing.
  • Biography
    Sarthak Misra joined the University of Twente in 2009. He is currently an Associate Professor in the Department of Biomechanical Engineering within the Faculty of Engineering Technology. He directs theSurgical Robotics Laboratory, and is affiliated with MIRA – Institute for Biomedical Technology and Technical Medicine. He is also affiliated with theDepartment of Biomedical Engineering, University of Groningen and University Medical Center Groningen. Sarthak obtained his doctoral degree in the Department of Mechanical Engineering at the Johns Hopkins University, Baltimore, USA. Prior to commencing his studies at Johns Hopkins, he worked for three years as a dynamics and controls analyst at MacDonald Dettwiler and Associates on the International Space Station Program. Sarthak received his Master of Engineering degree in Mechanical Engineering from McGill University, Montreal, Canada. He is the recipient of the European Research Council (ERC) Starting grant, Netherlands Organization for Scientific Research (NWO) VENI award, Link Foundation fellowship, McGill Major fellowship, and NASA Space Flight Awareness award. He is the co-chair of the IEEE Robotics and Automation Society Technical Committee on Surgical Robotics, and area co-chair of the IFAC Technical Committee on Biological and Medical Systems. Sarthak’s broad research interests are primarily in the area of applied mechanics at both macro and micro scales. He is interested in the modeling and control of electro-mechanical systems with applications to medical robotics.

Leonardo

Leonardo De Mattos, Istituto Italiano di Tecnologia, Italy

  • Robot-Assisted Laser Microsurgery: Overcoming Translational Barriers
  • Abstract
    This talk will present an overview of technology research, development and innovation activities in robot-assisted laser microsurgery at IIT. Lasers are being increasingly used in operating rooms as precision surgical tools for delicate surgeries on both soft and hard tissue. Applications range from fetal surgery to orthopedics, with significant examples in ophthalmology and laryngology. However, current clinical technologies do not properly support the accurate and intuitive use of lasers for high-precision high-quality microsurgeries, imposing severe challenges to the operations. To improve on this, new surgical robotic technologies and capabilities are being developed at IIT. Robot assistance is particularly suited and desired for laser microsurgery applications. Recent results prove they can augment the sensing and actuation capabilities of microsurgeons, allowing significant improvements in terms of surgical site accessibility and visualization, laser controllability, safety and surgical quality. Nevertheless, translating robotics research results to clinical practice is a long and challenging process in itself, involving a continuous refinement process with many system redesigns and simplifications. This talk will conclude presenting the IIT experience in this last phase, which is bringing microsurgical robotic systems closer to clinical trials.
  • Biography
    Leonardo S. Mattos is a Team Leader at the Istituto Italiano di Tecnologia (IIT) in Genoa. His research background includes robotic surgery, robot-assisted laser microsurgery, user interfaces, mixed and augmented reality, systems integration, automation, usability analysis, computer vision, micromanipulation, and mobile robots. Leonardo received his Ph.D. degree in electrical engineering from the North Carolina State University (NCSU, USA), where he worked as research assistant at the Center for Robotics and Intelligent Machines (CRIM) from 2002 until 2007. Leonardo has been a researcher at the IIT’s Department of Advanced Robotics since 2007. He is currently Head of the Biomedical Robotics Laboratory, leading a group of 12 researchers. Leonardo was the PI and coordinator of the EC funded project μRALP, a very successful project dedicated to the development of new tools and systems for robot-assisted laser microsurgery, which was rated with the highest grade of Excellent at its final review.

wpark_large[1]

Wooram Park, University of Texas at Dallas, USA

  • Robotic methods for insertion of flexible needles
  • Abstract
    A flexible needle has been recently introduced and improved for drug delivery and diagnosis. The flexible needle is made with a bevel tip so that it forms a bending curve as it is inserted into soft tissue. This enables to steer the needle by rotating and inserting. For the automatic control, the kinematic model and path planning for the needle insertion are developed. To capture and utilize the uncertainty in the needle insertion, a stochastic modeling technique is applied to a nonholonomic model for the flexible needle. The nonholonomic stochastic model is then used for the probability-based path planning. In this talk, a novel design idea and a new planning method for improved needle manipulation are also discussed.
  • Biography
    Wooram Park is an assistant professor in the Department of Mechanical Engineering at University of Texas at Dallas. His research mainly concerns medical robots, computational structural biology and human kinetics. Prior to joining UT Dallas in 2011, he was a postdoctoral fellow in Mechanical Engineering at Johns Hopkins University. He received his PhD degree in Mechanical Engineering from Johns Hopkins University in 2008. He also received the B.S.E. and M.S.E. degrees in Mechanical Engineering from Seoul National University, Seoul, Korea, in 1999 and 2003, respectively. He received the IEEE Transactions on Automation Science and Engineering Best Paper Award in 2015. He was also a recipient of Creel Family Fellowship at Johns Hopkins University in 2007. He is a member of IEEE and ASME.

kobayashi

Etsuko Kobayashi, University of Tokyo, Japan

  • Medical robot and navigation system using intraoperative information system for minimally invasive surgery
  • Abstract
    Endoscopic surgery is now very popular as a form of minimally invasive surgery in which surgeons perform the operation using forceps and an endoscope through trocar. However, while this surgery has lots of merits, there are disadvantages that the working flexibility and field of view are limited. To overcome this, we have developed information based robotic system. Intra-operative sensing information offers the right target position during surgery and also surgeon can confirm the reliability of the treatment. Then the robot can be controlled based on the accurate prediction of the biological response against the operation.
    In this talk, we will present the our recently developed robotic system and navigation system using intraoperative sensing information including vascular information based on ultrasound image and image mapping system for fetus surgery and force measurement system.
  • Biography
    Prof. Etsuko Kobayashi is currently an associate professor of department of precision engineering, school of engineering, the University of Tokyo. She received the B.S., M.S., and Ph.D. degree in precision machinery engineering from the University of Tokyo, Tokyo, Japan in 1995, 1997 and 2000, respectively. Her research interests include medical robotics, surgical navigation system and biomedical instrumentation. She has won 3rd prize of German Innovation Award and Young Investigator Award of Japan Society of Computer Aided Surgery in 2009. Now she is in charge of several research grants from Japan in Medical robotics and Computer Aided Surgery

Cleary_K

Kevin Cleary, Children’s National Medical Center, USA

  • MRI Compatible Robotics for Pediatrics
  • Abstract
    MRI compatible robotics as an area of increasing interest within the medical robotics community. In this talk I will review some of the work done to date and then present our work with MRI compatible robotics for pediatrics. Two clinical applications will be presented: shoulder arthrography and long bone biopsy.
  • Biography
    Kevin Cleary PhD is the Technical Director of the Bioengineering Initiative in the Sheikh Zayed Institute for Pediatric Surgical Innovation at Children’s National Health System in Washington DC. He is internationally known for his work in medical robotics and image-guided interventions. He currently leads a team of researchers developing biomedical devices for minimally invasive pediatric procedures.

RenHongliang-BME-NUS

Hongliang Ren, National University of Singaore, Singapore

  • Recent Progresses in Flexible Surgical Robotics and Navigation
  • Abstract
    Minimally invasive Robotic Surgery (MIRS) is emerging as a new paradigm for a wide range of surgical and interventional procedures. Common benefits of MIRS include shorter hospital stay, reduced trauma, better cosmesis, etc. Robotic devices used in MIRS are transforming from traditional rigid manipulators to flexible manipulators, which enable surgeons a wider range of operations with less trauma. Intelligence, flexibility and compliance are important features for flexible robotic systems in minimally invasive surgeries. We will discuss recent progresses while investigating tele-operated intelligent flexible surgical robotic systems. Various operation modes and intelligent navigation approaches will be discussed to cater for different clinical needs including semi-automatic teleoperation mode and full automatic image guidance. This talk will give a brief introduction to these ongoing research topics of intelligent and compliant robotics in surgical applications.
  • Biography
    Dr. Hongliang Ren is currently an assistant professor and leading a research group on medical mechatronics in the Biomedical Engineering Department of National University of Singapore (NUS). He is an affiliated Principal Investigator for the Singapore Institute of Neurotechnology (SINAPSE) and Advanced Robotics Center at National University of Singapore. Dr. Ren received his PhD in Electronic Engineering (Specialized in Biomedical Engineering) from The Chinese University of Hong Kong (CUHK) in 2008. After his graduation, he worked as a Research Fellow in the Laboratory for Computational Sensing and Robotics (LCSR) and the Engineering Center for Computer-Integrated Surgical Systems and Technology (ERC-CISST), Department of Biomedical Engineering and Department of Computer Science, The Johns Hopkins University, Baltimore, MD, USA, from 2008 to 2010. In 2010, he joined the Pediatric Cardiac Biorobotics Lab, Department of Cardiovascular Surgery, Children’s Hospital Boston & Harvard Medical School, USA, for investigating the beating heart robotic surgery system. Prior to joining NUS, he also worked in 2012 on a collaborative computer integrated surgery project, at the Surgical Innovation Institute of Children’s National Medical Center, USA. His main areas of interest include Biomedical Mechatronics, Computer-Integrated Surgery, and Dynamic Positioning in Medicine.

More information about BIOROB 2016 at Singapore

FYP: Towards Magnetic Actuated Drug Delivery

Project Goals

The objectives of this project are to design and evaluate the performance of an electromagnetic actuated (EMA) drug delivery system and explore the related issues.

Approaches

The EMA system consists of magneto-responsive microcapsules as drug carriers, a coil system with controlled currents flowing through, as well as a tracking algorithm for close loop feedback control.
The magneto-responsive and thermal sensitive microcapsules are prepared through an encapsulator. The properties can be further utilized for controlled drug release. The encapsulated microbubbles are prepared based on a gas foaming technique for enhancing the ultrasound imaging contrast.
The coil system consists of 2 Helmoholz coil pairs and 2 Maxwell coil pairs are fabricated with printed aluminum skeleton and copper wires. A current control system including 3 DC motor governors and a USB to RS485 converter are added to realize programmable current control. Hence, the magnetic fields generated by the coils are controlled by the signals sent by the computer. Figure 1 shows the principle of actuation over the microcapsules.
1
Fig. 1: Principle of Magnetic Actuation over the Microcapsules

Results

Figure 2 shows the preliminary set up for actuation over microparticles within the region of interest.
2
Fig. 2 Setup for Microparticles Actuation
Microcapsules with evenly distributed magnetic stripes have been fabricated. The stripes make the spherical microcapsules asymmetric so that their locomotion control is directed. Alignment and movement of the microcapsules are observed in the EMA system under DC output, while rotation is observed under sinusoidal output current.

4

Fig. 3 Microcapsules with magnetic CI strips. Scale bar: 200μm.
56
Fig.4 Magnetic actuation with (A)small cylindrical magnet and (B)magnetic microcapsules

People Involved

Staff: Shen Shen, Song Shuang and Zhu Jingling
PIs: Ren Hongliang and Li Jun

Experiment Videos

Presentations and Publications

1.Shen Shen, Shuang Song, Jingling Zhu, Max Q-H Meng, Jun Li and Hongliang Ren, Preliminary Design towards a Magnetic Actuated Drug Delivery System, 7th IEEE International Conference on Cybernetics and Intelligent Systems and the 7th IEEE International Conference on Robotics, Automation and Mechatronics, 2015.

Poster in BME Showcase 2015

7

Simultaneous Hand-Eye, Tool-Flange and Robot-Robot Calibration for Co-manipulators by Solving AXB=YCZ Problem

Abstract

Multi-robot co-manipulation shows great potential to address the limitations of using single robot in complicated tasks such as robotic surgeries. However, the dynamic setup poses great uncertainties in the circumstances of robotic mobility and unstructured environment. Therefore, the relationships among all the base frames (robot-robot calibration) and the relationships between the end-effectors and the other devices such as cameras (hand-eye calibration) and tools (tool-flange calibration) have to be determined constantly in order to enable robotic cooperation in the constantly changing environment. We formulated the problem of hand-eye, tool-flange and robot-robot calibration to a matrix equation AXB=YCZ. A series of generic geometric properties and lemmas were presented, leading to the derivation of the final simultaneous algorithm. In addition to the accurate iterative solution, a closed-form solution was also introduced based on quaternions to give an initial value. To show the feasibility and superiority of the simultaneous method, two non-simultaneous methods were also proposed for comparison. Furthermore, thorough simulations under different noise levels and various robot movements were carried out for both simultaneous and non-simultaneous methods. Experiments on real robots were also performed to evaluate the proposed simultaneous method. The comparison results from both simulations and experiments demonstrated the superior accuracy and efficiency of the simultaneous method.

Problem Formulation

Measurement Data:
Homogeneous transformations from the robot bases to end-effector (A and C), and from tracker to marker (B).
Unknowns:
Homogeneous transformations from one robot base frame to another (Y), and from eye/tool to robot hand/flange (X and Z).
The measurable data A, B and C, and the unknowns X, Y and Z form a transformation loop which can be formulated as, AXB=YCZ (1).
problem

Fig. 1: The relevance and differences among the problem defined in this paper and the other two classical problems in robotics. Our problem formulation can be considered as a superset of the other two.

Approaches

Non-simultaneous Methods

3-Step Method
In the non-simultaneous 3-Step method, the X and Z in (1) are separately calculated as two hand-eye/tool-flange calibrations which can be represented as an AX = XB problem in the first and second steps. This results in two data acquisition procedures, in which the two manipulators carry out at least two rotations whose rotational axes are not parallel or anti-parallel by turns while the other one being kept immobile. The last unknown robot-robot relationship Y could be solved directly using the previously retrieved data by the method of least squares.
2-Step Method
The non-simultaneous 2-Step method formulates the original calibration problem in successive processes which solve AX = XB firstly, and then the AX = YB. The data acquisition procedures and obtained data are the same with the 3-Step method. In contrast to solving robot-robot relationship independently, the 2-Step method solves tool-flange/hand-eye and robot-robot transforms in an AX = YB manner in the second step. This is possible because equation AXB = YCZ can be expressed as (AXB)inv(Z) = YC, which is in an AX = YB form with the solution of X known.

Simultaneous Method

Non-simultaneous methods face a problem of error accumulation, since in these methods the latter steps use the previous solutions as input. As a result, the inaccuracy produced in the former steps will accumulate to the subsequent steps. In addition to accuracy, it is preferred that the two robots participating the calibration procedure simultaneously, which will significantly save the total time required.
In regards to this, a simultaneous method is proposed to improve the accuracy and efficiency of the calibration by solving the original AXB = YCZ problem directly. During the data acquisition procedure, the manipulators simultaneously move to different configurations and the corresponding data set A, B and C are recorded. Then the unknown X, Y and Z are solved simultaneously.

Evaluations

Simulations

To illustrate the feasibility of the proposed methods, intensive simulations have been carried out under different noise situations and by using different numbers of data sets.
simulation

Fig. 2: A schematic diagram which shows the experiment setup consisting of two Puma 560 manipulators, a tracking sensor and a target marker to solve the hand-eye, tool-flange and robot-robot calibration problem.

Simulations Results

For the rotational part, the three methods perform evenly in the accuracy of Z. However, the simultaneous method slightly outperforms in the accuracy of X and significantly in the accuracy of Y than the other two non-simultaneous methods. The results of the translational part are similar to the rotational ones. For the solution of Z, the accuracy of the simultaneous method is as good as the 3-Step method but slightly worse than the 2-Step method. However, the simultaneous method achieves a significantly improvement in the accuracy of X and Y compared to the other two methods.
 
simulation1
simulation2
simulation3

Experiments Results

Besides the simulation, ample real experiments have been conceived and carried out under different configurations to evaluate the proposed methods. As shown in Fig. 6, the experiments involved a Staubli TX60 robot (6 DOFs, averaged repeatability 0.02mm), a Barrett WAM robot (4 DOFs, averaged repeatability 0.05mm) and a NDI Polaris optical tracker (RMS repeatability 0.10mm). The optical tracker was mounted to the last link of the Staubli robot, referred to as sensor robot. The corresponding reflective marker was mounted to the last link of the WAM robot, referred to as marker robot.
experiment

Fig. 6: The experiment is carried out by using a Staubli TX60 robot and a Barrett WAM robot. A NDI Polaris optical tracker is mounted to the Staubli robot to track a reflective marker (invisible from current camera angle) that is mounted to the WAM robot.

To demonstrate the superiority of the simultaneous method in the real experimental scenarios, a 5-fold cross-validation approach is implemented for 200 times for all the calibration methods under all system configurations. For simultaneous method, after data alignment and RANSAC processing, 80% of the remaining data are randomly selected to calculate unknown X, Y, and Z, and 20% are used as test data to evaluate the performance. For 2-Step and 3Step methods, after calculating the unknowns by each method, same test data from the simultaneous method are used to evaluate their performances.

In Fig. 7, the evaluated errors of 200 times 5-fold cross-validation for three proposed methods at three ranges are shown as box plots. Left-tail paired-samples t-tests have been carried out to compare the performances of simultaneous method versus 2-Step and 3-Step methods, respectively. The results indicate that the rotational and translational errors from the simultaneous method are very significantly smaller than the 2-Step and 3-Step methods. Only two non-significant results exist in the rotational performances at medium and far ranges when comparing the simultaneous method with the 3-Step one. Nevertheless, the simultaneous method outperforms the non-simultaneous ones for translation error at all ranges.

experiment1

Fig. 7: Results of 200 times 5-fold cross-validation and left-tail paired-samples t-test at the near, medium and far ranges. The box plots show the rotational and translational error distributions for three methods at three ranges. **, * and N.S. stands for very significant at 99% confidence level, significant and non-significant at 95% confidence level.

Related Publications

1. Liao Wu, Jiaole Wang, Max Q.-H. Meng, and Hongliang Ren, imultaneous Hand-Eye, Tool-Flange and Robot-Robot Calibration for Multi-robot Co-manipulation by Solving AXB = YCZ Problem, Robotics, IEEE Transactions on (Conditionally accepted)
2. Jiaole Wang, Liao Wu and Hongliang Ren, Towards simultaneous coordinate calibrations for cooperative multiple robots, Intelligent Robots and Systems (IROS 2014), 2014 IEEE/RSJ International Conference on. IEEE, 2014: 410-415.

Institute & People Involved

The Chinese University of Hong Kong (CUHK): Jiaole Wang, Student Member, IEEE; Max Q.-H. Meng, Fellow, IEEE
National University of Singapore (NUS): Liao Wu; Hongliang Ren, Member, IEEE

Videos

-Calibration Experiments
 

Surgical Tracking System for Laparoscopic Surgery

Collaborators:
ERC-CISST, LCSR Lab of Johns Hopkins University, USA
Fraunhofer Germany (FhG)

Laparoscopic surgery poses a challenging problem for a real-time intra-body navigation system: how to keep tracking the surgical instruments inside the human body intra-operatively. This project aims to develop surgical tracking technology that is accurate, robust against environmental disturbances, and does not require line-of-sight. The current approach is to combine electromagnetic and inertial sensing. Sensor fusion methods are proposed for a hybrid tracking system that incorporates a miniature inertial measurement unit and an electromagnetic navigation system, in order to obtain continuous position and orientation information, even in the presence of metal objects.
Additional information at [SMARTS Lab of Johns Hopkins University]