Author: hxwu

โœจ Boosting Robustness of Magnetic Tracking โœจ

  By

We are thrilled to share that our latest research paper โ€œEnhancing Anti-interference of Magnetic Tracking: A MagRobustNet-based Framework with Self-supervised Anomaly Detection and Measurements Recoveryโ€ has been accepted by IEEE Transactions on Industrial Informatics!

Magnetic tracking technology often suffers from diverse and unpredictable interferences in practical applications, such as hard-/soft-iron interferences and sensor saturation, leading to reduced localization accuracy or even tracking failure.

To address these issues, we propose a MagRobustNet-based framework with anomaly detection and measurement recovery. In the first step, disjoint mask sets are used in conjunction with MagRobustNet to detect anomalous measurements subject to disturbances. In the second step, the interfered regions are masked, and MagRobustNet is applied again to recover their expected measurements from neighboring normal data.

Our proposed method not only enhances the tracking systemโ€™s anti-interference capability, but also indicates the interfered regions, offering a new potential diagnostic method for localizing ingested foreign bodies in clinical practice.

Check out our video demonstration at https://lnkd.in/g9TNnsA4

Stay tuned for the paper publication!

Congrats to all co-authors: Shijian Su, Huxin Gao, and Hongliang Ren from the Department of Electronic Engineering, The Chinese University of Hong Kong; Hai Lan and Houde Dai from Quanzhou Institute of Equipment Manufacturing, Haixi Institute, CAS.

No alternative text description for this image
No alternative text description for this image

We are excited to share our latest research entitled โ€œDisentangling Contact Location for Stretchable Tactile Sensors from Soft Waveguide Ultrasonic Scatter Signalsโ€ collaborated by Zhiheng Li and Yuan Lin, has been accepted by Advanced Intelligent Systems!

  By

Flexible tactile sensors have garnered significant attention due to their flexible, lightweight, and comfortable nature, catering to the growing demands in various applications such as human-machine interfaces, healthcare, and robotics. However, it remains a challenge to achieve precise contact location sensing that is decoupled from sensor strain and touching forces. Thus, this paper proposes a novel data-driven approach for force contact location sensing (FCLS), based on scatter signals (SS) of the ultrasonic waveguide, with the influence of sensor strain and forces. This method utilizes deep CNNs to fuse local and global features of the soft waveguide ultrasonic SS and an MLP to perform regression modeling on the fused features and FCL, thereby obtaining FCL information. The experimental results indicate that the accuracy of the proposed FCLS method has an MAE loss of 0.627 mm and an MRE loss of 3.19%.

The full paper will be available at: https://lnkd.in/gKr22Uvi

Co-Authors: Zhiheng Li (CUHK), Yuan Lin (SJTU), Peter B. Shull (SJTU) and Hongliang Ren (CUHK). The first two authors contributed equally to this work.

diagram

Revolutionizing endomicroscopy with motor-free technology! โœจ

  By

Our team has developed a telerobotic OCT endoscope that overcomes the limitations of traditional systems. This innovative device offers high-resolution imaging, reduced thermal and electrical risks, and guidewire-independent navigation. Steerable angles up to 110ยฐ and distortion-free 3D imaging in vivo demonstrate its potential for improved patient outcomes.

Abstract:

Intraluminal epithelial abnormalities, potential precursors to significant conditions like cancer, necessitate early detection for improved prognosis. We present a motor-free telerobotic optical coherence tomography (OCT) endo scope that offers high-resolution intraluminal imaging and overcomes the limitations of traditional systems in navigating curved lumens. This system incorporates a compact magnetic rotor with a rotatable diametrically magnetized cylinder permanent magnet (RDPM) and a reflector, effectively mitigating thermal and electrical risks by utilizing an external magnetic field to maintain temperature increases below 0.5ยฐC and generated voltage under 0.02mV. Additionally, a learning-based method corrects imaging distortions resulting from nonuniform rotational speeds. Demonstrating superior maneuverability, the device achieves steerable angles up to 110ยฐ and operates effectively in vivo, providing distortion-free 3D programmable imaging in mouse colons. This advancement represents a significant step towards guidewire-independent endomicroscopy, enhancing biosafety and potential patient outcomes.

Check out our latest publication in Nature Communications: https://lnkd.in/guAazam9  

diagram, engineering drawing

We are excited to share our latest work, “PneumaOCT: Pneumatic optical coherence tomography endoscopy for targeted distortion-free imaging in tortuous and narrow internal lumens”, which has been published in Science Advances!!! ๐ŸŽ‰๐ŸŽ‰๐ŸŽ‰

  By

The complex anatomy of internal luminal organs, like bronchioles, poses challenges for endoscopic optical coherence tomography (OCT). These challenges include limited steerability for targeted imaging and nonuniform rotation distortion (NURD) with proximal scanning. Using rotary micromotors for distal scanning could address NURD but raises concerns about electrical safety and costs.

We present pneumaOCT, the first pneumatic OCT endoscope, comprising a steerable catheter with a soft pneumatic actuator and an imaging probe with a miniature pneumatic turbine. With a diameter of 2.8 mm, pneumaOCT allows for a bending angle of up to 237ยฐ, facilitating navigation through narrow turns. The pneumatic turbine enables adjustable imaging speeds from 51 to 446 revolutions per second. We demonstrate the pneumaOCT in vivo imaging of mouse esophagus and colon, as well as targeted and distortion-free imaging of peripheral bronchioles in a bronchial phantom and a porcine lung. This advancement substantially improves endoscopic OCT for navigational imaging in curved and narrow lumens.

In the future, we will further develop an autonomous pneumaOCT endoscopy system by integrating an automated external robotic system, navigation guidance (such as x-ray imaging or MRI), and intelligent sensing and control algorithms. This advancement will have a substantial impact on the development of robotic endoscopy techniques in complex clinical practice.

For more details, please find the paper at https://lnkd.in/gtcM7v7p

This is collaborative work between LabREN (http://www.labren.org/), ABI Lab (https://lnkd.in/gUuzQqDt) and Department of Surgery, CUHK. Congrats to all authors: Tinghua Zhang, Sishen YUAN, Chao Xu, Peng Liu, Hing-Chiu Chang, Sze Hang Calvin Ng, Hongliang Ren, and Wu ‘Scott’ YUAN.

An example application of pneumaOCT for targeted distortion-free imaging in the peripheral lung.
PneumaOCT imaging of ex vivo porcine lung.

Our latest research work titled โ€œA Real-Time Self-Sensing Approach to Sensor Array Configuration Fusing Prior Knowledge for Reconfigurable Magnetic Tracking Systemsโ€ has been accepted by IEEE/ASME Transactions on Mechatronics!

  By

Impact:

The reconfigurability of magnetic tracking systems (MTSs) allows for its application in workspaces of different sizes. For instance, with appropriate configuration adjustments, it can be utilized for tasks such as tongue tracking in the head, tracheal intubation navigation in the neck, and even muscle tracking in the legs and arms. However, the dynamic changes in sensor array configuration, known as deformation, caused by posture changes during long-duration examinations, impose significant challenges on MTSs that heavily rely on magnetometer poses.

Here we propose a real-time self-sensing method based on the sensor array structural model and magnetic dipole model, which simultaneously estimates the magnet pose and the hinge angles on the sensor array. The self-sensing capability opens up a new way for perceiving the morphology of origami robots and measuring the curvature of flexible catheters

Paper: https://lnkd.in/dnmcvusm

Authors: Shijian Su; Xindi Yang; Zhen Li; Hongliang Ren

No alternative text description for this image
No alternative text description for this image

We are excited to share our paper “OSSAR: Towards Open-Set Surgical Activity Recognition in Robot-assisted Surgery” which has been accepted for IEEE International Conference on Robotics and Automation (ICRA) 2024!

  By

In this work, we tackle the challenge of open-set recognition in surgical robotics. Our novel OSSAR framework improves the ability to classify known surgical activities while also detecting unknown activities that weren’t seen during training.

Key contributions:

โ€ข A hyperspherical reciprocal point strategy to better separate known and unknown classes

โ€ข A calibration technique to reduce overconfident misclassifications 

โ€ข New open-set benchmarks on the JIGSAWS dataset and our novel DREAMS dataset for endoscopic procedures

โ€ข State-of-the-art performance on open-set surgical activity recognition tasks

This research takes an important step towards more robust and generalizable AI systems for surgical robots. We hope it will help pave the way for safer and more capable robot-assisted surgeries.

Thank all the amazing co-authors Long Bai, Guankun Wang, Jie Wang, Xiaoxiao Yang, Huxin Gao, Xin Liang, An Wang, Mobarakol Islam, and Hongliang Ren

and our institutions (The Chinese University of Hong Kong, Beijing Institute of Technology, Qilu Hospital of Shandong University, Tongji University, University College London, National University of Singapore) for their support.

You can find more details in our paper https://lnkd.in/gDsjVDSP

No alternative text description for this image
No alternative text description for this image
No alternative text description for this image
No alternative text description for this image

We are happy to share our journal paper titled โ€œAMagPoseNet: Real-Time 6-DoF Magnet Pose Estimation by Dual-Domain Few-Shot Learning from Prior Modelโ€ published in IEEE Transactions on Industrial Informatics!

  By

Impact: Traditional magnetic tracking approaches based on mathematical models and optimization algorithms are computationally intensive, depend on initial guesses, and do not guarantee convergence to a global optimum. Here we propose an annular magnet pose estimation network (called AMagPoseNet) based on dual-domain few-shot learning from a prior mathematical model, featuring the following advantages:

1) Higher localization accuracy (1.87ยฑ1.14 mm, 1.89ยฑ0.81ยฐ), especially in the near field;

2) Enhanced robustness, as AMagPoseNet is just a single feed-forward neural network that does not rely on initial guesses and avoids the risk of falling into local optima;

3) Lower computational latency (2.08ยฑ0.02 ms) since the magnet pose is directly regressed from a single feed-forward network rather than iterative optimization;

4) Real-time estimation of 6-DoF pose if discriminative magnetic field features are provided.

Paper: https://lnkd.in/gXBSQ-hG

Dataset: https://lnkd.in/gXEFfwfA

Authors: Shijian Su; Sishen YUAN; Mengya Xu; Huxin Gao; Xiaoxiao Yang; and Prof Hongliang Ren

No alternative text description for this image
No alternative text description for this image

Excited to share our journal paper entitled โ€œMagnetic Tracking With Real-Time Geomagnetic Vector Separation for Robotic Dockable Chargingโ€ published in IEEE Transactions on Intelligent Transportation Systems! ๐ŸŽ‰

  By

Great collaboration between the Chinese University of Hong Kong and the Quanzhou Institute of Equipment Manufacturing, Haixi Institutes, Chinese Academy of Sciences. ๐Ÿค

The superposition of the geomagnetic vector and the magnetic field vector generated by the permanent magnet (PM) leads to the degrading of magnetic tracking performance. Here we present a real-time geomagnetic-vector-separation method to estimate the PM pose and geomagnetic vector simultaneously. This advancement promises to revolutionize autonomous robotic operations, offering a robust solution for seamless and reliable self-charging mechanisms, with far-reaching implications for various industries.

Paper: https://lnkd.in/gAbr82Dp

Authors: Shijian Su; Houde Dai; Yuanchao Zhang; Sishen YUAN; Prof Shuang Song and Prof Hongliang Ren.

No alternative text description for this image
No alternative text description for this image

Our latest research work titled โ€œA Wearable, Reconfigurable, and Modular Magnetic Tracking System for Wireless Capsule Robotsโ€ has been accepted by IEEE Transactions on Industrial Informatics! ๐ŸŽ‰

  By

Existing wearable MTSs are fixed in size and cannot accommodate patients with diverse abdominal circumferences. Here we propose a wearable and reconfigurable MTS. First, we design a reconfigurable sensor array inspired by the structure of bamboo slips, allowing it to conform to the abdominal surface and accommodate individuals with different abdominal circumferences. Next, we formulate a magnetic tracking optimization problem based on the magnetic dipole model and our established kinematic model of the reconfigurable sensor array.

Our proposed system is portable, reconfigurable and adaptable to different abdominal circumferences, offering valuable technological means for diagnosing and treating gastrointestinal disorders.

Paper: https://lnkd.in/g7HqPARr

Authors: Shijian Su; Sishen YUAN; Zhen Li; Yan Ma; Miaomiao Ma and Prof Hongliang Ren

No alternative text description for this image
No alternative text description for this image

We’re thrilled to share our work, “Inconstant curvature kinematics of parallel continuum robot without static model”, which has been accepted at the IEEE International Conference on Robotics and Automation (ICRA 2024)!

  By

In the study of minimally invasive surgical robots, a mini parallel continuum robot (PCR) has shown motion advantage after passing through a long and winding working channel. However, due to the interaction force between the elastic wires of the parallel robots during motion generation processes, the constant curvature assumption has shown modeling errors. This causes the current geometric kinematic model to become unreliable. This paper aims to solve this issue. The simulation in ANSYS is carried out, and the shape of one of the driving wires, when bending, is fitted by a two-segment polynomial curve. Then, the position of the distal wrist tip can be calculated based on the curve shape. To verify the accuracy of the proposed model, bending simulation and experiment are carried out. The accuracy of the proposed model is compared with that of the kinematic model based on constant curvature assumption. The result shows that the proposed model can get more accurate results, especially when the driving wire displacement increases.

Main contributions:

1. A two-segment polynomial curve was used to model the deformation of the parallel wrist joint. Compared with the kinematic model based on constant curvature assumption, the proposed curve has shown higher accuracy. 

2. The fitting results of the chosen NiTi wire with the proposed curve were verified by finite element simulation (Fig. 3).

3. The modeling accuracy was verified when the deflection angle was 0ยฐ, 30ยฐ and 45ยฐ respectively. The experimental results show that the accuracy of this method is improved significantly than that of the constant curvature model. Especially, when the bending angle is increased, its accuracy does not decrease significantly.

Stay tuned to more of our research on parallel continuum robot!!!

Co-authors: Tao Zhang, Huxin Gao and Prof Hongliang Ren.

No alternative text description for this image
No alternative text description for this image
No alternative text description for this image