Author: hxwu

This research presents an automatic calibration and dynamic registration method specifically designed for deformable tissues, integrating Augmented Reality (AR) technology to enhance surgical precision in Endoscopic Submucosal Dissection (ESD).

Our approach leverages a 6D pose estimator to align virtual and real-world target tissues seamlessly, utilizing the SuperGlue feature-matching network and the Metric3D depth estimation network for robust fusion. Additionally, our dynamic registration method enables real-time tracking of tissue deformation, ensuring more reliable surgical guidance.

Experimental validation demonstrated the effectiveness of our system, with automatic calibration experiments using cloth achieving a mean absolute error (MAE) of 3.79 ± 0.64 mm. Dynamic registration accuracy was assessed under varying tissue deformation, yielding an MAE of 6.03 ± 0.96 mm. Ex-vivo experiments with porcine small intestine tissue further validated our system’s performance, with an AR calibration MAE of 3.11 ± 0.56 mm and a dynamic registration MAE of 3.20 ± 1.96 mm.

The full paper is in production and will be available at https://lnkd.in/gMFCDWv4

In this paper, we explore the role of large vision models in advancing robot-assisted surgery, analyzing key developments and discussing future directions in AI-driven surgical innovation. By examining emerging trends and challenges, we contribute to the broader conversation on how intelligent visual systems can enhance precision, adaptability, and decision-making in surgical robotics.

Congrats to Prof. Zhe Min, Prof. Sam, Jiewen Lai, and Prof. Hongliang Ren!

Check out the full paper here: https://rdcu.be/elIbb

At the workshop “Origami and Kirigami: How Paper Folding and Cutting Have Revolutionized Soft Robotics and What’s Beyond” (https://lnkd.in/gsYvat-X), Professor Hongliang Ren delivered a keynote speech titled “Tetherless Reconfigurations at Origami-Continuum Interfaces.” Drawing from our group’s recent researches published in top-tier journals like Science Robotics, Nature Communications, Advanced Functional Materials, ACS Nano, and Advanced Materials Technology, he explored how innovative materials—such as high-temperature-resistant, phase-change elastic, and magnetically responsive ones—are revolutionizing origami-based robots. These works enable precise, continuous motion for in vivo medical applications and highlights the vast interdisciplinary opportunities in medical soft robotics.

Additionally, our PhD student WENCHAO YUE delivered three oral presentations based on our accepted papers. Specifically, he showcased a compact OCT-based tactile sensor (2 mm in diameter, developed in collaboration with ABI Lab (https://lnkd.in/gUuzQqDt) under Prof. Wu ‘Scott’ YUAN at CUHK BME), a bistable origami robot designed for microneedle puncture (in partnership with Xu’s Lab (https://lnkd.in/gdb-5tTv), led by Prof. CHENJIE XU at CityU BME), and an ink-based stretching sensor inspired by kirigami principles. His talks highlighted the latest research progress in our group and the exciting potential of these technologies in soft robotics for medical applications.

During his visit, our team had the opportunity to showcase our latest research efforts across a wide range of topics in surgical intelligence and automation, including augmented reality-assisted surgery, motion magnification, safe dissection trajectory planning, 3D scene understanding & reconstruction, vision-language action & navigation, robotic OCT, intubation, and ESD systems.

The discussions were both warm and thought-provoking, as we exchanged ideas and explored potential synergies between our research efforts. Prof. Padoy’s insights and expertise added immense value to our ongoing projects, and we’re excited about the possibilities for future collaboration.

1️⃣ #ICRA2025 – “ETSM: Automating Dissection Trajectory Suggestion and Confidence Map-Based Safety Margin Prediction for Robot-assisted Endoscopic Submucosal Dissection”

This work introduces a framework for predicting optimal dissection trajectories while integrating a confidence map-based safety margin to minimize risks like tissue perforation.

🚀 Key contributions:

– A novel dataset (ETSM), featuring over 1,800 annotated clips from robotic ESD procedures.

– The RCMNet model, which uses regression to predict confidence maps, guiding surgeons toward safer dissection zones.

– Outperformed baselines with a mean absolute error of 3.18, demonstrating robust performance even under visual challenges.

Authors: Mengya Xu, Wenjin Mo, Guankun Wang, Huxin Gao, An Wang, Long Bai, Chaoyang Lyu, Xiaoxiao Yang, Zhen Li, and Hongliang Ren

📚Paper link: https://lnkd.in/gUdr6XQc

2️⃣ #IPCAI2025 – “PDZSeg: Adapting the Foundation Model for Dissection Zone Segmentation with Visual Prompts in Robot-assisted Endoscopic Submucosal Dissection”

This paper adapts foundation models to enable region-specific dissection zone segmentation using flexible visual prompts (e.g., scribbles, bounding boxes).

🚀 Key insights:

– A novel ESD-DZSeg dataset tailored for prompt-based segmentation tasks.

– Fine-tuned the foundation model DINOv2 with LoRA for efficient adaptation to surgical tasks. Achieved state-of-the-art accuracy, with long scribble prompts yielding a mean Intersection over Union (IoU) of 74.06%.

– Empowers surgeons to intuitively refine segmentation through natural visual cues, enhancing real-time decision support.

Authors: Mengya Xu, Wenjin Mo, Guankun Wang, Huxin Gao, An Wang, Zhen Li, Xiaoxiao Yang and Hongliang Ren

📚Paper link: https://lnkd.in/ggUM-5Zc

Port wine stains (PWS) are congenital vascular malformations that can lead to significant psychological and physical complications if untreated. Photodynamic therapy (PDT) is a common treatment but shows variable efficacy due to the heterogeneous vascular architecture of PWS lesions. Current diagnostic methods relying on skin surface appearance often fail to reflect underlying structural differences of PWS lesions, leading to erratic treatment effects.

Optical coherence tomography (#OCT) and OCT angiography (#OCTA) are promising tools for imaging PWS lesions. However, existing OCTA quantitative metrics cannot show significant differences among the various PWS subtypes based on clinical skin appearance diagnosis. This work proposes a fine-grained classification method for PWS using OCT and OCTA features. Compared with current clinical diagnosis based on skin appearance, the proposed method reveals the angiopathological heterogeneity of hypodermic PWS lesions and has the potential to provide more effective subtyping and treatment strategies for PWS.

Research team: Xiaofeng Deng, Defu Chen, Bowen Liu, Xiwan Zhang, Haixia Qiu, Wu ‘Scott’ YUAN, and Hongliang Ren.

Paper available at: 10.1109/JBHI.2025.3545931

Our paper entitled “Advancing Dense Endoscopic Reconstruction with Gaussian Splatting-driven Surface Normal-aware Tracking and Mapping” has been accepted at #ICRA2025! 🎉

Multi-view inconsistencies in 3D Gaussian Splatting (3DGS) have long limited precise depth and surface reconstruction in minimally invasive surgery. Our team (Yiming Huang *, Beilei Cui *, Long Bai *, Zhen Chen, Jinlin Wu, Zhen Li, Hongbin Liu, Hongliang Ren) from The Chinese University of Hong Kong and Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, CAS introduces Endo-2DTAM—a real-time endoscopic SLAM system that combines 2DGS with a surface normal-aware pipeline to achieve geometrically accurate, high-quality reconstruction.

🔑 Key Innovations:

✅ Robust tracking via point-to-point/plane metrics

✅ Surface enhancement using normal consistency & depth distortion

✅ Pose-consistent keyframe sampling for coherence

📈 Results:

●    1.87±0.63 mm RMSE in depth reconstruction

●    Real-time rendering & efficient computation

●    Open-source code: GitHub (https://lnkd.in/gyRr2YxS)

Paving the way for safer, smarter surgical robotics! 🤖💡

This recognition highlights Prof. Ren’s contributions to intelligent medical robotics and AI. Join us in congratulating Prof. Ren on this well-deserved achievement! 🎉👏

🔗 Check the awards inauguration ceremony at https://lnkd.in/gEAcPecJ

– Background

Open tracheostomy (OT) remains the gold standard for managing airway obstruction; however, it is associated with stringent procedural requirements, scarring, and risks of infection. Percutaneous dilation tracheostomy (PDT), while more cost-effective, less invasive, and safer for surgeons, carries the risk of injuring the posterior tracheal wall and esophagus. Additionally, precise identification of tracheal rings and the optimal puncture site can be challenging.

– Our Contribution

To address these limitations and enhance the safety and simplicity of tracheostomy procedures, a minimally invasive, endotracheal inside-out flexible needle-driving system has been developed for microendoscope-guided robotic tracheostomy (MERT). This system integrates optical coherence tomography (OCT) and microendoscopic guidance to facilitate robotic insertion into the trachea, enabling an inside-out puncture with a flexible needle. The device is designed to operate through a standard endotracheal tube (ETT), and the puncture direction of the flexible needle is fully adjustable. Kinematic and static models of the flexible needle have been developed, and the system’s feasibility has been validated through porcine trachea puncture experiments.

This approach demonstrates significant potential for improving tracheostomy outcomes by minimizing invasiveness and enhancing procedural precision.

– Authors: Botao Lin, Sishen YUAN, Tinghua Zhang, Tao Zhang, Ruoyi Hao, Wu ‘Scott’ YUAN, Chwee Ming Lim, and Hongliang Ren

Looking forward to sharing our findings with the robotics and medical communities at #ICRA2025! 🌐🤖🩺

This work introduces a modular robotic platform designed to assist in nasotracheal intubation, a critical yet challenging procedure in clinical settings. While orotracheal intubation robots have seen significant development, nasotracheal intubation remains underdeveloped, particularly in terms of safety mechanisms. Our robot features a variable-stiffness fiberoptic bronchoscope (FOB) control module, enabling both low-stiffness mode for safe navigation through the nasal cavity and high-stiffness mode for enhanced load-bearing near the glottis. Additionally, a compact FOB feeding module with passive failure protection ensures controlled force application, minimizing risks during the procedure.

We’re grateful to our team (Ruoyi Hao, Sam, Jiewen Lai, Wenqi Zhong, Dihong Xie, Yu Tian, Tao Zhang, Zhang Yang, Catherine P. L. Chan, Jason Y. K. Chan, and Hongliang Ren) for their dedication and to #ICRA for this opportunity to share our work.

Looking forward to discussing this with the robotics and medical communities!