Author: hxwu

On August 11, Prof. Hongliang Ren’s research group from The Chinese University of Hong Kong (CUHK) was invited by the Department of Magnetic Surgery at the First Affiliated Hospital of Xi’an Jiaotong University (XJTU) to visit their specialized magnetic surgery ward and engage in a symposium with Prof. Xiaopeng Yan to explore cross-disciplinary medical-engineering research collaboration.

During the visiting, Prof. Yan presented the development history, landmark surgical cases, and distinct advantages of magnetic surgery compared to conventional surgical techniques. In particular, magnetic recanalization offers a highly effective, minimally invasive solution for severe complications like post-liver transplantation biliary obstruction—often termed the “Achilles’ heel” of liver transplantation—where traditional endoscopic or surgical interventions face major limitations. Prof. Yan also highlighted current clinical pain points and shared valuable insights regarding urgent operational needs.

Prof. Hongliang Ren introduced the group’s latest research achievements in advanced magnetic materials and magnetically controlled surgical robots. Both parties conducted fruitful discussions on the translational application of magnetic technologies in clinical settings, identified key prospective areas for joint research, and established plans for ongoing collaboration.

Special thanks to Prof. Xiaopeng Yan and the Department of Magnetic Surgery at XJTU First Affiliated Hospital for hosting this inspiring exchange!

🔖 #MagneticSurgery #MedicalRobotics #MedEngineering #SurgicalRobotics #CUHKEngineering #CUHK #XJTU

The award recognizes the work “Actively Controlled Continuous-Everting Capsule Robot: Design, Fabrication, and Validation.”

This research introduces a magnetically controlled capsule robot that uses a continuous-everting outer film to reduce sliding friction against delicate tissue, enabling smoother and safer navigation in complex gastrointestinal environments.

Following the conceptualization of this work, a prototype was manufactured and tested, and the functionality of the everting capsule robot was initially verified.

Figure 1. Max Meng and Botao Lin at the award ceremony

Figure 2. Working concept diagram of the continuous-everting capsule robot. Driven by an external magnetic field, the robot can move by continuously everting its outer film. Because this type of movement is unaffected by environmental friction, it can be applied to lumens of any suitable diameter.

Figure 3. Design and working principle of the continuous-everting capsule robot. (a) Design parameters of the continuous-everting capsule robot. (b) The forward motion cycle of the robot. Three colored markers (blue, red, and yellow) are utilized to track the movement of the outer film. During the traction eversion phase, magnetic attraction drives the inner core forward, which induces the eversion of the outer film via friction between the two components. In the inner core resetting phase, the external magnetic field rotates and translates, causing the inner core to simultaneously rotate and move backward. During this resetting phase, the outer film remains static.

Figure 4. Experimental validations. (a) The capsule robot prototype and the diagram of the actuation control board. (b) Experimental evaluation of the capsule robot’s eversion movement and the inner core’s resetting capability. During the eversion experiment, external magnetic traction pulled the robot forward approximately 4cm in 4 seconds. In the resetting experiment, the inner core retracted through the robot’s internal cavity in 9 seconds. Throughout this resetting process, the outer film remained entirely stationary. (c) The record of the positions of the robot and the inner core during the experiments. It can be measured that the average velocity of the robot moving and the inner core resetting are 7.5mm/s and 4.4mm/s, respectively.

We are excited to share our latest Comment published in #npj Digital Medicine:
“How can reasoning capability empower the AI copilot robot in endoscopic surgery”

Current AI copilots in endoscopic surgery are still largely reactive and vision-driven. While they can detect anatomy, instruments, and scene changes, they often still struggle to truly understand surgical intent, infer hidden tissue dynamics, and respond robustly to uncertainty—all of which are essential for safe and precise intraoperative assistance.

In this article, we highlight how reasoning capability can become a key enabler for the next generation of AI copilot robots in endoscopic surgery:

1. 🧠 Reasoning beyond perception: Enabling VLA-based surgical robots to go beyond simple visual recognition and translate high-level surgeon intent into precise, context-aware low-level motion goals.
2. 🔄 Multimodal and uncertainty-aware intelligence: Fusing endoscopic vision with preoperative imaging, intraoperative sensing, and tracking signals, while dynamically re-weighting information sources under occlusion, bleeding, smoke, and other uncertain conditions.
3. 🤝 Coordinated multi-instrument collaboration: Supporting synchronized control of multiple tools for subtasks such as traction, dissection, and hemostasis, with reasoning-guided adaptation to tissue deformation and workflow variation.
4. 🔮 Anticipatory and safer decision-making: Using chain-of-thought-style reasoning to forecast tissue response, evaluate possible action outcomes, and generate more conservative and interpretable assistance under risk.
5. 🏥 Surgeon-in-the-loop clinical deployment: Framing the AI copilot robot at LoA 2–3, where the system assists with task generation, monitoring, and bounded low-level execution under explicit safety constraints and continuous surgeon oversight.

We believe the future of endoscopic robotic assistance lies not only in systems that can see, but in systems that can reason, adapt, and collaborate. With reasoning-enabled VLA models, AI copilot robots may evolve from reactive executors into true cognitive partners in the operating room.

📃 Read the full paper here: https://www.nature.com/articles/s41746-026-02827-8

👏 Kudos to the team: Mr. Guankun Wang, Dr. Long Bai, and Prof. Hongliang Ren.

Thrilled to share our latest Advanced Science work on enabling highly transferable, autonomous navigation for wireless capsule endoscopy (WCE)—using a lightweight Edge-Contour-Depth Fusion module and deep reinforcement learning (DRL).

WCE has revolutionized GI diagnostics, but its potential is often restricted by incomplete mucosal coverage and the poor ability of existing AI navigation methods to adapt across different patient anatomies. This motivated us to ditch the heavy, brittle, traditional “end-to-end” visual video streams that cause AI models to overfit to a single patient.

🧠✨ What we developed:
A unified, clinically viable framework that features:
🔹 Anatomical Landmark Guidance: Operates on stable, low-dimensional coordinates of conserved gastric structures (the fundus and pyloric antrum) rather than high-dimensional raw video.
🔹 Lightweight Perception Module: Combines classical Canny edge detection and Hu moments with a compact monocular depth network (DispNet) to run efficiently on low-power clinical hardware.
🔹 Robust Sim-to-Real Pipeline: Utilizes a patient-specific digital twin combined with a model-free Adaptive Dynamic Programming (ADP) controller to actively neutralize real-world physical disturbances and actuator latency.

🎯 Key Results:
✅ >97% mucosal coverage achieved within 50 seconds across 8 diverse, patient-derived stomach models in simulation.
✅ 87% mean coverage stability and a 53% reduction in procedure time during real-world ex-vivo experiments compared to expert manual control.
✅ Drastically reduced computational overhead, allowing deployment on low-cost processors (<2 TOPS).

💡 Why it matters:
This study establishes a scalable paradigm that conquers the “reality gap” and patient anatomical variability in medical robotics. By decoupling perception from control, it removes the need for expensive, massive patient datasets and high-end GPUs, paving the way for operator-independent, intelligent GI diagnostics.

🌱 What’s next?
We are expanding our training to encompass extreme pathological distortions (like hiatal hernias) and advancing toward fully wireless clinical deployment with dynamic, target-reaching capabilities for intraoperative pathologies.

🔗 Paper Link: https://advanced.onlinelibrary.wiley.com/doi/10.1002/advs.202600008

Thrilled to share our newly accepted paper in IEEE Transactions on Medical Robotics and Bionics, where we introduce a soft, endoscope-deployable microfluidic suction robot that combines multimodal intraluminal locomotion with localized aspiration and sampling for targeted mucus clearance and liquid biopsy.

🧠✨ What we developed:

A soft intraluminal robotic platform that:

🔹 Integrates Locomotion + Sampling: A pneumatically controlled 2×2 pouch matrix for multimodal actuation, paired with an independent microfluidic suction module for active liquid extraction and sample recovery.

🔹 Enables Stable Pitch Control: A balloon-based pitch control mechanism improves controllability for intraluminal operation, with the best overall performance observed at an initial pressure range of 2–3 kPa.

🔹 Balances Compliance and Safety: Single-pouch characterization guided the selection of a 2 mm pouch radius, achieving up to 246.91% maximum deformation; burst tests show a system safety factor ≈ 5.27 under the reported operating conditions.

🔹 Targets Real Clinical Pain Points: Designed for constrained lumens (e.g., distal airway) where conventional airway clearance approaches struggle with reach and effectiveness.

🎯 Key Results:

✅ Multimodal mobility: Differential actuation achieves 26.9 mm/min forward speed and 4.86° yaw per drive cycle.

✅ Robust suction across viscosities: Efficiently extracts 20–80% glycerol solutions within 10 s (via parameter tuning).

✅ In vivo feasibility: Endoscope-assisted porcine validation confirmed sequential pouch-driven motion and successful recovery of biological samples containing mucus and tissue fragments after saline irrigation.

💡 Why it matters:

This work demonstrates a compliant, integrated “move + anchor + suction” approach for narrow lumens—supporting safer localized intervention and sampling, with a path toward distal airway translation.

🌱 What’s next?

We’re moving toward more automated closed-loop pneumatic control, improved steerability/navigation, and miniaturization for deeper airway access—while expanding validation in airway-specific models.

🤖🪢
Thrilled to share our latest work published on Nature Communication, which redefines actuation for tendon-driven continuum robots — achieving full 3D omnidirectional motion and body twist using only a single tendon.

🧠✨ What we developed: A new class of continuum robots that:
🔹 Breaks Design Constraints: Eliminates the inherent trade-off between miniaturization and 3D manipulability by replacing multiple tendons with a single eccentric one.
🔹 Push-Pull-Twist Actuation: Achieves complex spatial movement through a unique driving mechanism.
🔹 High Efficiency: Features an outer diameter of 2.0–3.5 mm with a hollow ratio exceeding 57% — doubling the spatial utilization of traditional designs.
🔹 Open-Source Support: Includes a derived kinematics model and an open-source simulator for the robotics community.

🎯 Key Results:

• ✅ >1,000-fold Improvement: Massive increase in manipulability compared to conventional multi-tendon mechanisms.
• ✅ High Force Retention: Retains at least 70% of tip force across all directions.
• ✅ Versatile Demonstration: Proven success in teleoperation, navigation through tortuous environments, and “chopstick-like” continuum grippers.

💡 Why it matters: This work proves that miniature robots can maintain high dexterity and power without the bulk of traditional hardware, pointing toward the next generation of surgical actuators.

🌱 What’s next? We are exploring potential medical applications and the integration of these actuators into complex surgical procedures.

Thrilled to share our latest international collaboration! At NVIDIA GTC 2026 in San Jose, CA, the team led by Professor Hongliang Ren from The Chinese University of Hong Kong (CUHK), in partnership with NVIDIA and 35 leading global institutions, officially released Open-H-Embodiment, the world’s first and largest open-source dataset for medical robotics, now available on HuggingFace.

During the GTC keynote, Kimberly Powell, NVIDIA’s VP of Healthcare, highlighted this milestone. Our lab is honored to be a primary contributor, filling the critical gap in Embodied AI for medical robotics by providing high-fidelity data for contact dynamics and closed-loop control.

🧠✨ What we contributed & developed:

This project breaks the “perception-heavy, execution-light” limitation of traditional medical AI. Key highlights include:

🔹 778 Hours of Massive Multimodal Data: The dataset covers 400 complete clinical surgeries and 9 major robotic platforms (e.g., dVRK, CMR Versius, Kuka). It includes 65% clinical data, 23% bench-top experiments, and 12% simulation data.

🔹 Three High-Value Specialized Datasets from Our Lab:

• Dual-Source Ultrasound Dataset: Experts-level trajectories covering in-vivo porcine EUS and human forearm scanning, overcoming complex organ environments and multi-device calibration.
• Robotic Surgery Skill Dataset: Multi-modal data (RGB/RGB-D + Kinematics) for tissue manipulation and suturing, featuring millisecond-level synchronization and dual-mode control (teleoperation & automation).
• Flexible Endoscope Tracking Baseline: A standardized dataset addressing hysteresis and deformation in flexible endoscopy, supporting nanosecond-level time synchronization.

🔹 Surgical VLA & World Models:

• GR00T-H: A 3B-parameter Vision-Language-Action model based on NVIDIA Isaac GR00T, capable of long-horizon dexterous tasks like end-to-end suturing.
• Cosmos-H-Surgical-Simulator: An action-conditioned world model that boosts simulation efficiency by over 70x, bridging the sim-to-real gap.

🎯 Key Results: ✅ Global Standardization: First effort to unify medical robotic data across different devices and institutions under CC-BY-4.0. ✅ Efficiency Boost: Accelerated surgical simulation (600 sims in 40 mins) to generate high-fidelity video-action pairs. ✅ Clinical Relevance: Successfully captured nearly 500 hours of real-world clinical data for hernia, gallbladder, and uterine surgeries.

💡 Why it matters: This initiative provides the foundational “bedrock” for Medical Physical AI. By sharing high-quality, synchronized data for surgery, ultrasound, and endoscopy, we are lowering the barrier for researchers worldwide to develop autonomous surgical agents that are both explainable and adaptive.

🌱 What’s next? Our lab is continuing to deepen research in: 🔹 Reasoning-based autonomous control for surgical robots. 🔹 Cross-platform generalization of Medical VLA models. 🔹 Clinical translation of Embodied AI to improve patient outcomes.

Datasets address: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Open-H-Embodiment

Project website: https://github.com/open-h

#NVIDIAGTC2026 #MedicalRobotics #EmbodiedAI #HuggingFace #CUHK #OpenSource #HealthcareInnovation

Thrilled to share our latest #𝗦𝗰𝗶𝗲𝗻𝗰𝗲𝗔𝗱𝘃𝗮𝗻𝗰𝗲 work on a 𝗰𝗲𝗻𝘁𝗶𝗺𝗲𝘁𝗲𝗿-𝘀𝗰𝗮𝗹𝗲, 𝗳𝘂𝗹𝗹𝘆 𝘀𝗲𝗮𝗹𝗲𝗱, 𝗹𝗲𝗴𝗹𝗲𝘀𝘀 𝗮𝗺𝗽𝗵𝗶𝗯𝗶𝗼𝘂𝘀 𝗿𝗼𝗯𝗼𝘁 that can crawl on sand, jump, and steerably swim – powered by a 𝘀𝗶𝗻𝗴𝗹𝗲 𝘃𝗮𝗿𝗶𝗮𝗯𝗹𝗲-𝗼𝘂𝘁𝗽𝘂𝘁 𝘃𝗼𝗶𝗰𝗲-𝗰𝗼𝗶𝗹 𝗺𝗼𝘁𝗼𝗿 (𝗩𝗖𝗠).

At small scales, reliable 𝘸𝘢𝘵𝘦𝘳𝘱𝘳𝘰𝘰𝘧 𝘴𝘦𝘢𝘭𝘪𝘯𝘨 is tough: transmissions and active mechanisms mean moving parts and dynamic seals that are fragile and 𝘭𝘦𝘢𝘬-𝘱𝘳𝘰𝘯𝘦. We wanted an amphibious robot that stays sealed and robust – yet still supports multiple locomotion modes.

🧠✨ 𝗪𝗵𝗮𝘁 𝘄𝗲 𝗱𝗲𝘃𝗲𝗹𝗼𝗽𝗲𝗱:

An 𝗶𝗻𝗲𝗿𝘁𝗶𝗮-𝗱𝗿𝗶𝘃𝗲𝗻 𝗮𝗰𝘁𝘂𝗮𝘁𝗶𝗼𝗻 + 𝗽𝗮𝘀𝘀𝗶𝘃𝗲 𝗽𝗿𝗼𝗽𝘂𝗹𝘀𝗶𝗼𝗻 𝗱𝗲𝘀𝗶𝗴𝗻 that:

🔹 Uses a variable-output VCM inside a fully sealed rigid shell (no external moving parts)

🔹 Switches among three modes: jumping, full-stroke vibration (land), and small-stroke vibration (water)

🔹 Uses asymmetric, tilted passive fins for frequency-tuned steering in water (IDMP)

🔹 Explains hydrodynamics via aquatic tests, high-speed PIV, and CFD

All of this – 𝗼𝗻𝗲 𝗺𝗮𝗶𝗻 𝗹𝗶𝗻𝗲𝗮𝗿 𝗮𝗰𝘁𝘂𝗮𝘁𝗼𝗿 + 𝗽𝗮𝘀𝘀𝗶𝘃𝗲 𝗳𝗶𝗻𝘀. No exposed legs, gears, or propellers – making sealing and durability much easier at the centimeter scale.

🎯 𝗞𝗲𝘆 𝗥𝗲𝘀𝘂𝗹𝘁𝘀:

✅ 24-g prototype (57.5 × 36 × 36 mm) with a fully enclosed shell

✅ ~1.4 BL/s (~78 mm/s) on dry sand; 41.6 mm/s on flat ground

✅ Jump height up to 17.16 mm at 15 V; continuous “tumbler” jumping

✅ Load carrying: 960 g (~40× body weight) and escape under 5-kg loads

✅ In water: min turning radius 5.6 mm; straight swim at 35/44/60 Hz (~32/35/28 mm/s); peak yaw -22°/s (30 Hz) or 16°/s (40 Hz)

💡 𝗪𝗵𝘆 𝗶𝘁 𝗺𝗮𝘁𝘁𝗲𝗿𝘀:

This work shows how inertia + mode-switchable actuation can bridge the 𝗺𝗼𝗺𝗲𝗻𝘁𝘂𝗺-𝗳𝗿𝗲𝗾𝘂𝗲𝗻𝗰𝘆 𝘁𝗿𝗮𝗱𝗲-𝗼𝗳𝗳, enabling jumping and swimming in the same tiny robot. Passive asymmetric fins turn simple reciprocation into steerable thrust – 𝘄𝗶𝘁𝗵 𝗻𝗼 𝗮𝗱𝗱𝗲𝗱 𝗮𝗰𝘁𝘂𝗮𝘁𝗼𝗿𝘀.

🌱 𝗪𝗵𝗮𝘁’𝘀 𝗻𝗲𝘅𝘁?

Future work will focus on improving environmental adaptability via 𝘧𝘦𝘦𝘥𝘣𝘢𝘤𝘬 𝘤𝘰𝘯𝘵𝘳𝘰𝘭 and adaptive structures, and optimizing energy efficiency + onboard power 𝘮𝘪𝘯𝘪𝘢𝘵𝘶𝘳𝘪𝘻𝘢𝘵𝘪𝘰𝘯.

Special shoutout to the team –

Lingqi Tang, Yongzun Yang (co-first authors), Bing Li, Bingfu Zhang, Qiguang He, Hongliang Ren, Yao Li – for making this project possible.

🔗 Paper link: https://lnkd.in/gFDfZYKq

🔖 #ScienceAdvances #Robotics #AmphibiousRobots #Microrobots #InertiaDriven #Locomotion #VCM #PIV #CFD

🏥 Accurate visualization of subtle vascular dynamics remains a significant challenge in minimally invasive surgery, where dynamic complexities often limit decision-making reliability. Our paper introduces 𝗘𝗻𝗱𝗼𝗖𝗼𝗻𝘁𝗿𝗼𝗹𝗠𝗮𝗴, a framework designed to 𝗲𝗻𝗵𝗮𝗻𝗰𝗲 𝘃𝗮𝘀𝗰𝘂𝗹𝗮𝗿 𝗺𝗼𝘁𝗶𝗼𝗻 𝘃𝗶𝘀𝗶𝗯𝗶𝗹𝗶𝘁𝘆 in endoscopic videos while preserving surrounding tissue structure. The approach integrates 𝗣𝗲𝗿𝗶𝗼𝗱𝗶𝗰 𝗥𝗲𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗥𝗲𝘀𝗲𝘁𝘁𝗶𝗻𝗴 to minimize error accumulation over time and 𝗛𝗶𝗲𝗿𝗮𝗿𝗰𝗵𝗶𝗰𝗮𝗹 𝗧𝗶𝘀𝘀𝘂𝗲-𝗮𝘄𝗮𝗿𝗲 𝗠𝗮𝗴𝗻𝗶𝗳𝗶𝗰𝗮𝘁𝗶𝗼𝗻 for adaptive vessel tracking.

📊 To validate robustness, we constructed 𝗘𝗻𝗱𝗼𝗩𝗠𝗠𝟮𝟰, a benchmark dataset spanning four surgical specialties and diverse intraoperative scenarios. Quantitative metrics and expert surgeon evaluations indicate improved magnification accuracy and image quality compared to existing methods.

🤝 We extend our sincere gratitude to our collaborators across The Chinese University of Hong Kong (An Wang, Mengya Xu, Yiting Chang, Prof Hongliang Ren), The University of Hong Kong (Rulin Zhou), Southern Medical University (苟龙飞, Prof Hao Chen), The First Affiliated Hospital of Wenzhou Medical University (Yiru Ye), Southern University of Science and

Technology (Prof Jiankun Wang), and Singapore General Hospital (Prof Chwee Ming Lim) for their invaluable contributions to this multidisciplinary work.

The paper is available at https://lnkd.in/ggPEFswF

#MotionMagnification #SurgicalAI #Endoscopy

Thrilled to share our latest work, 𝐆𝐞𝐨𝐋𝐚𝐧𝐆, a unified geometry-aware framework for language-guided robotic grasping.

Language-guided grasping is a key capability for intuitive human–robot interaction. A robot should not only detect objects but also understand natural instructions such as “pick up the blue cup behind the bowl.” While recent multimodal models have shown promising results, most existing approaches rely on multi-stage pipelines that loosely couple perception and grasp prediction. These methods often overlook the tight integration of geometry, language, and visual reasoning, making them fragile in cluttered, occluded, or low-texture environments. This motivated us to bridge the gap between semantic language understanding and precise geometric grasp execution.

🧠✨ 𝐖𝐡𝐚𝐭 𝐰𝐞 𝐝𝐞𝐯𝐞𝐥𝐨𝐩𝐞𝐝:

A novel unified framework for geometry-aware language-guided grasping that includes:

🔹 Unified RGB-D Multimodal Representation:

 We embed RGB, depth, and language features into a shared representation space, enabling consistent cross-modal semantic alignment for accurate target reasoning.

🔹 Depth-Guided Geometric Module (DGGM):

 Instead of treating depth as auxiliary input, we explicitly inject geometric priors derived from depth into the attention mechanism, strengthening object discrimination under occlusion and ambiguous visual conditions.

🔹 Adaptive Dense Channel Integration (ADCI):

 A dynamic multi-layer fusion strategy that balances global semantic cues and fine-grained geometric details for robust grasp prediction.

🎯  𝐊𝐞𝐲 𝐑𝐞𝐬𝐮𝐥𝐭𝐬:

✅ GeoLanG significantly outperforms prior multi-stage baselines on OCID-VLG for language-guided grasping.

✅ Demonstrates strong robustness in cluttered and heavily occluded scenes.

✅ Successfully validated on real robotic hardware, showing reliable sim-to-real transfer.

💡 𝐖𝐡𝐲 𝐢𝐭 𝐦𝐚𝐭𝐭𝐞𝐫𝐬:

This work shows that tightly coupling geometric reasoning with multimodal language understanding can significantly enhance robotic grasp reliability. By embedding depth-aware geometric priors directly into attention mechanisms, we reduce ambiguity and improve consistency in grasp decision-making.

GeoLanG provides a pathway toward more intelligent robotic systems that understand not just what object to grasp, but also how to grasp it robustly in complex real-world environments.

🌱 𝐖𝐡𝐚𝐭’𝐬 𝐧𝐞𝐱𝐭?

We are exploring extending this geometry-aware multimodal reasoning toward:

 🔹 Real-time interactive grasping

 🔹 Multi-step manipulation tasks

 🔹 Integration with motion planning and autonomous robotic control

#ICRA2026 #CUHK