Category: News

🤖🪢
Thrilled to share our latest work published on Nature Communication, which redefines actuation for tendon-driven continuum robots — achieving full 3D omnidirectional motion and body twist using only a single tendon.

🧠✨ What we developed: A new class of continuum robots that:
🔹 Breaks Design Constraints: Eliminates the inherent trade-off between miniaturization and 3D manipulability by replacing multiple tendons with a single eccentric one.
🔹 Push-Pull-Twist Actuation: Achieves complex spatial movement through a unique driving mechanism.
🔹 High Efficiency: Features an outer diameter of 2.0–3.5 mm with a hollow ratio exceeding 57% — doubling the spatial utilization of traditional designs.
🔹 Open-Source Support: Includes a derived kinematics model and an open-source simulator for the robotics community.

🎯 Key Results:

• ✅ >1,000-fold Improvement: Massive increase in manipulability compared to conventional multi-tendon mechanisms.
• ✅ High Force Retention: Retains at least 70% of tip force across all directions.
• ✅ Versatile Demonstration: Proven success in teleoperation, navigation through tortuous environments, and “chopstick-like” continuum grippers.

💡 Why it matters: This work proves that miniature robots can maintain high dexterity and power without the bulk of traditional hardware, pointing toward the next generation of surgical actuators.

🌱 What’s next? We are exploring potential medical applications and the integration of these actuators into complex surgical procedures.

On the afternoon of April 24, 2026, Professor Hongliang Ren’s research team from The Chinese University of Hong Kong visited The Third Affiliated Hospital of Sun Yat-sen University (SYSU Third Hospital) to hold an academic symposium and salon on Embodied Medical Robotics. The event was held in Conference Room 2006 of the Comprehensive Building, aiming to advance academic communication and potential research collaboration in medical robotics and intelligent healthcare.

The meeting was chaired by Dr. Liu Zifeng, Director of the Big Data and Artificial Intelligence Center of SYSU Third Hospital. Vice President Qintai Yang delivered a welcome speech, introducing the hospital’s clinical strengths and looking forward to close cooperation in medical robot research and translation.

Professor Hongliang Ren delivered a keynote report on minimally invasive flexible robotic systems and embodied intelligence in medicine. Members of his research team then presented recent advances in precision interventional tools, magnetic robotic systems, endoscopic intelligent navigation, autonomous surgical control, and medical image perception, showcasing the team’s innovative work in clinical-oriented medical robotics.

During the discussion session, clinicians and researchers from multiple departments of SYSU Third Hospital had in-depth exchanges on clinical demands, technical applications, and joint research plans. Both sides reached positive consensus on future cooperation in scientific research, talent development, and clinical translation.

At the end of the meeting, Vice President Qintai Yang delivered a concluding speech and spoke highly of this academic exchange. This symposium effectively bridged engineering research and clinical practice, and laid a solid foundation for the future development and application of embodied medical robotics.

About The Third Affiliated Hospital of Sun Yat-sen University

The Third Affiliated Hospital of Sun Yat-sen University was founded in 1971 and is a comprehensive Grade-A tertiary hospital directly administered by the National Health Commission of China. As a major clinical teaching base of Sun Yat-sen University, the hospital undertakes medical care, education, research, prevention, rehabilitation, and specialist training. It currently operates four campuses: Tianhe, Lingnan, Yuedong, and Zhaoqing. The hospital has developed distinctive strengths in liver disease, brain disorders, and immune-related diseases, supported by national key disciplines and national clinical key specialty programs. It is also recognized as a Guangdong High-level Hospital and serves as an output hospital for the development of national regional medical centers.

Thrilled to share our latest international collaboration! At NVIDIA GTC 2026 in San Jose, CA, the team led by Professor Hongliang Ren from The Chinese University of Hong Kong (CUHK), in partnership with NVIDIA and 35 leading global institutions, officially released Open-H-Embodiment, the world’s first and largest open-source dataset for medical robotics, now available on HuggingFace.

During the GTC keynote, Kimberly Powell, NVIDIA’s VP of Healthcare, highlighted this milestone. Our lab is honored to be a primary contributor, filling the critical gap in Embodied AI for medical robotics by providing high-fidelity data for contact dynamics and closed-loop control.

🧠✨ What we contributed & developed:

This project breaks the “perception-heavy, execution-light” limitation of traditional medical AI. Key highlights include:

🔹 778 Hours of Massive Multimodal Data: The dataset covers 400 complete clinical surgeries and 9 major robotic platforms (e.g., dVRK, CMR Versius, Kuka). It includes 65% clinical data, 23% bench-top experiments, and 12% simulation data.

🔹 Three High-Value Specialized Datasets from Our Lab:

• Dual-Source Ultrasound Dataset: Experts-level trajectories covering in-vivo porcine EUS and human forearm scanning, overcoming complex organ environments and multi-device calibration.
• Robotic Surgery Skill Dataset: Multi-modal data (RGB/RGB-D + Kinematics) for tissue manipulation and suturing, featuring millisecond-level synchronization and dual-mode control (teleoperation & automation).
• Flexible Endoscope Tracking Baseline: A standardized dataset addressing hysteresis and deformation in flexible endoscopy, supporting nanosecond-level time synchronization.

🔹 Surgical VLA & World Models:

• GR00T-H: A 3B-parameter Vision-Language-Action model based on NVIDIA Isaac GR00T, capable of long-horizon dexterous tasks like end-to-end suturing.
• Cosmos-H-Surgical-Simulator: An action-conditioned world model that boosts simulation efficiency by over 70x, bridging the sim-to-real gap.

🎯 Key Results: ✅ Global Standardization: First effort to unify medical robotic data across different devices and institutions under CC-BY-4.0. ✅ Efficiency Boost: Accelerated surgical simulation (600 sims in 40 mins) to generate high-fidelity video-action pairs. ✅ Clinical Relevance: Successfully captured nearly 500 hours of real-world clinical data for hernia, gallbladder, and uterine surgeries.

💡 Why it matters: This initiative provides the foundational “bedrock” for Medical Physical AI. By sharing high-quality, synchronized data for surgery, ultrasound, and endoscopy, we are lowering the barrier for researchers worldwide to develop autonomous surgical agents that are both explainable and adaptive.

🌱 What’s next? Our lab is continuing to deepen research in: 🔹 Reasoning-based autonomous control for surgical robots. 🔹 Cross-platform generalization of Medical VLA models. 🔹 Clinical translation of Embodied AI to improve patient outcomes.

Datasets address: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Open-H-Embodiment

Project website: https://github.com/open-h

#NVIDIAGTC2026 #MedicalRobotics #EmbodiedAI #HuggingFace #CUHK #OpenSource #HealthcareInnovation

Thrilled to share our latest #𝗦𝗰𝗶𝗲𝗻𝗰𝗲𝗔𝗱𝘃𝗮𝗻𝗰𝗲 work on a 𝗰𝗲𝗻𝘁𝗶𝗺𝗲𝘁𝗲𝗿-𝘀𝗰𝗮𝗹𝗲, 𝗳𝘂𝗹𝗹𝘆 𝘀𝗲𝗮𝗹𝗲𝗱, 𝗹𝗲𝗴𝗹𝗲𝘀𝘀 𝗮𝗺𝗽𝗵𝗶𝗯𝗶𝗼𝘂𝘀 𝗿𝗼𝗯𝗼𝘁 that can crawl on sand, jump, and steerably swim – powered by a 𝘀𝗶𝗻𝗴𝗹𝗲 𝘃𝗮𝗿𝗶𝗮𝗯𝗹𝗲-𝗼𝘂𝘁𝗽𝘂𝘁 𝘃𝗼𝗶𝗰𝗲-𝗰𝗼𝗶𝗹 𝗺𝗼𝘁𝗼𝗿 (𝗩𝗖𝗠).

At small scales, reliable 𝘸𝘢𝘵𝘦𝘳𝘱𝘳𝘰𝘰𝘧 𝘴𝘦𝘢𝘭𝘪𝘯𝘨 is tough: transmissions and active mechanisms mean moving parts and dynamic seals that are fragile and 𝘭𝘦𝘢𝘬-𝘱𝘳𝘰𝘯𝘦. We wanted an amphibious robot that stays sealed and robust – yet still supports multiple locomotion modes.

🧠✨ 𝗪𝗵𝗮𝘁 𝘄𝗲 𝗱𝗲𝘃𝗲𝗹𝗼𝗽𝗲𝗱:

An 𝗶𝗻𝗲𝗿𝘁𝗶𝗮-𝗱𝗿𝗶𝘃𝗲𝗻 𝗮𝗰𝘁𝘂𝗮𝘁𝗶𝗼𝗻 + 𝗽𝗮𝘀𝘀𝗶𝘃𝗲 𝗽𝗿𝗼𝗽𝘂𝗹𝘀𝗶𝗼𝗻 𝗱𝗲𝘀𝗶𝗴𝗻 that:

🔹 Uses a variable-output VCM inside a fully sealed rigid shell (no external moving parts)

🔹 Switches among three modes: jumping, full-stroke vibration (land), and small-stroke vibration (water)

🔹 Uses asymmetric, tilted passive fins for frequency-tuned steering in water (IDMP)

🔹 Explains hydrodynamics via aquatic tests, high-speed PIV, and CFD

All of this – 𝗼𝗻𝗲 𝗺𝗮𝗶𝗻 𝗹𝗶𝗻𝗲𝗮𝗿 𝗮𝗰𝘁𝘂𝗮𝘁𝗼𝗿 + 𝗽𝗮𝘀𝘀𝗶𝘃𝗲 𝗳𝗶𝗻𝘀. No exposed legs, gears, or propellers – making sealing and durability much easier at the centimeter scale.

🎯 𝗞𝗲𝘆 𝗥𝗲𝘀𝘂𝗹𝘁𝘀:

✅ 24-g prototype (57.5 × 36 × 36 mm) with a fully enclosed shell

✅ ~1.4 BL/s (~78 mm/s) on dry sand; 41.6 mm/s on flat ground

✅ Jump height up to 17.16 mm at 15 V; continuous “tumbler” jumping

✅ Load carrying: 960 g (~40× body weight) and escape under 5-kg loads

✅ In water: min turning radius 5.6 mm; straight swim at 35/44/60 Hz (~32/35/28 mm/s); peak yaw -22°/s (30 Hz) or 16°/s (40 Hz)

💡 𝗪𝗵𝘆 𝗶𝘁 𝗺𝗮𝘁𝘁𝗲𝗿𝘀:

This work shows how inertia + mode-switchable actuation can bridge the 𝗺𝗼𝗺𝗲𝗻𝘁𝘂𝗺-𝗳𝗿𝗲𝗾𝘂𝗲𝗻𝗰𝘆 𝘁𝗿𝗮𝗱𝗲-𝗼𝗳𝗳, enabling jumping and swimming in the same tiny robot. Passive asymmetric fins turn simple reciprocation into steerable thrust – 𝘄𝗶𝘁𝗵 𝗻𝗼 𝗮𝗱𝗱𝗲𝗱 𝗮𝗰𝘁𝘂𝗮𝘁𝗼𝗿𝘀.

🌱 𝗪𝗵𝗮𝘁’𝘀 𝗻𝗲𝘅𝘁?

Future work will focus on improving environmental adaptability via 𝘧𝘦𝘦𝘥𝘣𝘢𝘤𝘬 𝘤𝘰𝘯𝘵𝘳𝘰𝘭 and adaptive structures, and optimizing energy efficiency + onboard power 𝘮𝘪𝘯𝘪𝘢𝘵𝘶𝘳𝘪𝘻𝘢𝘵𝘪𝘰𝘯.

Special shoutout to the team –

Lingqi Tang, Yongzun Yang (co-first authors), Bing Li, Bingfu Zhang, Qiguang He, Hongliang Ren, Yao Li – for making this project possible.

🔗 Paper link: https://lnkd.in/gFDfZYKq

🔖 #ScienceAdvances #Robotics #AmphibiousRobots #Microrobots #InertiaDriven #Locomotion #VCM #PIV #CFD

🏥 Accurate visualization of subtle vascular dynamics remains a significant challenge in minimally invasive surgery, where dynamic complexities often limit decision-making reliability. Our paper introduces 𝗘𝗻𝗱𝗼𝗖𝗼𝗻𝘁𝗿𝗼𝗹𝗠𝗮𝗴, a framework designed to 𝗲𝗻𝗵𝗮𝗻𝗰𝗲 𝘃𝗮𝘀𝗰𝘂𝗹𝗮𝗿 𝗺𝗼𝘁𝗶𝗼𝗻 𝘃𝗶𝘀𝗶𝗯𝗶𝗹𝗶𝘁𝘆 in endoscopic videos while preserving surrounding tissue structure. The approach integrates 𝗣𝗲𝗿𝗶𝗼𝗱𝗶𝗰 𝗥𝗲𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗥𝗲𝘀𝗲𝘁𝘁𝗶𝗻𝗴 to minimize error accumulation over time and 𝗛𝗶𝗲𝗿𝗮𝗿𝗰𝗵𝗶𝗰𝗮𝗹 𝗧𝗶𝘀𝘀𝘂𝗲-𝗮𝘄𝗮𝗿𝗲 𝗠𝗮𝗴𝗻𝗶𝗳𝗶𝗰𝗮𝘁𝗶𝗼𝗻 for adaptive vessel tracking.

📊 To validate robustness, we constructed 𝗘𝗻𝗱𝗼𝗩𝗠𝗠𝟮𝟰, a benchmark dataset spanning four surgical specialties and diverse intraoperative scenarios. Quantitative metrics and expert surgeon evaluations indicate improved magnification accuracy and image quality compared to existing methods.

🤝 We extend our sincere gratitude to our collaborators across The Chinese University of Hong Kong (An Wang, Mengya Xu, Yiting Chang, Prof Hongliang Ren), The University of Hong Kong (Rulin Zhou), Southern Medical University (苟龙飞, Prof Hao Chen), The First Affiliated Hospital of Wenzhou Medical University (Yiru Ye), Southern University of Science and

Technology (Prof Jiankun Wang), and Singapore General Hospital (Prof Chwee Ming Lim) for their invaluable contributions to this multidisciplinary work.

The paper is available at https://lnkd.in/ggPEFswF

#MotionMagnification #SurgicalAI #Endoscopy

Thrilled to share our latest work, 𝐆𝐞𝐨𝐋𝐚𝐧𝐆, a unified geometry-aware framework for language-guided robotic grasping.

Language-guided grasping is a key capability for intuitive human–robot interaction. A robot should not only detect objects but also understand natural instructions such as “pick up the blue cup behind the bowl.” While recent multimodal models have shown promising results, most existing approaches rely on multi-stage pipelines that loosely couple perception and grasp prediction. These methods often overlook the tight integration of geometry, language, and visual reasoning, making them fragile in cluttered, occluded, or low-texture environments. This motivated us to bridge the gap between semantic language understanding and precise geometric grasp execution.

🧠✨ 𝐖𝐡𝐚𝐭 𝐰𝐞 𝐝𝐞𝐯𝐞𝐥𝐨𝐩𝐞𝐝:

A novel unified framework for geometry-aware language-guided grasping that includes:

🔹 Unified RGB-D Multimodal Representation:

 We embed RGB, depth, and language features into a shared representation space, enabling consistent cross-modal semantic alignment for accurate target reasoning.

🔹 Depth-Guided Geometric Module (DGGM):

 Instead of treating depth as auxiliary input, we explicitly inject geometric priors derived from depth into the attention mechanism, strengthening object discrimination under occlusion and ambiguous visual conditions.

🔹 Adaptive Dense Channel Integration (ADCI):

 A dynamic multi-layer fusion strategy that balances global semantic cues and fine-grained geometric details for robust grasp prediction.

🎯  𝐊𝐞𝐲 𝐑𝐞𝐬𝐮𝐥𝐭𝐬:

✅ GeoLanG significantly outperforms prior multi-stage baselines on OCID-VLG for language-guided grasping.

✅ Demonstrates strong robustness in cluttered and heavily occluded scenes.

✅ Successfully validated on real robotic hardware, showing reliable sim-to-real transfer.

💡 𝐖𝐡𝐲 𝐢𝐭 𝐦𝐚𝐭𝐭𝐞𝐫𝐬:

This work shows that tightly coupling geometric reasoning with multimodal language understanding can significantly enhance robotic grasp reliability. By embedding depth-aware geometric priors directly into attention mechanisms, we reduce ambiguity and improve consistency in grasp decision-making.

GeoLanG provides a pathway toward more intelligent robotic systems that understand not just what object to grasp, but also how to grasp it robustly in complex real-world environments.

🌱 𝐖𝐡𝐚𝐭’𝐬 𝐧𝐞𝐱𝐭?

We are exploring extending this geometry-aware multimodal reasoning toward:

 🔹 Real-time interactive grasping

 🔹 Multi-step manipulation tasks

 🔹 Integration with motion planning and autonomous robotic control

#ICRA2026 #CUHK

Thrilled to share our latest work on enabling robust sparse-to-dense reconstruction for endoscopic surgical robots — bridging the gap between 𝐬𝐩𝐚𝐫𝐬𝐞 𝐬𝐞𝐧𝐬𝐨𝐫 𝐝𝐚𝐭𝐚 𝐚𝐧𝐝 𝐡𝐢𝐠𝐡-𝐪𝐮𝐚𝐥𝐢𝐭𝐲 𝟑𝐃 𝐦𝐚𝐩𝐩𝐢𝐧𝐠 using a novel 𝐝𝐢𝐟𝐟𝐮𝐬𝐢𝐨𝐧-𝐛𝐚𝐬𝐞𝐝 framework.

Fine-tuning foundational models often fails due to a lack of dense ground truth, and self-supervised methods struggle with scale ambiguity, sparse depth sensors offer a reliable geometric prior.

This motivated us to develop EndoDDC, a method that robustly generates dense depth maps by fusing RGB images with sparse depth inputs.

🧠✨ 𝐖𝐡𝐚𝐭 𝐰𝐞 𝐝𝐞𝐯𝐞𝐥𝐨𝐩𝐞𝐝:

A diffusion-driven depth completion architecture that:

🔹 Integrates sparse depth and RGB inputs to overcome the limitations of pure visual estimation.

🔹 Utilizes a Multi-scale Feature Extraction and Depth Gradient Fusion module to capture fine-grained surface orientation and local structure.

🔹 Optimizes depth maps iteratively using a conditional diffusion model, refining geometry even in regions with weak textures or reflections.

🎯 𝐊𝐞𝐲 𝐑𝐞𝐬𝐮𝐥𝐭𝐬:

✅ 25.55% and 9.03% improvement in accuracy on the StereoMIS and C3VD dataset compared to SOTA surgical estimators like EndoDAC.

✅ 7.35% and 5.28% reduction in RMSE on StereoMIS and C3VD compared to the best depth completion baseline (OGNI-DC).

✅ Outperformed foundational models (DepthAnything-v2) and standard depth completion (Marigold-DC) methods in both accuracy and robustness.

💡 𝐖𝐡𝐲 𝐢𝐭 𝐦𝐚𝐭𝐭𝐞𝐫𝐬:

This work demonstrates that diffusion models can effectively solve the “sparse-to-dense” challenge in medical imaging. By providing accurate depth completion despite complex lighting and texture conditions, EndoDDC has the potential to significantly enhance autonomous navigation, procedural safety, and spatial awareness in minimally invasive surgery.

🔖 #DepthCompletion #DiffusionModel #EndoscopicSurgery #SurgicalNavigation #ICRA #CUHKEngineering #CUHK

We present 𝐍𝐞𝐮𝐫𝐨-𝐕𝐋𝐀, an scenario-aware model designed for the motion control of a parallel continuum neurosurgical robot.

Robotic surgery systems have garnered significant attention for their precision and efficiency, yet achieving autonomous tasks in complex neurosurgical environments remains challenging. Although Vision-Language-Action (VLA) models hold great potential, their development is constrained by the scarcity of data from surgical environments and robotic kinematics. To address this issue, this paper proposes NeuroVLA: a VLA model specifically designed for neurosurgical robotic tumor debulking tasks. Through phantom experiments conducted on a flexible parallel continuum robot, we constructed a dataset and decomposed the debulking task into four skill-based instructions. NeuroVLA utilizes a Vision-Language Model (VLM) as its backbone for scene reasoning, enabling the robot to comprehend the surgical scene and its own state. Experimental results demonstrate that after training on 90 debulking segments, NeuroVLA can infer actions based on images, language instructions, and the robot’s state. It achieved average pixel distance errors of 29.10 pixels and 21.55 pixels for the “alignment” and “transfer” skills, respectively, and success rates of 88.89% and 100% for the “grasping” and “release” skills.

🧠 Technical Framework:

●     End-to-End scenario-aware VLA model

●     Skill-based scenario infer mechanism

●     Debulking task dataset in neurosurgery

🎯 Experimental Results:

●     NeuroVLA demonstrates significantly lower pixel distance (PD) errors in the “alignment” and “transfer” skills (29.10 px / 21.55 px), far surpassing the performance of baseline models (such as Octo’s 79.72 px / 65.46 px).

In the “grasping” and “release” skills, NeuroVLA exhibits greater robustness, achieving a grasping success rate of 88.89% and a release success rate of 100%. In contrast, baseline models often misinterpret incomplete forceps closure as task completion, leading to grasping failures.

#ICRA2026

We present 𝐊𝐢𝐫𝐢-𝐂𝐚𝐩𝐬𝐮𝐥𝐞, a swallowable kirigami-inspired capsule robot that enables minimally invasive GI biopsy—pushing capsule endoscopy from imaging to tissue sampling.

Wireless capsule endoscopy is comfortable and accessible, but cannot collect biopsy tissue, while histology is still the gold standard. Our work targets safe, depth-controlled, retrievable sampling in a capsule form factor.

🧠 Technical Framework:

●     Kirigami PI skin: flat during locomotion, deploys sharp protrusions when stretched

●     Dual-cam actuation: compact, repeatable deployment + recovery

●     Rotary scraping + internal storage: detach tissue and store it in internal cavities for retrieval

🎯 Experimental Results:

●     Penetration depth: median ~0.61 mm (0.46–0.66 mm) on ex vivo porcine tissue

●     Biopsy yield: ~10.9 mg (stomach) / 18.9 mg (small intestine), histology-ready (mucosa + submucosa)

●     Forces within reported GI biopsy safety ranges

💡 Significance:

A practical route toward capsule-based biopsy with controlled shallow interaction and retrievable specimens—without bulky endoscopic tools.

🌱 Next:

Moving toward untethered actuation (e.g., magnetic drive) and multi-site sampling to reduce mixing risk.

🔖 #ICRA2026 #SoftRobotics #MedicalRobotics #CapsuleEndoscopy #Kirigami #BioInspiredRobotics #MinimallyInvasive #Biopsy #GI #RoboticsResearch

𝐔𝐧𝐝𝐞𝐫𝐬𝐭𝐚𝐧𝐝𝐢𝐧𝐠 𝐰𝐢𝐭𝐡 𝐋𝐚𝐫𝐠𝐞 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐌𝐨𝐝𝐞𝐥 𝐢𝐧 𝐑𝐨𝐛𝐨𝐭-𝐚𝐬𝐬𝐢𝐬𝐭𝐞𝐝 𝐒𝐮𝐫𝐠𝐞𝐫𝐲!

Thrilled to share our latest work, 𝐒𝐮𝐫𝐠𝐕𝐢𝐝𝐋𝐌, the first video-language model specifically designed to address both full and fine-grained surgical video comprehension.

Surgical scene understanding is critical for training and robotic decision-making. While current Multimodal Large Language Models (MLLMs) excel at image analysis, they often overlook the fine-grained temporal reasoning required to capture detailed task execution and specific procedural processes within a surgery. This motivated us to bridge the gap between global video understanding and micro-action analysis.

🧠✨ What we developed:

A novel framework and resource for surgical video reasoning that includes:

🔹 𝐓𝐰𝐨-𝐬𝐭𝐚𝐠𝐞 𝐒𝐭𝐚𝐠𝐞𝐅𝐨𝐜𝐮𝐬 𝐦𝐞𝐜𝐡𝐚𝐧𝐢𝐬𝐦: The first stage extracts global procedural context, while the second stage performs high-frequency local analysis for fine-grained task execution.

🔹 𝐌𝐮𝐥𝐭𝐢-𝐟𝐫𝐞𝐪𝐮𝐞𝐧𝐜𝐲 𝐅𝐮𝐬𝐢𝐨𝐧 𝐀𝐭𝐭𝐞𝐧𝐭𝐢𝐨𝐧 (𝐌𝐅𝐀): Effectively integrates low-frequency global features with high-frequency local details to ensure comprehensive scene perception.

🔹 𝐒𝐕𝐔-𝟑𝟏𝐊 𝐃𝐚𝐭𝐚𝐬𝐞𝐭: We constructed a large-scale dataset with over 31,000 video-instruction pairs, featuring hierarchical knowledge representation for enhanced visual reasoning.

🎯 Key Results:

✅ SurgVidLM significantly outperforms existing models (like Qwen2-VL) in multi-grained surgical video understanding tasks.

✅ Capable of inferring anatomical landmarks (e.g., Denonvilliers’ fascia) and providing clinical motivation, moving beyond simple visual description.

✅ Demonstrated strong performance on unseen surgical tasks, proving the robustness of our hierarchical training approach.

💡 Why it matters:

This work shows that by combining global context with localized high-frequency focus, we can significantly reduce “hallucinations” in surgical AI. It provides a pathway toward more intelligent, context-aware surgical assistants that can understand not just what is happening, but how and why specific steps are performed.

🌱 What’s next?

We are exploring how to extend this multi-grained understanding to real-time intraoperative guidance and integrating it with physical robotic control for autonomous sub-tasks.