Robotics, Embodied AI, Navigation in vivo

The 3rd C4SR+ Workshop: Continuum, Compliant, Cooperative, Cognitive Surgical Robotic Systems in the Embodied Al Era.

In this project, we tackled the unique challenges of robotic endoscopy by integrating vision, language grounding, and motion planning into one end-to-end framework. EndoVLA enables:

– Precise polyp tracking through surgeon-issued prompts

– Delineation and following of abnormal mucosal regions

– Adherence to circumferential cutting markers during resections

We introduced a dual-phase training strategy:

1. 𝐒𝐮𝐩𝐞𝐫𝐯𝐢𝐬𝐞𝐝 𝐟𝐢𝐧𝐞-𝐭𝐮𝐧𝐢𝐧𝐠 on our new 𝐄𝐧𝐝𝐨𝐕𝐋𝐀-𝐌𝐨𝐭𝐢𝐨𝐧 dataset

2. 𝐑𝐞𝐢𝐧𝐟𝐨𝐫𝐜𝐞𝐦𝐞𝐧𝐭 𝐟𝐢𝐧𝐞-𝐭𝐮𝐧𝐢𝐧𝐠 with task-aware rewards

This approach impressively boosts tracking accuracy and achieves zero-shot generalization across diverse GI scenes.

The paper is available at: https://lnkd.in/g35DF7Fq

We built 𝐂𝐨𝐏𝐄𝐒𝐃 to help AI better understand surgical workflows—especially the complex motions involved in 𝐄𝐧𝐝𝐨𝐬𝐜𝐨𝐩𝐢𝐜 𝐒𝐮𝐛𝐦𝐮𝐜𝐨𝐬𝐚𝐥 𝐃𝐢𝐬𝐬𝐞𝐜𝐭𝐢𝐨𝐧 (𝐄𝐒𝐃). The dataset includes:

📹 35+ hours of annotated surgical videos

🖼️ 17,679 labeled frames

🔍 88,395 motion annotations across multiple levels

We designed a hierarchical annotation scheme to capture fine-grained surgical motions, especially focusing on the submucosal dissection phase. Our goal is to enable 𝐯𝐢𝐬𝐢𝐨𝐧-𝐥𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐦𝐨𝐝𝐞𝐥𝐬 that can one day assist surgeons in real-time—like a smart co-pilot in the OR.

Thanks to all collaborators (Guankun Wang, Han Xiao, Huxin Gao, Renrui Zhang, Long Bai, Xiaoxiao Yang, Zhen Li, Hongsheng Li,

Hongliang Ren) and institutions (CUHK, Shanghai AI Lab, Qilu Hospital of SDU) involved. We’re excited to see how this dataset can push forward research in 𝐬𝐮𝐫𝐠𝐢𝐜𝐚𝐥 𝐀𝐈, 𝐫𝐨𝐛𝐨𝐭𝐢𝐜𝐬, and 𝐦𝐮𝐥𝐭𝐢𝐦𝐨𝐝𝐚𝐥 𝐥𝐞𝐚𝐫𝐧𝐢𝐧𝐠.

📄 Check out the paper: https://lnkd.in/gkF6A4QY

In this work, we tackled the challenge of training models that can keep learning new surgical instruments over time, without forgetting the old ones. Data imbalance made this especially tricky, so we proposed a plug-and-play framework that balances the data using inpainting and blending techniques, and introduced a new loss function to reduce confusion between similar-looking tools.

Big thanks to our amazing team (Shifang Zhao, Long Bai, Kun Yuan, Feng Li, Jieming YU, Wenzhen Dong, Guankun Wang, Prof. Mobarak I. Hoque, Prof. Nicolas Padoy, Prof. Nassir Navab, Prof. Hongliang Ren) from CUHK, TUM, Strasbourg, and UCL. This collaboration truly brought together ideas from different corners of the world 🌍

The paper is now online: https://lnkd.in/gemZFNUK Code coming soon 👨‍💻

Join us in the breathtaking landscapes of 𝐙𝐡𝐚𝐧𝐠𝐲𝐞, China for the 𝟐𝟎𝟐𝟓 𝐈𝐧𝐭𝐞𝐫𝐧𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐂𝐨𝐧𝐟𝐞𝐫𝐞𝐧𝐜𝐞 𝐨𝐧 𝐁𝐢𝐨𝐦𝐢𝐦𝐞𝐭𝐢𝐜 𝐈𝐧𝐭𝐞𝐥𝐥𝐢𝐠𝐞𝐧𝐜𝐞 𝐚𝐧𝐝 𝐑𝐨𝐛𝐨𝐭𝐢𝐜𝐬 (𝐈𝐂𝐁𝐈𝐑), an affiliated event of the 𝐐𝟏 Elsevier journal 𝐁𝐢𝐨𝐦𝐢𝐦𝐞𝐭𝐢𝐜 𝐈𝐧𝐭𝐞𝐥𝐥𝐢𝐠𝐞𝐧𝐜𝐞 𝐚𝐧𝐝 𝐑𝐨𝐛𝐨𝐭𝐢𝐜𝐬 (IF 5.4).

We welcome original contributions covering:

• Biomimetic design, materials & actuation

• Bio-inspired sensing, perception & navigation

• Learning-based control & embodied AI

• Soft & adaptive robotics

• Novel real-world applications integrating theory and practice

All accepted papers will be published by Elsevier and indexed in EI & Scopus. Top-ranked submissions will earn best-paper awards and invitations to submit expanded versions to Biomimetic Intelligence and Robotics and other leading journals.

𝐊𝐞𝐲 𝐃𝐚𝐭𝐞𝐬 • Full-Paper (or Short Abstract) submissions due → July 20, 2025 • Acceptance notifications → August 1, 2025 • Registration & final manuscript → August 10, 2025

Learn more & submit at ▶️ http://www.icbir.org

Let’s decode nature’s genius and engineer the next generation of intelligent machines—together! 🌿🤖

The paper is accepted for presentation at IPCAI2025, and we’re especially humbled to receive the 𝗹𝗛𝗨 𝗦𝘁𝗿𝗮𝘀𝗯𝗼𝘂𝗿𝗴 𝗮𝗻𝗱 𝗡𝗗𝗹 𝗕𝗲𝗻𝗰𝗵 𝘁𝗼 𝗕𝗲𝗱𝘀𝗶𝗱𝗲 𝗔𝘄𝗮𝗿𝗱: 𝗛𝗼𝗻𝗼𝗿𝗮𝗯𝗹𝗲 𝗠𝗲𝗻𝘁𝗶𝗼𝗻.

In this work, we address the challenge of accurately delineating the dissection zones during endoscopic submucosal dissection procedures. By integrating flexible visual cues—such as scribbles and bounding boxes—directly onto surgical images, our PDZSeg model guides segmentation for both better precision and enhanced safety. Leveraging a state-of-the-art foundation model (DINOv2) and an efficient LoRA training strategy, we fine-tuned our approach on the specialized ESD-DZSeg dataset. Our experimental results show promising improvements over traditional methods, offering robust support for intraoperative guidance and remote surgical training.

Our sincere thanks to every colleague (Mengya Xu, Wenjin Mo, Guankun Wang, Huxin Gao, An Wang), mentor (Dr. Ning Zhong, Dr. Zhen Li, Dr. Xiaoxiao Yang, Prof. Hongliang Ren), and community member whose support has been indispensable. This achievement reaffirms our collective effort and inspires us to further refine robotic-assisted techniques towards enhanced safety and effectiveness.

Paper available at: https://lnkd.in/g7KcytnE

Dr. Yuan’s research advances the field of medical robotics, paving the way for innovative healthcare solutions. A well-deserved milestone after years of dedication and research!

Special thanks to his supervisor: Prof. Hongliang Ren and Prof. Max Q.-H. Meng, as well as the examining committee members: Prof. Gao Shichang, Prof. Wu ‘Scott’ YUAN, and Prof. Zhidong Wang for their invaluable guidance and support.

🔗 Learn more about Dr. Yuan’s research on https://lnkd.in/g7GVhwtv.

“𝐿𝑜𝑤-𝑆𝑡𝑟𝑎𝑖𝑛 𝐹𝑙𝑒𝑥𝑖𝑏𝑙𝑒 𝐼𝑛𝑘-𝐵𝑎𝑠𝑒𝑑 𝑆𝑒𝑛𝑠𝑜𝑟𝑠 𝐸𝑛𝑎𝑏𝑙𝑒 𝐻𝑦𝑝𝑒𝑟𝑒𝑙𝑎𝑠𝑡𝑖𝑐 𝐼𝑛𝑓𝑙𝑎𝑡𝑖𝑜𝑛 𝑃𝑒𝑟𝑐𝑒𝑝𝑡𝑖𝑜𝑛 𝑉𝑖𝑎 𝐺𝑒𝑜𝑚𝑒𝑡𝑟𝑖𝑐 𝑃𝑎𝑡𝑡𝑒𝑟𝑛𝑖𝑛𝑔”

📄 What we explored:

– Designed Ω-shaped flexible sensors using conductive ink to reduce strain mismatches on inflatable robots

– Developed a light-curing transfer method for precise sensor attachment

– Tested integration with balloon-type robots showing improved deformation tracking at >300% expansion

🩺 Our sensor system has the potential to enable:

🔹 Real-time inflation tracking for safer human-robot interaction

🔹 Spatial perception in biomedical devices (catheters, surgical tools)

🔹 5x reduction in circumference/area error vs. conventional designs

Congrats to the authors: WENCHAO YUE, Shuoyuan Chen, Yan Ke, Yingyi Wen, Ruijie Tang, Guohua Hu, and Hongliang Ren.

📄 Paper Open Access at: https://lnkd.in/gm9DRybP

We extend our sincere gratitude to Prof. Tan Lee, Prof. Qi Dou, and Prof. S. Kevin Zhou for serving as examiners during Long Bai’s defense. Special thanks to his supervisors, Prof. Hongliang Ren and Prof. Jiewen Lai, for their invaluable guidance throughout his Ph.D. journey.

During his time at CUHK RenLab, Dr. Bai has made impressive contributions to surgical and medical artificial intelligence, particularly in multimodal AI.

🔗 For more details about his research, visit his personal website: longbai-cuhk.github.io.

Wishing Dr. Long Bai all the best in his future endeavors! 🚀👏