During the 2nd Hong Kong Clinical-Driven Robotics and Embodied AI TEchnology (#CREATE) Symposium held by Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, CAS, our lab lead Prof. Hongliang Ren gave a talk entiled “Compliant Endoscopic Multisensory Guidance With Soft Flexible Robotics”, featuring our recent research progress in multisensory perception for endoluminal soft robotic operation.
A particular highlight was our effort towards automated and intelligent Endoscopic Submucosal Dissection (ESD) surgery, empowered by our indigenous DREAMS (Dual-arm Robotic Endoscopic Assistant for Minimally Invasive Surgery, https://lnkd.in/gQ3PcMsn) platform. We demonstrated how this platform can enhance ESD procedures when equipped with advanced features such as precise trajectory planning, intelligent cutting decision support, accurate reconstruction, and granular analysis and prediction of motion (https://lnkd.in/gkF6A4QY) empowered by Large Visual-Language Models (LVLM).
In addition to our progress in ESD, we also presented other projects, including surgical scene reconstruction & depth estimation (https://lnkd.in/gviyHxAp, https://lnkd.in/gtDwbyWg), augmented reality applications in surgery, the skull-mounted neuro-interventional robot (SkullBot, https://lnkd.in/gn8N9zdU), and OCT for port-wine stain analysis (https://lnkd.in/gm6iH3-m).
This visit not only provided an opportunity for knowledge exchange and collaboration but also reaffirmed our lab’s commitment to pushing the boundaries of innovation in the fields of medical robotics and intelligent surgical technologies. We look forward to furthering these conversations and forging new partnerships that will drive the future of healthcare forward.
On the Info Day for Undergraduate Admissions 2024, we were thrilled to see a big crowd of visitors joining the admission talk and special event โ๐๐๐ฅ๐ฅ-๐๐จ๐ฎ๐ง๐๐๐ ๐๐ง๐ ๐ข๐ง๐๐๐ซ ๐๐ง๐๐ฎ๐๐๐ญ๐ข๐จ๐ง ๐๐ก๐จ๐ฐ๐ซ๐จ๐จ๐ฆโ to learn the experience in developing APP / Products / Services.
Our lab member, Sishen YUAN, along with others, showcased our progress in augmented reality for neuro-interventional head-mounted robotics (SkullBot), attracting a lot of interest from the visitors and sparking engaging conversations about the future of surgical technology.
Thank you to everyone who joined us and showed interest in our work. We look forward to more opportunities to share our passion and advancements with the surgical robotics community!
We are thrilled to share a new research paper that’s just been published in *Communications Engineering*, showcasing a remarkable advancement in the field of origami-inspired tiny robots. ๐ฐโจ
**๐ [Untethered Bistable Origami Crawler for Confined Applications](https://lnkd.in/gpJHEM-q)
### What’s the Buzz About?
This research introduces a **magnetically actuated bistable origami crawler**, a miniature robot designed to navigate and perform tasks in confined spaces which are challenging for traditional tethered or wired devices. ๐
### Key Highlights:
– **Shape-Morphing Capability**: The crawler can transform between an undeployed locomotion state and a deployed load-bearing state, thanks to its bistable design. ๐ง
– **Robust Locomotion**: Utilizes out-of-plane crawling for bi-directional locomotion and navigation, exhibiting robust navigation even in high-friction environments. ๐ค๏ธ
– **Load-Bearing Applications**: The deployed state allows the crawler to execute tasks like microneedle insertion, opening up possibilities for medical interventions. ๐ฉบ
– **Untethered Operation**: Equipped with internal permanent magnets, this crawler operates without the need for external tethers, enhancing its maneuverability and miniaturizability. ๐ช
### Why It Matters:
This technology could provide an alternative approach to solve problems encountered in confined environments, from medical procedures in the gastrointestinal tract to complex engineering tasks in tight spots. ๐
### What’s Next:
The concept proposed in this work can also be adapted and applied to a variety of other deployable and load-bearing applications, such as fluid collection, stents, and airway support mechanisms. The proposed mechanism design can also be potentially integrated with other actuation methods like pneumatic systems for larger scale applications. ๐
### Join the Conversation:
This work is a collaborative effort between Dr. Catherine Cai from National University of Singapore, Dr. Hui Huang from A*STAR – Agency for Science, Technology and Research and Prof. Hongliang Ren from The Chinese University of Hong Kong.
We’re eager to hear your thoughts on what we hope is an innovative research! How do you envision this technology being used in your field? Share your ideas and let’s discuss the future of robotics and origami engineering! ๐ค๐
—
*Don’t forget to check out the full paper for a deep dive into the mechanics, applications, and implications of this incredible new technology. It’s a must-read for anyone interested in the cutting edge of robotics and engineering innovation!* ๐๐ก
We’re delighted to share that 6 papers from our lab have been accepted at IEEE ROBIO 2024 (https://lnkd.in/gXQYD7By)!
Our team has made significant contributions to the field of robotic surgery, with a diverse range of research topics
including:
– Neural Rendering
– Gaussian Splatting
– Registration and Reconstruction
– Augmented Reality for surgical planning and training
– Soft Continuum Robots
– Vision-guided surgery
Get a sneak peek at our digest figures below. We’ll be sharing more details soon! If you’re attending #ROBIO2024 in Bangkok,
we’d love to connect and discuss potential collaborations!
Let’s build connections and drive innovation together!
Our paper entitled “LighTDiff: Surgical Endoscopic Image Low-Light Enhancement with T-Diffusion”, led by our former lab member Tong Chen, Qingcheng Lyu, and our PhD student Long Bai, has been awarded the ๐MICCAI 2024 Best Paper Runner-Up!๐
This is a collaborative work between the lab of Prof Luping Zhou from The University of Sydney, our #LabREN (http://www.labren.org/mm/), and Qilu Hospital of Shandong University. Congrats to all coauthors!!!
We have also ranked #2nd in the very competitive BraTS Challenge on Sub-Sahara-Africa Adult Glioma (BraTS-SSA)! Our methodology is so-called “Transferring Knowledge from High-Quality to Low-Quality MRI for Adult Glioma Diagnosis”. Stay tuned for more updates!
Congrats to Team members: Yanguang Zhao, Long Bai, Zhaoxi Zhang, and Prof Hongliang Ren from #LabREN at The Chinese University of Hong Kong, Dr. Yanan Wu from China Medical University, and Dr. Mobarakol Islam from University College London.
Robotic endoscopes play a crucial role in diagnosing gastrointestinal disease and performing tumor resections. While current research primarily focuses on autonomously controlling rigid robots, establishing control models for flexible robots remains challenging. To address this, model-free deep reinforcement learning (DRL) presents a promising approach for enabling agents to make decisions under uncertainty.
In this paper, we investigate the control policy of a flexible endoscope using Simulation Open Framework Architecture (SOFA) platform. We design a flexible tendon-driven robotic endoscope (TDRE) and develop a custom simulation environment within SOFA to train DRL agents. Our approach involves implementing the Proximal Policy Optimization (PPO) algorithm to approximate an optimal policy for trajectory planning. The optimal policy facilitates trajectory tracking tasks for the TDREโs end-effector, such as circle trajectories and action disturbances, without requiring fine-tuning policy network parameters. Experimental results demonstrate that our approach achieves near real-time performance (30 FPS). The feedforward neural network of the policy provides feedback, enabling closed-loop control of TDRE. Furthermore, our experiments show that the navigation success rate of TDRE exceeds 90% within a tolerant error of 3 mm in free space. Notably, compared to direct training with contact, navigation tasks with contact retrained by a pre-trained policy in free space exhibit enhanced navigation capabilities.
Paper link: https://lnkd.in/gDH5xYA5
Co-authors: Chi Kit Ng; Huxin Gao; Tian-Ao Ren; Sam, Jiewen Lai; Hongliang Ren
We will present 4 main conference papers and 4 workshop/challenge papers in #MICCAI2024, Marrakesh, Morocco. These cover interesting topics such as #DDPM, Low-light Image Enhancement, #GaussianSplatting, Depth Reconstruction, Data Robustness, and Medical Image Segmentation. Congrates to all of our awesome collaborators! Do drop by our poster and oral sessions if you are interested in our work!
Main Conference 1: EndoUIC: Promptable Diffusion Transformer for Unified Illumination Correction in Capsule Endoscopy
Long Bai, Oct 08, Poster Session 3, 10:30 – 11:30
Poster ID: T-AM-091
Paper: https://lnkd.in/gV5MzKct
Code: https://lnkd.in/ghYauAGM
Main Conference 2: Endo-4DGS: Endoscopic Monocular Scene Reconstruction with 4D Gaussian Splatting
YIMING HUANG, Oct 08, Poster Session 4, 15:00 – 16:30
Post ID: T-PM-074
Paper: https://lnkd.in/gtDwbyWg
Code: https://lnkd.in/ggaDgVxW
Main Conference 3: EndoDAC: Efficient Adapting Foundation Model for Self-Supervised Depth Estimation from Any Endoscopic Camera
Beilei Cui, Oct 08, Poster Session 4, 15:00 – 16:30
Poster ID: T-PM-076
Paper: https://lnkd.in/gQgpFFpq
Code: https://lnkd.in/g_bfk56S
Main Conference 4: LighTDiff: Surgical Endoscopic Image Low-Light Enhancement with T-Diffusion
Tong Chen, Oct 09, Oral Session 16, 13:30 – 15:00, Poster Session 6, 15:00 – 16:30
Poster ID: W-PM-154
Paper: https://lnkd.in/gybFHPmu
Code: https://lnkd.in/gmmWXCnd
Workshop 1: A Review of 3D Reconstruction Techniques for Deformable Tissues in Robotic Surgery
Long Bai, Oct 06, Oral & Poster
Embodied AI and Robotics for HealTHcare (EARTH) Workshop
Paper: https://lnkd.in/gD6juYyV
Code: https://lnkd.in/gnwhzrQn
Workshop 2: Benchmarking Robustness of Endoscopic Depth Estimation with Synthetically Corrupted Data
Beilei Cui, Oct 10, Poster, 15:10 – 16:05
9th International Workshop on Simulation and Synthesis in Medical Imaging (SASHIMI)
Paper: https://lnkd.in/gnWtK37B
Code: https://lnkd.in/gjc2FBWT
Challenge 1: Transferring Knowledge from High-Quality to Low-Quality CT for Adult Glioma Diagnosis
Long Bai, Oct 06, Oral (Top-performing Team)
BraTS Challenge on Sub-Sahara-Africa Adult Glioma (BraTS-SSA)
Challenge 2: Ensembling Multi-scale Networks for Accurate Adult Glioma Diagnosis
Long Bai, Oct 06, Poster
BraTS Adult Glioma Post Treatment Challenge (BraTS-GLI)
This system facilitates targeted, minimally invasive neuroimaging with an axial resolution of about 2.4 ฮผm and a transverse resolution of around 4.5 ฮผm in the deep brain in vivo. The skullbot enables precise deployment of the neuroendoscope with a targeting accuracy of ยฑ1.5 mm transversely and ยฑ0.25 mm longitudinally, confirmed through optical phantom studies. The skullbot can be securely attached to the head, allowing for motion-insensitive stereotactic imaging within the brain.
We validated the system’s capabilities by demonstrating targeted imaging of a tumor in a brain phantom and conducting in vivo micro-resolution volumetric neuroimaging of fine structures within a mouse brain. This advanced device offers in situ disease evaluation at a micro- resolution level and serves as a promising intraoperative imaging tool, complementing existing4 clinical whole-brain imaging modalities such as MRI.
Our findings suggest that the neuroOCT system can significantly advance minimally invasive high-resolution targeted neuroendoscopy, thereby improving patient safety during neurosurgical procedures.
The paper will be available at https://lnkd.in/gFREQFbS
Stay tuned!
This is a collabrative work between CUHK ABI Lab (https://lnkd.in/gUuzQqDt) and REN Lab (http://www.labren.org/mm/).
Congrates to authors: Chao Xu+, Zhiwei Fang+, Huxin Gao, Tinghua Zhang, Tao Zhang, Peng Liu, Hongliang Ren*, and Wu ‘Scott’ YUAN*
We are thrilled to share that our latest research paper โEnhancing Anti-interference of Magnetic Tracking: A MagRobustNet-based Framework with Self-supervised Anomaly Detection and Measurements Recoveryโ has been accepted by IEEE Transactions on Industrial Informatics!
Magnetic tracking technology often suffers from diverse and unpredictable interferences in practical applications, such as hard-/soft-iron interferences and sensor saturation, leading to reduced localization accuracy or even tracking failure.
To address these issues, we propose a MagRobustNet-based framework with anomaly detection and measurement recovery. In the first step, disjoint mask sets are used in conjunction with MagRobustNet to detect anomalous measurements subject to disturbances. In the second step, the interfered regions are masked, and MagRobustNet is applied again to recover their expected measurements from neighboring normal data.
Our proposed method not only enhances the tracking systemโs anti-interference capability, but also indicates the interfered regions, offering a new potential diagnostic method for localizing ingested foreign bodies in clinical practice.
Check out our video demonstration at https://lnkd.in/g9TNnsA4
Stay tuned for the paper publication!
Congrats to all co-authors: Shijian Su, Huxin Gao, and Hongliang Ren from the Department of Electronic Engineering, The Chinese University of Hong Kong; Hai Lan and Houde Dai from Quanzhou Institute of Equipment Manufacturing, Haixi Institute, CAS.
News