Developing a multi-functional magnetic-driven soft robot to carry out various medical missions remains challenging. In this work, we design a tripedal soft magnetic robot with three radial magnetized cylindric permanent magnets embedded in three soles. The motion modalities for movement include butterfly crawling (along the x and y axis), scorpion crawling, and rolling. At different frequencies, the robot exhibits different behaviors in terms of speed and trajectory under different moving modalities. The maximum velocity of the butterfly crawling and scorpion crawling motion at the frequency of 1 Hz is measured to be 5.30 mm/s and 9.06 mm/s.
For details, please check the paper at https://lnkd.in/gfbRzNYu
Our work proposes the Curriculum-based Augmented Fourier Domain Adaptation (Curri-AFDA) and proves to achieve superior adaptation, generalization, and robustness performance for medical image segmentation.
**Motivation**
Medical image segmentation is key to improving computer-assisted diagnosis and intervention autonomy. However, due to domain gaps between different medical sites, deep learning-based segmentation models frequently encounter performance degradation when deployed in a novel domain. Moreover, model robustness is also highly expected to mitigate the effects of data corruption.
**Methodology**
Considering all these demanding yet practical needs to automate medical applications and benefit healthcare, we propose the Curriculum-based Fourier Domain Adaptation (Curri-AFDA) for medical image segmentation. Specifically, we design a novel curriculum strategy to progressively transfer amplitude information in the Fourier space from the target domain to the source domain to mitigate domain gaps and incorporate the chained augmentation mixing to further improve the generalization and robustness ability.
**Performance**
Extensive experiments on two segmentation tasks with cross-domain datasets show the consistent superiority of our method regarding adaptation and generalization on multiple testing domains and robustness against synthetic corrupted data. Besides, our approach is independent of image modalities because its efficacy does not rely on modality-specific characteristics. In addition, we demonstrate the benefit of our method for image classification besides segmentation in the ablation study. Therefore, our method can potentially be applied in various medical applications and yield improved performance.
This paper is an extended version of our #ICRA2023 Surgical-VQLA. Our method can serve as an effective and reliable tool to assist in surgical education and clinical decision-making by providing more insightful analyses of surgical scenes.
โจ Key Contributions in the journal version:
– A dual calibration module is proposed to align and normalize multimodal representations.
– A contrastive training strategy with adversarial examples is employed to enhance robustness.
– Various optimization function is widely explored.
– The EndoVis-18-VQLA & EndoVis-17-VQLA datasets are further extended.
– Our proposed solution presents superior performance and robustness against real-world image corruption.
Conference Version (ICRA 2023): https://lnkd.in/gHscT3eN
Journal Version (Information Fusion): https://lnkd.in/gQNWwHmt
In this work, through incorporating a set of learnable parameters to prompt the learning targets, the diffusion model can effectively address the unified illumination correction challenge in capsule endoscopy. We also propose a new capsule endoscopy dataset including underexposed and overexposed images, as well as the ground truth.
Thanks to all of our collaborators from multiple institutions: Long Bai, Qiaozhi Tan, Zhicheng He, Sishen YUAN, Prof. Hongliang Ren from CUHK & SZRI, Tong Chen from USYD, Wan Jun Nah from Universiti Malaya, Yanheng Li from CityU HK, Prof. Zhen CHEN, Prof. Jinlin Wu, Prof. Hongbin Liu from CAIR HK, Dr. Mobarakol Islam from WEISS, UCL, and Dr. Zhen Li from Qilu Hospital of SDU.
We’re excited to share our latest research on the Segment Anything Model (SAM) 2. This empirical evaluation uncovers SAM 2’s robustness and generalization capabilities in surgical image/video segmentation, a critical component for enhancing precision and safety in the operating room.
๐ฌ **Key Findings**:
– In general, SAM 2 outperforms its predecessor in instrument segmentation, showing a much improved zero-shot generalization capability to the surgical domain.
– Utilizing bounding box prompts, SAM 2 achieves remarkable results, setting a new benchmark in the surgical image segmentation.
– With a single point as the prompt on the first frame, SAM 2 demonstrates substantial improvements on video segmentation over SAM, which requires point prompts on every frames. This suggests great potential in addressing video-based surgical tasks.
– Resilience Under Common Corruptions: SAM 2 shows impressive robustness against real-world image corruption, maintaining performance under various challenges such as compression, noise, and blur.
๐ง **Practical Implications**:
– With faster inference speeds, SAM 2 is poised to provide quick, accurate segmentation, making it a valuable asset in the clinical setting.
๐ **Learn More**:
For those interested in the technical depth, our paper is available on [arXiv](https://lnkd.in/gHfdrvj3).
We’re eager to engage with the community and explore how SAM 2 can revolutionize surgical applications.
In our recent work, “Privacy-Preserving Synthetic Continual Semantic Segmentation for Robotic Surgery”, which was published in IEEE Transactions on Medical Imaging, we propose a state-of-the-art framework for continual semantic segmentation in robotic surgery. This breakthrough addresses catastrophic forgetting in DNNs, enhancing surgical precision without compromising patient privacy.
๐ Privacy-First Synthetic Data: We’ve crafted a solution that blends open-source instrument data with synthesized backgrounds, ensuring real patient data remains confidential.
๐ก Innovative Features:
– Class-Aware Temperature Normalization (CAT) to prevent forgetting of previously learned tasks.
– Multi-Scale Shifted-Feature Distillation (SD) to preserve spatial relationships for robust feature learning.
Check the paper at https://lnkd.in/eTy8KAC5
Code is also available at https://lnkd.in/eMzNs2Be
