Thrilled to share our latest work on enabling robust sparse-to-dense reconstruction for endoscopic surgical robots โ bridging the gap between ๐ฌ๐ฉ๐๐ซ๐ฌ๐ ๐ฌ๐๐ง๐ฌ๐จ๐ซ ๐๐๐ญ๐ ๐๐ง๐ ๐ก๐ข๐ ๐ก-๐ช๐ฎ๐๐ฅ๐ข๐ญ๐ฒ ๐๐ ๐ฆ๐๐ฉ๐ฉ๐ข๐ง๐ using a novel ๐๐ข๐๐๐ฎ๐ฌ๐ข๐จ๐ง-๐๐๐ฌ๐๐ framework.
Fine-tuning foundational models often fails due to a lack of dense ground truth, and self-supervised methods struggle with scale ambiguity, sparse depth sensors offer a reliable geometric prior.
This motivated us to develop EndoDDC, a method that robustly generates dense depth maps by fusing RGB images with sparse depth inputs.
๐ง โจ ๐๐ก๐๐ญ ๐ฐ๐ ๐๐๐ฏ๐๐ฅ๐จ๐ฉ๐๐:
A diffusion-driven depth completion architecture that:
๐น Integrates sparse depth and RGB inputs to overcome the limitations of pure visual estimation.
๐น Utilizes a Multi-scale Feature Extraction and Depth Gradient Fusion module to capture fine-grained surface orientation and local structure.
๐น Optimizes depth maps iteratively using a conditional diffusion model, refining geometry even in regions with weak textures or reflections.
๐ฏ ๐๐๐ฒ ๐๐๐ฌ๐ฎ๐ฅ๐ญ๐ฌ:
โ 25.55% and 9.03% improvement in accuracy on the StereoMIS and C3VD dataset compared to SOTA surgical estimators like EndoDAC.
โ 7.35% and 5.28% reduction in RMSE on StereoMIS and C3VD compared to the best depth completion baseline (OGNI-DC).
โ Outperformed foundational models (DepthAnything-v2) and standard depth completion (Marigold-DC) methods in both accuracy and robustness.
๐ก ๐๐ก๐ฒ ๐ข๐ญ ๐ฆ๐๐ญ๐ญ๐๐ซ๐ฌ:
This work demonstrates that diffusion models can effectively solve the “sparse-to-dense” challenge in medical imaging. By providing accurate depth completion despite complex lighting and texture conditions, EndoDDC has the potential to significantly enhance autonomous navigation, procedural safety, and spatial awareness in minimally invasive surgery.
๐ #DepthCompletion #DiffusionModel #EndoscopicSurgery #SurgicalNavigation #ICRA #CUHKEngineering #CUHK
