John J. Han

John J. Han

PhD Student in Computer Science at Vanderbilt University

Publications

MeshBrush: Painting the Anatomical Mesh with Neural Stylization for Endoscopy

Published in Medical Image Computing and Computer Assisted Intervention (MICCAI) 2024, 2024

Style transfer is a promising approach to close the sim-to-real gap in medical endoscopy. Rendering realistic endoscopic videos by traversing pre-operative scans (such as MRI or CT) can generate realistic simulations as well as ground truth camera poses and depth maps. Although image-to-image (I2I) translation models such as CycleGAN perform well, they are unsuitable for video-to-video synthesis due to the lack of temporal consistency, resulting in artifacts between frames. We propose MeshBrush, a neural mesh stylization method to synthesize temporally consistent videos with differentiable rendering. MeshBrush uses the underlying geometry of patient imaging data while leveraging existing I2I methods. With learned per-vertex textures, the stylized mesh guarantees consistency while producing high-fidelity outputs. We demonstrate that mesh stylization is a promising approach for creating realistic simulations for downstream tasks such as training and preoperative planning.

Download here

Improving the Temporal Accuracy of Eye Gaze Tracking for the da Vinci Surgical System through Automatic Detection of Decalibration Events and Recalibration

Published in Journal of Medical Robotics Research, 2024

Robust and accurate eye gaze tracking can advance medical telerobotics by providing complementary data for surgical training, interactive instrument control, and augmented human�robot interactions. However, current gaze tracking solutions for systems such as the da Vinci Surgical System (dVSS) are limited to complex hardware installations. Additionally, existing methods do not account for operator head movement inside the surgeon console, invalidating the original calibration. This work provides an initial solution to these challenges that can seamlessly integrate into console devices beyond the dVSS. Our approach relies on simple and unobtrusive wearable eye tracking glasses and provides calibration routines that can contend with operator-head movements. An external camera measures movement of the glasses through trackers mounted on the glasses to detect invalidation of the prior calibration from head movement and slippage.

Recommended citation: B�ter, Regine, et al. "Improving the Temporal Accuracy of Eye Gaze Tracking for the da Vinci Surgical System through Automatic Detection of Decalibration Events and Recalibration."�Journal of Medical Robotics Research�(2024): 2440001. https://www.worldscientific.com/doi/abs/10.1142/S2424905X24400014

Depth Anything in Medical Images: A Comparative Study

Published in arXiv Preprint, 2024

Monocular depth estimation (MDE) is a critical component of many medical tracking and mapping algorithms, particularly from endoscopic or laparoscopic video. However, because ground truth depth maps cannot be acquired from real patient data, supervised learning is not a viable approach to predict depth maps for medical scenes. Although self-supervised learning for MDE has recently gained attention, the outputs are difficult to evaluate reliably and each MDE's generalizability to other patients and anatomies is limited. This work evaluates the zero-shot performance of the newly released Depth Anything Model on medical endoscopic and laparoscopic scenes. We compare the accuracy and inference speeds of Depth Anything with other MDE models trained on general scenes as well as in-domain models trained on endoscopic data.

Recommended citation: Han, John J., et al. "Depth Anything in Medical Images: A Comparative Study."�arXiv preprint arXiv:2401.16600�(2024). https://arxiv.org/abs/2401.16600