A curated list of reviews/perspectives and technical papers from the intersection of surgery and data science.
This list is providing an overview of recent publications from surgical data science. Here we present a curated subset of relevant papers in the field. Articles are presented in chronological order. In the event that important papers are missing feel free to send a pull request.
Schmidgall, S., Kim, J. W., Kuntz, A., Ghazi, A. E., & Krieger, A. General-purpose foundation models for increased autonomy in robot-assisted surgery. arXiv (2024)
Yip, M., Salcudean, S., Goldberg, K., Althoefer, K., Menciassi, A., Opfermann, J. D., ... & Lee, I. C. (2023). Artificial intelligence meets medical robotics. Science, 381(6654), 141-146.
Maier-Hein, L., Eisenmann, M., Sarikaya, D., März, K., Collins, T., Malpani, A., ... & Speidel, S. (2022). Surgical data science–from concepts toward clinical translation. Medical image analysis, 76, 102306.
Zemmar, A., Lozano, A. M., & Nelson, B. J. (2020). The rise of robots in surgical environments during COVID-19. Nature Machine Intelligence, 2(10), 566-572.
Maier-Hein, L., Vedula, S. S., Speidel, S., Navab, N., Kikinis, R., Park, A., ... & Jannin, P. (2017). Surgical data science for next-generation interventions. Nature Biomedical Engineering, 1(9), 691-696.
Lam, K., Chen, J., Wang, Z., Iqbal, F. M., Darzi, A., Lo, B., ... & Kinross, J. M. (2022). Machine learning for technical skill assessment in surgery: a systematic review. NPJ digital medicine, 5(1), 24.
Zhang, Y., Weng, Y., & Lund, J. (2022). Applications of explainable artificial intelligence in diagnosis and surgery. Diagnostics, 12(2), 237.
Garrow, C. R., Kowalewski, K. F., Li, L., Wagner, M., Schmidt, M. W., Engelhardt, S., ... & Nickel, F. (2021). Machine learning for surgical phase recognition: a systematic review. Annals of surgery, 273(4), 684-693.
Bates, D. W., Levine, D., Syrowatka, A., Kuznetsova, M., Craig, K. J. T., Rui, A., ... & Rhee, K. (2021). The potential of artificial intelligence to improve patient safety: a scoping review. NPJ digital medicine, 4(1), 54.
Hashimoto, D. A., Rosman, G., Rus, D., & Meireles, O. R. (2018). Artificial intelligence in surgery: promises and perils. Annals of surgery, 268(1), 70.
Vedula, S. S., & Hager, G. D. (2017). Surgical data science: the new knowledge domain. Innovative surgical sciences, 2(3), 109-121.
Kiyasseh, D., Ma, R., Haque, T. F., Miles, B. J., Wagner, C., Donoho, D. A., ... & Hung, A. J. (2023). A vision transformer for decoding surgeon activity from surgical videos. Nature Biomedical Engineering, 1-17.
Scheikl, P. M., Gyenes, B., Younis, R., Haas, C., Neumann, G., Wagner, M., & Mathis-Ullrich, F. (2023). LapGym--An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery. arXiv preprint arXiv:2302.09606.
Schmidgall, S., Krieger, A., & Eshraghian, J. (2023). Surgical Gym: A high-performance GPU-based platform for reinforcement learning with surgical robots. arXiv preprint arXiv:2310.04676.
Scheikl, P. M., Tagliabue, E., Gyenes, B., Wagner, M., Dall'Alba, D., Fiorini, P., & Mathis-Ullrich, F. (2022). Sim-to-real transfer for visual reinforcement learning of deformable object manipulation for robot-assisted surgery. IEEE Robotics and Automation Letters, 8(2), 560-567.
Wang, Y., Long, Y., Fan, S. H., & Dou, Q. (2022, September). Neural rendering for stereo 3d reconstruction of deformable tissues in robotic surgery.** In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 431-441). Cham: Springer Nature Switzerland.
Tang, W., He, F., Liu, Y., & Duan, Y. (2022). MATR: Multimodal medical image fusion via multiscale adaptive transformer. IEEE Transactions on Image Processing, 31, 5134-5149.
Lu, J., Jayakumari, A., Richter, F., Li, Y., & Yip, M. C. (2021, May). Super deep: A surgical perception framework for robotic tissue manipulation using deep learning for feature extraction. In 2021 IEEE International Conference on Robotics and Automation (ICRA) (pp. 4783-4789). IEEE.
Chiu, Z. Y., Richter, F., Funk, E. K., Orosco, R. K., & Yip, M. C. (2021, May). Bimanual regrasping for suture needles using reinforcement learning for rapid motion planning. In 2021 IEEE International Conference on Robotics and Automation (ICRA) (pp. 7737-7743). IEEE.
Gao, X., Jin, Y., Long, Y., Dou, Q., & Heng, P. A. (2021). Trans-svnet: Accurate phase recognition from surgical videos via hybrid embedding aggregation transformer. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24 (pp. 593-603). Springer International Publishing.
Barnoy, Y., O'Brien, M., Wang, W., & Hager, G. (2021). Robotic surgery with lean reinforcement learning. arXiv preprint arXiv:2105.01006.
Czempiel, T., Paschali, M., Ostler, D., Kim, S. T., Busam, B., & Navab, N. (2021). Opera: Attention-regularized transformers for surgical phase recognition. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24 (pp. 604-614). Springer International Publishing.
Li, Y., Richter, F., Lu, J., Funk, E. K., Orosco, R. K., Zhu, J., & Yip, M. C. (2020). Super: A surgical perception framework for endoscopic tissue manipulation with surgical robotics. IEEE Robotics and Automation Letters, 5(2), 2294-2301.
Gao, X., Jin, Y., Dou, Q., & Heng, P. A. (2020, May). Automatic gesture recognition in robot-assisted surgery with reinforcement learning and tree search. In 2020 IEEE International Conference on Robotics and Automation (ICRA) (pp. 8440-8446). IEEE.
Vercauteren, T., Unberath, M., Padoy, N., & Navab, N. (2019). Cai4cai: the rise of contextual artificial intelligence in computer-assisted interventions. Proceedings of the IEEE, 108(1), 198-214.
Yu, F., Croso, G. S., Kim, T. S., Song, Z., Parker, F., Hager, G. D., ... & Sikder, S. (2019). Assessment of automated identification of phases in videos of cataract surgery using machine learning and deep learning techniques. JAMA network open, 2(4), e191860-e191860.
Richter, F., Orosco, R. K., & Yip, M. C. (2019). Open-sourced reinforcement learning environments for surgical robotics. arXiv preprint arXiv:1903.02090.
Liu, D., & Jiang, T. (2018). Deep reinforcement learning for surgical gesture segmentation and classification. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part IV 11 (pp. 247-255). Springer International Publishing.
Liu, X., Sinha, A., Unberath, M., Ishii, M., Hager, G. D., Taylor, R. H., & Reiter, A. (2018). Self-supervised Learning for Dense Depth Estimation in Monocular Endoscopy. arXiv preprint arXiv:1806.09521.
Bier, B., Unberath, M., Zaech, J. N., Fotouhi, J., Armand, M., Osgood, G., ... & Maier, A. (2018, September). X-ray-transform invariant anatomical landmark detection for pelvic trauma surgery. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 55-63). Cham: Springer International Publishing.
Ye, M., Johns, E., Handa, A., Zhang, L., Pratt, P., & Yang, G. Z. (2017). Self-supervised siamese learning on stereo image pairs for depth estimation in robotic surgery. arXiv preprint arXiv:1705.08260.
Garcia-Peraza-Herrera, L. C., Li, W., Fidon, L., Gruijthuijsen, C., Devreker, A., Attilakos, G., ... & Ourselin, S. (2017, September). Toolnet: holistically-nested real-time segmentation of robotic surgical tools. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 5717-5722). IEEE.
Ahmidi, N., Tao, L., Sefati, S., Gao, Y., Lea, C., Haro, B. B., ... & Hager, G. D. (2017). A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery. IEEE Transactions on Biomedical Engineering, 64(9), 2025-2041.
Thananjeyan, B., Garg, A., Krishnan, S., Chen, C., Miller, L., & Goldberg, K. (2017, May). Multilateral surgical pattern cutting in 2D orthotropic gauze with deep reinforcement learning policies for tensioning. In 2017 IEEE International Conference on Robotics and Automation (ICRA) (pp. 2371-2378). IEEE.
Ahmidi, N., Tao, L., Sefati, S., Gao, Y., Lea, C., Haro, B. B., ... & Hager, G. D. (2017). A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery. IEEE Transactions on Biomedical Engineering, 64(9), 2025-2041.
DiPietro, R., Lea, C., Malpani, A., Ahmidi, N., Vedula, S. S., Lee, G. I., ... & Hager, G. D. (2016). Recognizing surgical activities with recurrent neural networks. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part I 19 (pp. 551-558). Springer International Publishing.
Zappella, L., Béjar, B., Hager, G., & Vidal, R. (2013). Surgical gesture classification from video and kinematic data. Medical image analysis, 17(7), 732-745.
Padoy, N., & Hager, G. D. (2011, May). Human-machine collaborative surgery using learned models. In 2011 IEEE International Conference on Robotics and Automation (pp. 5285-5292). IEEE.
Reiley, C. E., Plaku, E., & Hager, G. D. (2010, August). Motion generation of robotic surgical tasks: Learning from expert demonstrations. In 2010 Annual international conference of the IEEE engineering in medicine and biology (pp. 967-970). IEEE.
Varadarajan, B., Reiley, C., Lin, H., Khudanpur, S., & Hager, G. (2009). Data-derived models for segmentation with application to surgical assessment and training. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2009: 12th International Conference, London, UK, September 20-24, 2009, Proceedings, Part I 12 (pp. 426-434). Springer Berlin Heidelberg.
Lin, H. C., Shafran, I., Yuh, D., & Hager, G. D. (2006). Towards automatic skill evaluation: Detection and segmentation of robot-assisted surgical motions. Computer aided surgery, 11(5), 220-230.
Adida, S., Legarreta, A. D., Hudson, J. S., McCarthy, D., Andrews, E., Shanahan, R., ... & Gerszten, P. C. (2022). Machine Learning in Spine Surgery: A Narrative Review. Neurosurgery, 10-1227.
Li, B., Feridooni, T., Cuen-Ojeda, C., Kishibe, T., de Mestral, C., Mamdani, M., & Al-Omran, M. (2022). Machine learning in vascular surgery: a systematic review and critical appraisal. NPJ Digital Medicine, 5(1), 7.
Buchlak, Q. D., Esmaili, N., Leveque, J. C., Farrokhi, F., Bennett, C., Piccardi, M., & Sethi, R. K. (2020). Machine learning applications to clinical decision support in neurosurgery: an artificial intelligence augmented systematic review. Neurosurgical review, 43, 1235-1253.
** Tempalate from Awesome NeuroAI Papers