Challenges in surgical video annotation

References

• Hashimoto DA, Rosman G, Rus D, et al. Artificial Intelligence in surgery: promises and perils. Ann Surg. 2018;268(1):70–76.
   PubMed Web of Science ®Google Scholar
• Deal SB, Stefanidis D, Brunt LM, et al. Development of a multimedia tutorial to educate how to assess the critical view of safety in laparoscopic cholecystectomy using expert review and crowd-sourcing. Am J Surg. 2017;213:988–990.
   PubMed Web of Science ®Google Scholar
• Maier-Hein L, Mersmann S, Kondermann D, et al. Can masses of non-experts train highly accurate image classifiers? In: Golland P, Hata N, Barillot C, Hornegger J, Howe R, editors. Medical image computing and computer-assisted intervention – MICCAI 2014. Cham: Springer International Publishing; 2014. p. 438–445.
   Google Scholar
• Deal SB, Stefanidis D, Telem D, et al. Evaluation of crowd-sourced assessment of the critical view of safety in laparoscopic cholecystectomy. Surg Endosc. 2017;31(12):5094–5100.
   PubMed Web of Science ®Google Scholar
• Malpani A, Vedula SS, Chen CCG, et al. A study of crowdsourced segment-level surgical skill assessment using pairwise rankings. Int J Comput Assist Radiol Surg. 2015;10(9):1435–1447.
   PubMed Web of Science ®Google Scholar
• Holst D, Kowalewski TM, White LW, et al. Crowd-sourced assessment of technical skills: differentiating animate surgical skill through the wisdom of crowds. J Endourol. 2015;29(10):1183–1188.
   PubMed Web of Science ®Google Scholar
• Chen C, White L, Kowalewski T, et al. Crowd-sourced assessment of technical skills: a novel method to evaluate surgical performance. J Surg Res. 2014;187(1):65–71.
   PubMed Web of Science ®Google Scholar
• Pugh CM, Hashimoto DA, Korndorffer JRJ. The what? How? And Who? Of video based assessment. Am J Surg. 2021;221(1):13–18.
   PubMed Web of Science ®Google Scholar
• Twinanda AP. Vision-based approaches for surgical activity recognition using laparoscopic and RBGD videos. [updated online 2017 Jan 27; cited 2020 Sept 10]. https://www.theses.fr/2017STRAD005.
   Google Scholar
• Twinanda AP, Shehata S, Mutter D, et al. Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging. 2016;36(1):86–97.
   PubMed Web of Science ®Google Scholar
• Hashimoto DA, Rosman R, Witkowski E, et al. Computer vision analysis of intraoperative video: automated recognition of operative steps in laparoscopic sleeve gastrectomy. Ann Surg. 2019;270(3):414–421.
   PubMed Web of Science ®Google Scholar
• Ward TM, Hashimoto DA, Ban Y, et al. Automated operative phase identification in peroral endoscopic myotomy. Surg Endosc. 2020. DOI:https://doi.org/10.1007/s00464-020-07833-9
   Google Scholar
• Gwet KL. Testing the difference of correlated agreement coefficients for statistical significance. Educ Psychol Meas. 2016;76(4):609–637.
   PubMed Web of Science ®Google Scholar
• Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol. 1990;43(6):543–549.
   PubMed Web of Science ®Google Scholar
• Ward TM, Hashimoto D, Ban Y, et al. Training with pooled annotations from multiple surgeons has no effect on a deep learning artificial intelligence model’s performance. J Am Coll Surg. 2020;231(4):e203.
   PubMed Web of Science ®Google Scholar
• Madani A, Namazi B, Altieri MS, et al. Artificial intelligence for intraoperative guidance: using semantic segmentation to identify surgical anatomy during laparoscopic cholecystectomy. Ann Surg. 2020. DOI:https://doi.org/10.1097/SLA.0000000000004594
   Google Scholar
• Gibaud B, Forestier G, Feldmann C, et al. Toward a standard ontology of surgical process models. Int J Comput Assist Radiol Surg. 2018;13(9):1397–1408.
   PubMed Web of Science ®Google Scholar
• Ban Y, Rosman G, Ward T, et al. Aggregating long-term context for learning laparoscopic and robot-assisted surgical workflows. ArXiv200900681 Cs. [updated online 2020 Dec 3; 2021 cited 28]. http://arxiv.org/abs/2009.00681.
   Google Scholar
• Nwoye CI, Gonzalez C, Yu T, et al. Recognition of instrument-tissue interactions in endoscopic videos via action triplets. In: Medical image computing and computer-assisted intervention – MICCAI. Heidelberg (Germany): Springer International Publishing; 2020.
   Google Scholar
• Lin HC, Shafran I, Yuh D, et al. Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions. Comput Aided Surg. 2006;11(5):220–230.
   PubMedGoogle Scholar
• Jung JJ, Jüni P, Gee DW, et al. Development and evaluation of a novel instrument to measure severity of intraoperative events using video data. Ann Surg. 2020;272(2):220–226.
   PubMed Web of Science ®Google Scholar
• Vezhnevets A, Buhmann JM, Ferrari V. Active learning for semantic segmentation with expected change. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition. 2012. p. 3162–3169.
   Google Scholar
• Kim T, Lee K, Ham S, et al. Active learning for accuracy enhancement of semantic segmentation with CNN-corrected label curations: Evaluation on kidney segmentation in abdominal CT. Sci Rep. 2020;10(1):366.
   PubMed Web of Science ®Google Scholar
• Rocha C. d C, Padoy N, Rosa B. Self-supervised surgical tool segmentation using kinematic information. In: 2019 International Conference on Robotics and Automation (ICRA). 2019. p. 8720–8726.
   Google Scholar
• Kaafarani HM, Mavros MN, Hwabejire J, et al. Derivation and validation of a novel severity classification for intraoperative adverse events. J Am Coll Surg. 2014;218(6):1120–1128.
   PubMed Web of Science ®Google Scholar
• Bohnen JD, Mavros MN, Ramly EP, et al. Intraoperative adverse events in abdominal surgery: what happens in the operating room does not stay in the operating room. Ann Surg. 2017;265(6):1119–1125.
   PubMed Web of Science ®Google Scholar
• Rice DC, Memon MA, Jamison RL, et al. Long-term consequences of intraoperative spillage of bile and gallstones during laparoscopic cholecystectomy. J Gastrointest Surg. 1997;1(1):85–91.
   PubMedGoogle Scholar
• Martin J, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84(2):273–278.
   PubMed Web of Science ®Google Scholar
• Hashimoto DA, Sirimanna P, Gomez ED, et al. Deliberate practice enhances quality of laparoscopic surgical performance in a randomized controlled trial: from arrested development to expert performance. Surg Endosc. 2015;29(11):3154–3162.
   PubMed Web of Science ®Google Scholar
• Levin M, McKechnie T, Khalid S, et al. Automated methods of technical skill assessment in surgery: a systematic review. J Surg Educ. 2019;76(6):1629–1639.
   PubMed Web of Science ®Google Scholar
• Feldman LS, Pryor AD, Gardner AK, et al. SAGES Video-Based Assessment (VBA) program: a vision for life-long learning for surgeons. Surg Endosc. 2020;34:3285–3288.
   PubMed Web of Science ®Google Scholar
• Ritter EM, Gardner AK, Dunkin BJ, et al. Video-based assessment for laparoscopic fundoplication: initial development of a robust tool for operative performance assessment. Surg Endosc. 2020;34(7):3176–3178.
   PubMed Web of Science ®Google Scholar
• Scott DJ, Rege RV, Bergen PC, et al. Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation. J Laparoendosc Adv Surg Tech. 2000;10(4):183–190.
   PubMed Web of Science ®Google Scholar
• Birkmeyer JD, Finks JF, O'Reilly A, et al. Surgical skill and complication rates after bariatric surgery. N Engl J Med. 2013;369(15):1434–1442.
   PubMed Web of Science ®Google Scholar
• Curtis NJ, Foster JD, Miskovic D, et al. Association of surgical skill assessment with clinical outcomes in cancer surgery. JAMA Surg. 2020;155(7):590–598.
   PubMed Web of Science ®Google Scholar
• Soomro N, Hashimoto DA, Porteous A, et al. Systematic review of learning curves in robot-assisted surgery. BJS Open. 2020;4(1):27–44.
   PubMed Web of Science ®Google Scholar
• Curtis NJ, Stevenson ARL, Francis NK. Assessment of surgical skill and performance variability-reply. JAMA Surg. 2020;155(12):1175.
   PubMed Web of Science ®Google Scholar
• Bianco S, Ciocca G, Napoletano P, et al. An interactive tool for manual, semi-automatic and automatic video annotation. Comput Vis Image Underst. 2015;131:88–99.
   Web of Science ®Google Scholar
• Maier-Hein L, Vedula SS, Speidel S, et al. Surgical data science for next-generation interventions. Nat Biomed Eng. 2017;1(9):691–696.
   PubMed Web of Science ®Google Scholar
• Maier-Hein L, Eisenmann M, Sarikaya D, et al. Surgical data science–from concepts to clinical translation. ArXiv Prepr ArXiv201102284. Published online 2020.
   Google Scholar
• Arbelaez P, Maire M, Fowlkes C, et al. Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell. 2010;33(5):898–916.
   Web of Science ®Google Scholar
• Madani A, Watanabe Y, Bilgic E, et al. Measuring intra-operative decision-making during laparoscopic cholecystectomy: validity evidence for a novel interactive Web-based assessment tool. Surg Endosc. 2017;31(3):1203–1212.
   PubMed Web of Science ®Google Scholar
• Madani A, Grover K, Watanabe Y. Measuring and teaching intraoperative decision-making using the visual concordance test: deliberate practice of advanced cognitive skills. JAMA Surg. 2020;155(1):78–79.
   PubMed Web of Science ®Google Scholar
• Krause J, Perer A, Bertini E. A user study on the effect of aggregating explanations for interpreting machine learning models. In: ACM Workshop on Interactive Data Exploration and Analytics. New York (NY): ACM; 2018.
   Google Scholar