Medical Multimedia Information Systems
The 1-page abstract “Pixel-Based Iris and Pupil Segmentation in Cataract Surgery Videos Using Mask R-CNN” was accepted at the workshop “Deep Learning for Biomedical Image Reconstruction” of the International Symposium on Biomedical Imaging that will take place in Iowa-City, Iowa, USA, 3-7 April.
Authors:
Natalia Sokolova, Mario Taschwer, Klaus Schoeffmann
Acknowledment:
This work was funded by the FWF Austrian Science Fund under grant P 31486-N31
At the 26th International Conference on MultiMedia Modeling (MMM 2020) in Daejeon, Korea, researchers from ITEC have successfully presented several scientific contributions to the multimedia community. First, Natalia Sokolova presented her first paper on “Evaluating the Generalization Performance of Instrument Classification in Cataract Surgery Videos”. Next, Sabrina Kletz presented her work on “Instrument Recognition in Laparoscopy for Technical Skill Assessment”. Finally, Andreas Leibetseder talked about “GLENDA: Gynecologic Laparoscopy Endometriosis Dataset”.
VBS 2020 in Daejeon (South Korea) was an amazing event with a lot of fun! Eleven teams, each consisting of two users (coming from 11 different countries) competed against each other in both a private session for about 5 hours and a public session for almost 3 hours. ITEC did also participate with two teams. In total all teams had to solve 22 challenging video retrieval tasks, issued on a shared dataset consisting of 1000 hours of content (V3C1)! Many thanks go to the VBS teams but also to the VBS organizers as well as the local organizers, who did a great job and made VBS2020 a wonderful and entertaining event!
Sabrina Kletz presented the paper “Learning the Representation of Instrument Images in Laparoscopy Video” at the MIAR Workshop @ MICCAI 2019 in Shenzhen, China.
Authors: Sabrina Kletz, Klaus Schoeffmann, Heinrich Husslein
Abstract: Automatic recognition of instruments in laparoscopy videos poses many challenges that need to be addressed like identifying multiple instruments appearing in various representations and in different lighting conditions which in turn may be occluded by other instruments, tissue, blood or smoke. Considering these challenges it may be beneficial for recognition approaches that instrument frames are first detected in a sequence of video frames for further investigating only these frames. This pre-recognition step is also relevant for many other classification tasks in laparoscopy videos such as action recognition or adverse event analysis. In this work, we address the task of binary classification to recognize video frames as either instrument or non-instrument images. We examine convolutional neural network models to learn the representation of instrument frames in videos and take a closer look at learned activation patterns. For this task, GoogLeNet together with batch normalization is trained and validated using a publicly available dataset for instrument count classifications. We compare transfer learning with learning from scratch and evaluate on datasets from cholecystectomy and gynecology. The evaluation shows that fine-tuning a pre-trained model on the instrument and non-instrument images is much faster and more stable in learning than training a model from scratch.
Conference: 2019 International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), October 13–17, 2018, Shenzhen, China
Track: Medical Imaging and Augmented Reality (MIAR) Workshop @MICCAI
Our paper has been accepted for publication at the MMM 2020 Conference on Multimedia Modeling. The work was conducted in the context of the ongoing OVID project.
Authors: Natalia Sokolova, Klaus Schoeffmann, Mario Taschwer (AAU Klagenfurt); Doris
Putzgruber-Adamitsch, Yosuf El-Shabrawi (Klinikum Klagenfurt)
Abstract:
In the field of ophthalmic surgery, many clinicians nowadays record their microscopic procedures with a video camera and use the recorded footage for later purpose, such as forensics, teaching, or training. However, in order to efficiently use the video material after surgery, the video content needs to be analyzed automatically. Important semantic content to be analyzed and indexed in these short videos are operation instruments, since they provide an indication of the corresponding operation phase and surgical action. Related work has already shown that it is possible to accurately detect instruments in cataract surgery videos. However, their underlying dataset (from the CATARACTS challenge) has very good visual quality, which is not reflecting the typical quality of videos acquired in general hospitals. In this paper, we therefore analyze the generalization performance of deep learning models for instrument recognition in terms of dataset change. More precisely, we trained such models as ResNet-50, Inception v3 and NASNet Mobile using a dataset of high visual quality (CATARACT) and test it on another dataset with low visual quality (Cataract-101), and vice versa. Our results show that the generalizability is rather low in general, but clearly worse for the model trained on the high-quality dataset. Another important observation is the fact that the trained models are able to detect similar instruments in the other dataset even if their appearance is different.
The paper “GLENDA: Gynecologic Laparoscopy Endometriosis Dataset” has been accepted for publication at the Multimedia Datasets for Repeatable Experimentation (MDRE) special session, co-located at the 26th International Conference on Multimedia Modeling, MMM 2020 to be held at Daejon, Korea (January 5-8, 2020).
Authors: Andreas Leibetseder, Sabrina Kletz, Klaus Schoeffmann (Alpen-Adria Universität Klagenfurt), Simon Keckstein (Ludwig-Maximilians-University Munich), Jörg Keckstein (Ulm University)
Abstract: Gynecologic laparoscopy as a type of minimally invasive surgery (MIS) is performed via a live feed of a patient’s abdomen surveying the insertion and handling of various instruments for conducting treatment. Adopting this kind of surgical intervention not only facilitates a great variety of treatments, the possibility of recording said video streams is as well essential for numerous post-surgical activities, such as treatment planning, case documentation and education. Nonetheless, the process of manually analyzing surgical recordings, as it is carried out in current practice, usually proves tediously time-consuming. In order to improve upon this situation, more sophisticated computer vision as well as machine learning approaches are actively developed. Since most of such approaches heavily rely on sample data, which especially in the medical field is only sparsely available, with this work we publish the Gynecologic Laparoscopy ENdometriosis DAtaset (GLENDA) – an image dataset containing region-based annotations of a common medical condition named endometriosis, i.e. the dislocation of uterine-like tissue. The dataset is the first of its kind and it has been created in collaboration with leading medical experts in the field.
Keywords: lesion detection, endometriosis localization, medical dataset, region-based annotations, gynecologic laparoscopy
Acknowledgement: This work was funded by the FWF Austrian Science Fund under grant P 32010-N38.
Within the scope of the AAU’s young Scientists Mentoring Programme, Andreas Leibetseder is visiting his mentor Oge Marques, Professor at the Department of Computer and Electrical Engineering and Computer Science and Florida Atlantic University (FAU) in Boca Raton, Florida. During his stay over the course of April, he intends to advance in his PhD studies by focusing on the sub-topic of region-based Endometriosis classification in laparoscopic media. He intends to approach this problem by adapting and applying deep learning technologies, profiting from the knowledge and insights of his mentor as well as other students of the local mlab research group (http://mlab.fau.edu/www/). Currently, several work packages have been defined, which include investigating lesion detection approaches on radiological image datasets for application in the endoscopic domain.
Last week, Klaus Schoeffmann co-organized the 8th Video Browser Showdown (VBS) at MMM2019 in Thessaloniki, and it was a great success. For the first time they used the V3C1 dataset (Part 1 of the Vimeo Creative Commons Collection), which consists of 7475 video files that amount for about 1000 hours of content. The six participating teams (including an ITEC team with Andreas Leibetseder) could solve all visual and textual Known-Item Search (KIS) tasks, as well as all Ad-Hoc Video Search (AVS) tasks within a short amount of time! The teams have clearly demonstrated that their sophisticated video retrieval systems are very powerful and allow fast and effective content-based search in videos. They look forward to the next VBS in January 2020 in Daejeon, Korea at MMM2020! More information here: www.VideoBrowserShowdown.org
On Reducing Effort in Evaluating Laparoscopic Skills
Abstract: Training and evaluation of laparoscopic skills have become an important aspect of young surgeons’ education. The evaluation process is currently performed manually by experienced surgeons through reviewing video recordings of laparoscopic procedures for detecting technical errors using conventional video players and specific pen and paper rating schemes. The problem is, that the manual review process is time-consuming and exhausting, but nevertheless necessary to support young surgeons in their educational training. Motivated by the need to reduce the effort in evaluating laparoscopic skills, this PhD project aims at investigating state-of-the-art content analysis approaches for finding error-prone video sections in surgery videos. In this proposal, the focus specifically lies on performance assessment in gynecologic laparoscopy using the Generic Error Rating Tool (GERT).
Conference: 2018 ACM Multimedia Conference, October 22–26, 2018, Seoul, Republic of Korea
Track: Doctoral Symposium
Mathias Lux was invited to give a talk at Simula Metropolitan, a joint research center of SIMULA research labs and Oslo Metropolitan University. Besides the talk he took the opportunity to work for two days with the people at SIMULA and talk about future and ongoing projects. Read more