Medical Multimedia Information Systems

Title: Action Recognition in Video Recordings from Gynecologic Laparoscopy

Authors: Sahar Nasirihaghighi, Negin Ghamsarian, Daniela Stefanics, Klaus Schoeffmann and Heinrich Husslein

IEEE 36th International Symposium on Computer-Based Medical Systems 2023

Abstract: Action recognition is a prerequisite for many applications in laparoscopic video analysis including but not limited to surgical training, operation room planning, follow-up surgery preparation, post-operative surgical assessment, and surgical outcome estimation. However, automatic action recognition in laparoscopic surgeries involves numerous challenges such as (I) cross-action and intra-action duration variation, (II) relevant content distortion due to smoke, blood accumulation, fast camera motions, organ movements, object occlusion, and (III) surgical scene variations due to different illuminations and viewpoints. Besides, action annotations in laparoscopy surgeries are limited and expensive due to requiring expert knowledge. In this study, we design and evaluate a CNN-RNN architecture as well as a customized training-inference framework to deal with the mentioned challenges in laparoscopic surgery action recognition. Using stacked recurrent layers, our proposed network takes advantage of inter-frame dependencies to negate the negative effect of content distortion and variation in action recognition. Furthermore, our proposed frame sampling strategy effectively manages the duration variations in surgical actions to enable action recognition with high temporal resolution. Our extensive experiments confirm the superiority of our proposed method in action recognition compared to static CNNs.

Mathias Lux

Creating games for 48 hours at 1770 meters of altitude. The 2nd Hüttenjam took place April 13-16, 2023, on the Turracher Höhe. After settling on a topic for the jam on Thursday – extreme conditions – 40 participants worked on 8 games to be presented on Saturday evening. The jam took place in a chilled atmosphere and allowed for networking, winter sports, a sauna, and a lot of creative space. A video summary of the event is available at You can also play the games developed there at
The Hüttenjam is a joint event of Game Dev Graz and the University of Klagenfurt / ITEC:


Christina Obmann, one of our first Game Studies and Engineering students, has been recognized as outstanding in the Carinthia region by the local newspaper Kleine Zeitung. Besides her interest and work in games, she’s teaching at the university, learning Chinese, and was awarded a scholarship from Huawei.



For the quality and timeliness of his reviews, Klaus Schöffmann has been awarded with the Outstanding Reviewer Award at the ACM International Conference on Multimedia Retrieval (ICMR) 2022, which was held at Newark, NJ, USA in June 2022.


On Friday and Saturday (July 1 and July 2, 2022), Sebastian Uitz presented his game “A Webbing Journey” with his partner Manuel Santner at the Level Up event in the Messezentrum Salzburg. This was the biggest event they presented their game at, with a 300m² area for indie games and over 20.000m² in total. Over 6.500 people visited the event, and 2 PCs were provided for their game. In the end, 6 PCs were running “A Webbing Journey” due to some game developers not making it or leaving the event early. All the PCs were in constant use, and all players had so much fun playing the game, especially kids. This event and the last two provided so much feedback in the form of player tests, which have helped progress the game. The next goal is to implement all the feedback and releasing the new demo version on Steam.

On Saturday (June 18, 2022), Sebastian Uitz presented his game “A Webbing Journey” with his partner Manuel Santner at the A1 Austria eSports Festival in the Austria Center Vienna. The game didn’t fit into the Esports-themed event, but the 2 PCs prepared to play the game were constantly used by players of all ages. Especially the target audience (children 8-12) had so much fun, and the parents had to wait until the kids were done playing. The event was a great success as it was the biggest play session yet and resulted in tons of feedback that will be implemented and bugs that will be fixed until the next event.

IEEE 35th International Symposium on Computer Based Medical Systems (CBMS), July 21-23, Shenzen, China

Authors: Natalia Mathá, Klaus Schoeffmann, Stephanie Sarny, Doris Putzgruber-Adamitsch and Yosuf El-Shabrawi

Abstract: In recent years, the utilization intensity and thus the demand for storing cataract surgery videos for different purposes has increased. Hospitals continuously improve their technical recording equipment, i.e., cameras, to enhance the post-operative processing efficiency of the recordings. However, afterward, the videos are stored on hospitals’ internal data servers in their original size, which leads to a massive storage consumption. In this paper, we propose a relevance-based compression scheme. First, we perform a user study with clinicians to define the relevance rates of regular cataract surgery phases. Then, we compress different phases based on the determined relevance rates, using different encoding parameters and two coding standards, namely H.264/AVC and AV1. Afterward, the medical experts evaluate the visual quality of the encoded videos. Our results show a storage-saving potential for H.264/AVC of up to 95.94% and up to 98.82% for AV1, excluding idle phases (no tools are visible).



25th International Conference on Medical Image Computing and Computer Assisted Intervention, September 18-22, 2022, Singapore

Authors: Negin Ghamsarian, Mario Taschwer, Raphael Sznitman, Klaus Schoeffmann

Abstract: Semantic segmentation in cataract surgery has a wide range of applications contributing to surgical outcome enhancement and clinical risk reduction. However, the varying issues in segmenting the different relevant structures in these surgeries make the designation of a unique network quite challenging. This paper proposes a semantic segmentation network termed, DeepPyram, that can deal with these challenges using three novelties: (1) a Pyramid View Fusion module which provides a varying-angle global view of the surrounding region centering at each pixel position in the input convolutional feature map; (2) a Deformable Pyramid Reception module which enables a wide deformable receptive field that can adapt to geometric transformations in the object of interest; and (3) a dedicated Pyramid Loss that adaptively supervises multi-scale semantic feature maps. Compbined we show that these can effectively boost semantic segmentation performance, especially in the case of transparency, deformability, scalability, and blunt edges in objects. We demonstrate that our approach performs at a state-of-the-art level and outperforms a number of existing methods without imposing additional trainable parameters.


For his continuous support of the International Conference on MultiMedia Modeling (MMM) and his ongoing contributions to the MMM community, yesterday Assoc.Prof. Klaus Schöffmann was awarded with the MMM’22 Community Leadership Award at the Gala Dinner in Phu Quoc, Vietnam.


Over two days (June 6-7, 2022) the 11th Video Browser Showdown (VBS) –, co-organized by Klaus Schöffmann, took place in Phu Quoc, Vietnam. Sixteen teams participated  in about six hours of competition to solve video content search tasks in a large video dataset (V3C1+V3C2), comprising 2300 hours of video content (17235 video files). Eleven teams on site, together with 5 more teams online, accepted the challenge, competed with each other, and showcased their state-of-the-art video search systems to the conference public of MMM 2022 (  Congratulations to the winning teams of different search categories: vibro (Germany), CVHunter (Czech Republic), VISIONE (Italy).