Medical Multimedia Information Systems

,

Sahar Nasirihaghighi presented at CBMS 2023

Sahar Nasirihaghighi presented the paper titled “Action Recognition in Video Recordings from Gynecology Laparoscopy” at IEEE 36th International Symposium on Computer-Based Medical Systems 2023.

Authors: Sahar Nasirihaghighi, Negin Ghamsarian, Daniela Stefanics, Klaus Schoeffmann and Heinrich Husslein

Abstract: Action recognition is a prerequisite for many applications in laparoscopic video analysis including but not limited to surgical training, operation room planning, follow-up surgery preparation, post-operative surgical assessment, and surgical outcome estimation. However, automatic action recognition in laparoscopic surgeries involves numerous challenges such as (I) cross-action and intra-action duration variation, (II) relevant content distortion due to smoke, blood accumulation, fast camera motions, organ movements, object occlusion, and (III) surgical scene variations due to different illuminations and viewpoints. Besides, action annotations in laparoscopy surgeries are limited and expensive due to requiring expert knowledge. In this study, we design and evaluate a CNN-RNN architecture as well as a customized training-inference framework to deal with the mentioned challenges in laparoscopic surgery action recognition. Using stacked recurrent layers, our proposed network takes advantage of inter-frame dependencies to negate the negative effect of content distortion and variation in action recognition. Furthermore, our proposed frame sampling strategy effectively manages the duration variations in surgical actions to enable action recognition with high temporal resolution. Our extensive experiments confirm the superiority of our proposed method in action recognition compared to static CNNs.

,

A Webbing Journey @ A1 Austria eSports Festival 2023

Sebastian Uitz and Michael Steinkellner showcased their game, A Webbing Journey, at the A1 Austria eSports Festival in the Austria Center Vienna on May 27, 2023. The booth, featuring two PCs, a Steam Deck, and a Nintendo Switch, offered players of all ages a delightful experience. Valuable feedback was gathered, fueling the team’s determination to enhance the game for future events.

We are grateful for the positive response and eagerly await incorporating the feedback received. With its endearing storyline and unique gameplay mechanics, the game continues to build anticipation for its official release, offering an enchanting adventure filled with exploration and heartwarming quests.

Paper Accepted: Domain Adaptation for Medical Image Segmentation using Transformation-Invariant Self-Training

Authors: Negin Ghamsarian, Javier Gamazo Tejero, Pablo Márquez Neila, Sebastian Wolf, Martin Zinkernagel, Klaus Schoeffmann, and Raphael Sznitman

26th Medical Image Computing and Computer-Assisted Intervention 2023 (MICCAI 2023), Vancouver, Canada, 8-12 October 2023

Abstract: Models capable of leveraging unlabelled data are crucial in overcoming large distribution gaps between the acquired datasets across different imaging devices and configurations. In this regard, self-training techniques based on pseudo-labeling have been shown to be highly effective for unsupervised domain adaptation. However, the unreliability of pseudo labels can hinder the capability of self-training techniques to induce abstract representation from the unlabeled target dataset, especially in the case of large distribution gaps. Since the neural network performance should be invariant to image transformations, we look to this fact to identify uncertain pseudo labels. Indeed, we argue that transformation invariant detections can provide more reasonable approximations of ground truth. Accordingly, we propose an unsupervised domain adaptation strategy termed transformation-invariant self-training (TI-ST) to assess pixel-wise pseudo-labels’ reliability and filter out unreliable detections during self-training. We perform comprehensive evaluations for domain adaptation using three different modalities of medical images, two different network architectures, and several alternative state-of-the-art domain adaptation methods. Experimental results confirm the superiority of our proposed method in mitigating the lack of target domain annotation and boosting segmentation performance in the target domain.

,

ITEC Wins the International Lifelog Search Challenge 2023 (LSC2023)

We are very happy to announce that Klaus Schoeffmann, with his lifelog retrieval system lifeXplore, has won the Lifelog Search Challenge 2023 (LSC2023). The LSC2023 was performed on June 12, 2023, in Thessaloniki, Greece, as a workshop at the ACM International Conference on Multimedia Retrieval (ICMR2023). In total, 30 different search tasks (known-item search, ad-hoc topic search, and question answering) had to be solved by all 14 international teams. After four hours of strong competition, lifeXplore came out on top of the other search systems and scored first.

Mathias Lux
,

Developing a Virtual-Reality Game that Fosters Empathy and Perspective-Taking

Prejudice and discrimination against immigrants have been a constant presence in Western Europe for a long time, fueled considerably by the refugee crisis in 2015. This project, supervised by researchers from Game Studies & Engineering and Psychology, explores the possibilities of using virtual reality (VR)-based video games to increase empathy and positive attitudes toward the integration of refugees in our society. Virtual-reality games have enormous potential for immersing players emotionally in situations outside their usual experience and in the perspectives of others. The project involves the development and evaluation of a game in which participants go through typical experiences of refugees in a European country and an experimental study that assesses the effects of playing the game on empathy, perspective-taking, and implicit and explicit attitudes toward refugees. We believe that this approach has the potential to open up a new, attractive route for changing people’s attitudes through immersive virtual experiences.

The project was proposed by Judith Glück from the psychology department and Felix Schniz and Mathias Lux from ITEC and accepted for funding within the Ada Lovelace program of the University of Klagenfurt. The project will start in Q3 or Q4 2023 and has a duration of 4 years.

,

Paper accepted: Action Recognition in Video Recordings from Gynecologic Laparoscopy

Title: Action Recognition in Video Recordings from Gynecologic Laparoscopy

Authors: Sahar Nasirihaghighi, Negin Ghamsarian, Daniela Stefanics, Klaus Schoeffmann and Heinrich Husslein

IEEE 36th International Symposium on Computer-Based Medical Systems 2023

Abstract: Action recognition is a prerequisite for many applications in laparoscopic video analysis including but not limited to surgical training, operation room planning, follow-up surgery preparation, post-operative surgical assessment, and surgical outcome estimation. However, automatic action recognition in laparoscopic surgeries involves numerous challenges such as (I) cross-action and intra-action duration variation, (II) relevant content distortion due to smoke, blood accumulation, fast camera motions, organ movements, object occlusion, and (III) surgical scene variations due to different illuminations and viewpoints. Besides, action annotations in laparoscopy surgeries are limited and expensive due to requiring expert knowledge. In this study, we design and evaluate a CNN-RNN architecture as well as a customized training-inference framework to deal with the mentioned challenges in laparoscopic surgery action recognition. Using stacked recurrent layers, our proposed network takes advantage of inter-frame dependencies to negate the negative effect of content distortion and variation in action recognition. Furthermore, our proposed frame sampling strategy effectively manages the duration variations in surgical actions to enable action recognition with high temporal resolution. Our extensive experiments confirm the superiority of our proposed method in action recognition compared to static CNNs.

Mathias Lux

Hüttenjam 2023 – Awesome Moments

Creating games for 48 hours at 1770 meters of altitude. The 2nd Hüttenjam took place April 13-16, 2023, on the Turracher Höhe. After settling on a topic for the jam on Thursday – extreme conditions – 40 participants worked on 8 games to be presented on Saturday evening. The jam took place in a chilled atmosphere and allowed for networking, winter sports, a sauna, and a lot of creative space. A video summary of the event is available at https://youtu.be/8IwSBsk-3Fc You can also play the games developed there at https://itch.io/jam/2nd-huettenjam
The Hüttenjam is a joint event of Game Dev Graz and the University of Klagenfurt / ITEC: https://www.xn--httenjam-65a.at-

 

,

Success story @ Game Studies and Engineering

Christina Obmann, one of our first Game Studies and Engineering students, has been recognized as outstanding in the Carinthia region by the local newspaper Kleine Zeitung. Besides her interest and work in games, she’s teaching at the university, learning Chinese, and was awarded a scholarship from Huawei.

 

 

,

Outstanding Reviewer Award at ACM ICMR 2022 for Klaus Schöffmann

For the quality and timeliness of his reviews, Klaus Schöffmann has been awarded with the Outstanding Reviewer Award at the ACM International Conference on Multimedia Retrieval (ICMR) 2022, which was held at Newark, NJ, USA in June 2022.

 

,

A Webbing Journey @ Level Up 2022

On Friday and Saturday (July 1 and July 2, 2022), Sebastian Uitz presented his game “A Webbing Journey” with his partner Manuel Santner at the Level Up event in the Messezentrum Salzburg. This was the biggest event they presented their game at, with a 300m² area for indie games and over 20.000m² in total. Over 6.500 people visited the event, and 2 PCs were provided for their game. In the end, 6 PCs were running “A Webbing Journey” due to some game developers not making it or leaving the event early. All the PCs were in constant use, and all players had so much fun playing the game, especially kids. This event and the last two provided so much feedback in the form of player tests, which have helped progress the game. The next goal is to implement all the feedback and releasing the new demo version on Steam.