Multimedia Information Systems (MMIS)

We at ITEC strongly believe that multimedia content creation, retrieval, sharing and processing offer great research challenges and research in this area is beneficial to cultural and technology development of our society.

More specifically, we focus our research on

• Interactive multimedia search and retrieval
• Multimedia analytics
• Medical multimedia information systems
• Multimedia indexing and retrieval
• User-centered multimedia and multimedia experience

One of our main research topics is medical multimedia information systems, where we investigate novel methods to automatically analyze the content of medical videos (from endoscopic inspections/surgeries, or microscopic surgeries) and use analysis results to build up an information retrieval system for a medical video database. This includes automatic filtering of relevant content, effective domain-specific compression of video data, automatic content indexing and detection of semantics, such as anatomical structures, instruments, surgical actions, and surgical errors, as well as the investigation of efficient and appropriate interfaces to provide browsing and searching (i.e., content-based retrieval) in the underlying video data. Relevant research projects in this field are: EndoVIP, EndoVIP2, CODE-MM, and KISMET. More details on our research on medical multimedia information systems can be found here.

Another important topic is interactive video retrieval (for videos in general), where we investigate novel interfaces and approaches to improve content-based search in large video archives. For that effort, we organize an annual international video retrieval competition since 2012, called the Video Browser Showdown. There we evaluate video retrieval systems developed by international researchers in the field of multimedia, which compete in solving Known-Item Search (KIS) as well as Ad-Hoc Video Search (AVS) tasks as quickly as possible for a shared video dataset. For the last three years we used the IAAC.3 dataset for that purpose, consisting of 4.593 Internet Archive videos, covering 600h of video content, in collaboration with NIST/TRECVID, which also provided challenging AVS tasks.

ITEC also has a strong track record in visual information retrieval including local and global visual features, temporal features for video retrieval and lately deep learning for visual information retrieval. The popular open source software libraries Lire and LireSolr are just two examples of successful research to application projects. We also consider multimedia not independent from users and investigate user intentions and user goals in context of multimedia consumption and retrieval systems, i.e., by investigating relevance of search results in comparison to the question why people actually triggered a search. In this context we also organize tracks at MultimediaEval on a regular basis.