Multimedia Communication

,

Paper Accepted: Characterization of the Quality of Experience and Immersion of Point Cloud Video Sequences through a Subjective Study @ IEEE Access

IEEE Access, A Multidisciplinary, Open-access Journal of the IEEE

Title: Characterization of the Quality of Experience and Immersion of Point Cloud Video Sequences through a Subjective Study @ IEEE Access

Authors: Minh Nguyen, Shivi Vats, Sam Van Damme (Ghent University – imec and KU Leuven, Belgium), Jeroen van der Hooft (Ghent University – imec, Belgium), Maria Torres Vega (Ghent University – imec and KU Leuven, Belgium), Tim Wauters (Ghent University – imec, Belgium), Filip De Turck (Ghent University – imec, Belgium), Christian Timmerer, Hermann Hellwagner

Abstract: Point cloud streaming has recently attracted research attention as it has the potential to provide six degrees of freedom movement, which is essential for truly immersive media. The transmission of point clouds requires high-bandwidth connections, and adaptive streaming is a promising solution to cope with fluctuating bandwidth conditions. Thus, understanding the impact of different factors in adaptive streaming on the Quality of Experience (QoE) becomes fundamental. Point clouds have been evaluated in Virtual Reality (VR), where viewers are completely immersed in a virtual environment. Augmented Reality (AR) is a novel technology and has recently become popular, yet quality evaluations of point clouds in AR environments are still limited to static images.

In this paper, we perform a subjective study of four impact factors on the QoE of point cloud video sequences in AR conditions, including encoding parameters (quantization parameters, QPs), quality switches, viewing distance, and content characteristics. The experimental results show that these factors significantly impact the QoE. The QoE decreases if the sequence is encoded at high QPs and/or switches to lower quality and/or is viewed at a shorter distance, and vice versa. Additionally, the results indicate that the end user is not able to distinguish the quality differences between two quality levels at a specific (high) viewing distance. An intermediate-quality point cloud encoded at geometry QP (G-QP) 24 and texture QP (T-QP) 32 and viewed at 2.5 m can have a QoE (i.e., score 6.5 out of 10) comparable to a high-quality point cloud encoded at 16 and 22 for G-QP and T-QP, respectively, and viewed at a distance of 5 m. Regarding content characteristics, objects with lower contrast can yield better quality scores. Participants’ responses reveal that the visual quality of point clouds has not yet reached an immersion level as desired. The average QoE of the highest visual quality is less than 8 out of 10. There is also a good correlation between objective metrics (e.g., color Peak Signal-to-Noise Ratio (PSNR) and geometry PSNR) and the QoE score. Especially the Pearson correlation coefficients of color PSNR is 0.84. Finally, we found that machine learning models are able to accurately predict the QoE of point clouds in AR environments.

The subjective test results and questionnaire responses are available on Github: https://github.com/minhkstn/QoE-and-Immersion-of-Dynamic-Point-Cloud.

,

Paper accepted: Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video Streaming Quality @ CoNEXT, Paris December 5-8 2023

The 19th International Conference on emerging Networking EXperiments and Technologies (CoNEXT) Paris, France, December 5-8, 2023

Authors: Leonardo Peroni (IMDEA Networks Institute), Sergey Gorinsky (IMDEA Networks Institute), Farzad Tashtarian (Alpen-Adria-Universität Klagenfurt, Austria), and Christian Timmerer (Alpen-Adria-Universität Klagenfurt, Austria).

Abstract: Quality of Experience (QoE) and QoE models are of an increasing importance to networked systems. The traditional QoE modeling for video streaming applications builds a one-size-fits-all QoE model that underserves atypical viewers who perceive QoE differently. To address the problem of atypical viewers, this paper proposes iQoE (individualized QoE), a method that employs explicit, expressible, and actionable feedback from a viewer to construct a personalized QoE model for this viewer. The iterative iQoE design exercises active learning and combines a novel sampler with a modeler. The chief emphasis of our paper is on making iQoE sample-efficient and accurate.
By leveraging the Microworkers crowdsourcing platform, we conduct studies with 120 subjects who provide 14,400 individual scores. According to the subjective studies, a session of about 22 minutes empowers a viewer to construct a personalized QoE model that, compared to the best of the 10 baseline models, delivers the average accuracy improvement of at least 42% for all viewers and at least 85\% for the atypical viewers. The large-scale simulations based on a new technique of synthetic profiling expand the evaluation scope by exploring iQoE design choices, parameter sensitivity, and generalizability.

 

,

Paper accepted: Towards Low-Latency and Energy-Efficient Hybrid P2P-CDN Live Video Streaming @ IEEE COMSOC MMTC

Special Issue on Sustainable Multimedia Communications and Services, IEEE COMSOC MMTC Communications – Frontiers

Title: Towards Low-Latency and Energy-Efficient Hybrid P2P-CDN Live Video Streaming

Authors: Reza Farahani, Christian Timmerer, and Hermann Hellwagner

Abstract: Streaming segmented videos over the Hypertext Transfer Protocol (HTTP) is an increasingly popular approach in both live and video-on-demand (VoD) applications. However, designing a scalable and adaptable framework that reduces servers’ energy consumption and supports low latency and high quality services, particularly for live video streaming scenarios, is still challenging for Over-The-Top (OTT) service providers. To address such challenges, this paper introduces a new hybrid P2P-CDN framework that leverages new networking and computing paradigms, i.e., Network Function Virtualization (NFV) and edge computing for live video streaming. The proposed framework introduces a multi-layer architecture and a tree of possible actions therein (an action tree), taking into account all available resources from peers, edge, and CDN servers to efficiently distribute video fetching and transcoding tasks across a hybrid P2P-CDN network, consequently enhancing the users’ latency and video quality. We also discuss our testbed designed to validate the framework and compare it with baseline methods. The experimental results indicate that the proposed framework improves user Quality of Experience (QoE), reduces client serving latency, and improves edge server energy consumption compared to baseline approaches.

,

Paper accepted: Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Streaming @ IEEE VCIP´23

International Conference on Visual Communications and Image Processing (IEEE VCIP’23)

http://www.vcip2023.org/

Authors: Vignesh V Menon, Reza Farahani, Prajit T Rajendran, Samira Afzal, Klaus Schoeffmann, Christian Timmerer

Abstract: With the emergence of multiple modern video codecs, streaming service providers are forced to encode, store, and transmit bitrate ladders of multiple codecs separately, consequently suffering from additional energy costs for encoding, storage, and transmission. To tackle this issue, we introduce an online energy-efficient Multi-Codec Bitrate ladder Estimation scheme (MCBE) for adaptive video streaming applications. In MCBE, quality representations within the bitrate ladder of new-generation codecs (e.g., HEVC, AV1) that lie below the predicted rate-distortion curve of the AVC codec are removed. Moreover, perceptual redundancy between representations of the bitrate ladders of the considered codecs is also minimized based on a Just Noticeable Difference (JND) threshold. Therefore, random forest-based models predict the VMAF of bitrate ladder representations of each codec. In a live streaming session where all clients support the decoding of AVC, HEVC, and AV1, MCBE achieves impressive results, reducing cumulative encoding energy by 56.45%, storage energy usage by 94.99%, and transmission energy usage by 77.61% (considering a JND of six VMAF points). These energy reductions are in comparison to a baseline bitrate ladder encoding based on current industry practice.

,

Paper accepted: Community-Based QoE Enhancement for User-Generated Content Live Streaming @ 13th International Conference on Computer and Knowledge Engineering (ICCKE)

Authors: Reza Saeedinia (University of Tehran), S. Omid Fatemi (University of Tehran),Daniele Lorenzi (Alpen-Adria Universität Klagenfurt), Farzad Tashtarian (Alpen-Adria Universität Klagenfurt),  Christian Timmerer (Alpen-Adria Universität Klagenfurt)

Abstract: Live user-generated content (UGC) has increased significantly in video streaming applications. Improving the quality of experience (QoE) for users is a crucial consideration in UGC live streaming, where a user can be both a subscriber and a streamer. Resource allocation is an NP-complete task in UGC live streaming due to many subscribers and streamers with varying requests, bandwidth limitations, and network constraints. In this paper, to decrease the execution time of the resource allocation algorithm, we first process streamers’ and subscribers’ requests and then aggregate them into a limited number of groups based on their preferences. Second, we
perform resource allocation for these groups that we call communities. We formulate the resource allocation problem for communities into an optimization problem. With an efficient aggregation of subscribers and streamers at the core of the proposed architecture, the computational complexity of the optimization problem is reduced, consequently improving QoE. This improvement occurs because of the prompt reaction to the bandwidth fluctuations and, subsequently, appropriate resource allocation by the proposed model. We conduct experiments in various scenarios. The results show an average of 41% improvement in execution time. To evaluate the impact of bandwidth fluctuations on the proposed algorithm, we employ two network traces: AmazonFCC and NYUBUS. The results show 4%, and 28% QoE improvement in a scenario with 5
streamers over the AmazonFCC and the NYUBUS network traces, respectively.

Link: 13th International Conference on Computer and Knowledge Engineering (ICCKE)

,

Kurt Horvath presented his work at EURO-PAR 2023

Kurt Horvath presented the paper titled A distributed geofence-based discovery scheme for the computing Continuum at 29th International European Conference on Parallel and Distributed Computing (EURO-PAR 2023)

Authors: Kurt Horvath, Dragi Kimovski, Christoph Uran, Helmut Wöllik, and Radu Prodan

Abstract: Service discovery is a vital process that enables low latency provisioning of Internet of Things applications across the computing continuum. Unfortunately, it becomes increasingly difficult to identify a proper service within strict time constraints due to the high heterogeneity of the computing continuum. Moreover, the plethora of network technologies and protocols commonly used by the Internet of Things applications further hinders service discovery. To address these issues, we introduce a novel mobile edge service discovery algorithm named Mobile Edge Service Discovery using the DNS (MESDD), which utilizes intermediate code to identify a suitable service instance across the computing continuum based on the naming scheme used to identify the users’ location. MESDD utilizes geofences to aid this process, which enables fine-grained resource discovery. We deployed a real-life distributed computing continuum testbed and compared MESDD with three related methods. The evaluation results show that MESDD outperforms the other approaches by 60% after eight discovery iterations.

, ,

IEEE MIPR 2023: Keynote Speaker

Prof. Hermann Hellwagner is a keynote speaker at IEEE MIPR, 30th August – 1st September 2023.

Title: Advances in Edge-Based and In-Network Media Processing for Adaptive Video Streaming

Talk Abstract: Media traffic (mainly, video) on the Internet is constantly growing; networked multimedia applications consume a predominant share of the available Internet bandwidth. A major technical breakthrough and enabler in multimedia systems research was the HTTP Adaptive Streaming (HAS) technique. While this technique is widely used and works well in industrial networked multimedia services today, challenges exist for future multimedia systems, dealing with the trade-offs between (i) the ever-increasing content complexity, (ii) various requirements with respect to time (most importantly, low latency), and (iii) quality of experience (QoE). This situation sets the stage for our research work in the ATHENA Christian Doppler (CD) Laboratory (Adaptive Streaming over HTTP and Emerging Networked Multimedia Services; https://athena.itec.aau.at/), jointly funded by public sources and industry.

In this talk, I’ll explore one facet of the ATHENA research, namely how and with which benefits edge-based and in-network media processing can cope with adverse network conditions and/or improve media quality/perception. Content Delivery Networks (CDNs) are the classical example of supporting content distribution on today’s Internet. In recent years, though, techniques like Multi-access Edge Computing (MEC), Software Defined Networking (SDN), Network Function Virtualization (NFV), Peer Assistance (PA) for CDNs, and Machine Learning (ML) have emerged that can additionally be leveraged to support adaptive video streaming services. In the talk, I’ll present several approaches of edge-based and in-network media processing in support of adaptive streaming, in four groups:

  1. Edge Computing (EC) support, for instance transcoding, content prefetching, and adaptive bitrate algorithms at the edge.
  2. Virtualized Network Function (VNF) support for live video streaming.
  3. Hybrid P2P, Edge and CDN support including content caching, transcoding, and super-resolution at various layers of the system.
  4. Machine Learning (ML) techniques facilitating various (end-to-end) properties of an adaptive streaming system.

ARTEMIS: Adaptive Bitrate Ladder Optimization for Live Video Streaming

21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 2024), APRIL 16–18, 2024, SANTA CLARA, CA, USA

Authors: Farzad Tashtarian (Alpen-Adria Universität Klagenfurt),  Abdelhak Bentaleb (Concordia University), Hadi Amirpour (Alpen-Adria Universität Klagenfurt) , Sergey Gorinsky (IMDEA Networks Institute),  Junchen Jiang (University of Chicago), Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt) , Christian Timmerer (Alpen-Adria Universität Klagenfurt)

Live streaming of segmented videos over the Hypertext Transfer Protocol (HTTP) is increasingly popular and serves heterogeneous clients by offering each segment in multiple representations. A bitrate ladder expresses this choice as an ordered list of bitrate-resolution pairs. Whereas existing solutions for HTTP-based live streaming use a static bitrate ladder, the fixed ladders struggle to appropriately accommodate the dynamics in the video content and network-conditioned client capabilities. This paper proposes ARTEMIS as a practical scalable alternative that dynamically configures the bitrate ladder depending on the content complexity, network conditions, and clients’ statistics. ARTEMIS seamlessly integrates with the end-to-end streaming pipeline and operates transparently to video encoders and clients. We develop a cloud-based implementation of ARTEMIS and conduct extensive real-world and trace-driven experiments. The experimental comparison vs. existing prominent bitrate ladders demonstrates that live streaming with ARTEMIS outperforms all baselines, reduces encoding computation by 25%, end-to-end latency by 18%, and increases quality of experience (QoE) by 11%.

,

A Webbing Journey @ Level Up 2023

Sebastian Uitz and Hannes Dermutz had an amazing time showcasing their highly anticipated game, A Webbing Journey, at the Level Up event at Messe Salzburg on July 1 and 2, 2023. The event was a vibrant gathering of game developers and enthusiasts, providing the perfect platform to connect with fellow game devs and experience many fantastic games.

At our booth, attendees had the opportunity to immerse themselves in the enchanting world of “A Webbing Journey” on the PC, Steam Deck, and Nintendo Switch. Players of all ages were captivated by the game’s endearing storyline and unique gameplay mechanics, embarking on a spider’s extraordinary adventure. The valuable feedback from the event-goers will be crucial in further refining and enhancing the game for its upcoming release.

In addition to the exhilarating gameplay experience, we had the privilege of sitting down for an insightful interview with the FM4 radio channel. It was an incredible opportunity to discuss the inspiration behind “A Webbing Journey” and delve into the game’s captivating features. We’re grateful for the chance to share our journey with a broader audience and promote the excitement surrounding our game.

IEEE Network Special Issue on Advancements in Network-Assisted Video Streaming: Optimization and Performance Analysis

Call for Papers

Network-assisted video streaming has become a substantial part of modern multimedia applications, enabling users to access high-quality video content over different networks, including the Internet and wireless networks. Efficiently delivering video content over networks poses numerous challenges, such as limited bandwidth, varying network conditions, heterogeneous end devices, and diverse user preferences. Network-assisted video streaming approaches leverage modern networking technologies, such as Software-Defined Networking (SDN), Network Function Virtualization (NFV), and edge computing, to not only improve the users’ Quality of Experience (QoE) but also enhance network utilization. Read more