Multimedia Communication

Vignesh V Menon

Paper accepted – ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming

2022 IEEE International Conference on Image Processing (ICIP)

October 16-19, 2022 | Bordeaux, France

Conference Website

[Video]

Vignesh V Menon (Alpen-Adria-Universität Klagenfurt),  Hadi Amirpour (Alpen-Adria-Universität Klagenfurt), Mohammad Ghanbari (School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK), and Christian Timmerer (Alpen-Adria-Universität Klagenfurt)

Abstract:

In two-pass encoding, also known as multi-pass encoding, the input video content is analyzed in the first-pass to help the second-pass encoding utilize better encoding decisions and improve overall compression efficiency. In live streaming applications, a single-pass encoding scheme is mainly used to avoid the additional first-pass encoding run-time to analyze the complexity of every video content. This paper introduces an Efficient low-latency Two-Pass encoding Scheme (ETPS) for live video streaming applications. In this scheme, Discrete Cosine Transform (DCT)-energy-based low-complexity spatial and temporal features for every video segment are extracted in the first-pass to predict each target bitrate’s optimal constant rate factor (CRF) for the second-pass constrained variable bitrate (cVBR) encoding. Experimental results show that, on average, ETPS compared to a traditional two-pass average bitrate encoding scheme yields encoding time savings of 43.78% without any noticeable drop in compression efficiency. Additionally, compared to a single-pass constant bitrate (CBR) encoding, it yields bitrate savings of 10.89% and 8.60% to maintain the same PSNR and VMAF, respectively.

ETPS architecture

Paper accepted: Towards Better Quality of Experience in HTTP Adaptive Streaming

16th International Conference on Signal Image Technology & Internet based Systems – Dijon, France – October 19-21, 2022

Conference Website

Babak Taraghi (Alpen-Adria-Universität Klagenfurt, Austria), Selina Zoë Haack (Alpen-Adria-Universität Klagenfurt, Austria), and Christian Timmerer (Alpen-Adria-Universität Klagenfurt, Austria)

Abstract: HTTP Adaptive Streaming (HAS) is nowadays a popular solution for multimedia delivery. The novelty of HAS lies in the possibility of continuously adapting the streaming session to current network conditions, facilitated by Adaptive Bitrate (ABR) algorithms. Various popular streaming and Video on Demand services such as Netflix, Amazon Prime Video, and Twitch use this method. Given this broad consumer base, ABR algorithms continuously improve to increase user satisfaction. The insights for these improvements are, among others, gathered within the research area of Quality of Experience (QoE). Within this field, various researchers have dedicated their works to identifying potential impairments and testing their impact on viewers’ QoE. Two frequently discussed visual impairments influencing QoE are stalling events and quality switches. So far, it is commonly assumed that those stalling events have the worst impact on QoE. This paper challenged this belief and reviewed this assumption by comparing stalling events with multiple quality and high amplitude quality switches. Two subjective studies were conducted. During the first subjective study, participants received a monetary incentive, while the second subjective study was carried out with volunteers. The statistical analysis demonstrated that stalling events do not result in the worst degradation of QoE. These findings suggest that a reevaluation of the effect of stalling events in QoE research is needed. Therefore, these findings may be used for further research and to improve current adaptation strategies in ABR algorithms.

,

Paper accepted: ARARAT: A Collaborative Edge-Assisted Framework for HTTP Adaptive Video Streaming

IEEE Transactions on Network and Service Management (TNSM)

Journal Website

Authors: Reza Farahani (Alpen-Adria-Universität Klagenfurt, Austria), Mohammad Shojafar (University of Surry, UK), Christian Timmerer (Alpen-Adria-Universität Klagenfurt, Austria), Farzad Tashtarian (Alpen-Adria-Universität Klagenfurt, Austria), Mohammad Ghanbari (University of Essex, UK), and Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt, Austria)

Abstract: With the ever-increasing demands for high-definition and low-latency video streaming applications, network-assisted video streaming schemes have become a promising complementary solution in the HTTP Adaptive Streaming (HAS) context to improve users’ Quality of Experience (QoE) as well as network utilization. Edge computing is considered one of the leading networking paradigms for designing such systems by providing video processing and caching close to the end-users. Despite the wide usage of this technology, designing network-assisted HAS architectures that support low-latency and high-quality video streaming, including edge collaboration is still a challenge. To address these issues, this article leverages the Software-Defined Networking (SDN), Network Function Virtualization (NFV), and edge computing paradigms to propose A collaboRative edge-Assisted framewoRk for HTTP Adaptive video sTreaming (ARARAT). Aiming at minimizing HAS clients’ serving time and network cost, besides considering available resources and all possible serving actions, we design a multi-layer architecture and formulate the problem as a centralized optimization model executed by the SDN controller. However, to cope with the high time complexity of the centralized model, we introduce three heuristic approaches that produce near-optimal solutions through efficient collaboration between the SDN controller and edge servers. Finally, we implement the ARARAT framework, conduct our experiments on a large-scale cloud-based testbed including 250 HAS players, and compare its effectiveness with state-of-the-art systems within comprehensive scenarios. The experimental results illustrate that the proposed ARARAT methods (i) improve users’ QoE by at least 47%, (ii) decrease the streaming cost, including bandwidth and computational costs, by at least 47%, and (iii) enhance network utilization, by at least 48% compared to state-of-the-art approaches.

Hadi

Hadi Amirpour to give a talk at INSA France

LiVE: Toward Better Live Video Experience

INSA, France

 27th September 2022 | Rennes, France

 

Abstract: In this presentation, we first introduce the principles of video streaming and the existing challenges. While live video streaming is expected to continue growing at an accelerated pace, one potential area for optimization that has remained relatively untapped is the use of content-aware encoding to improve the quality of live contribution streams due to avoid of latency. In this talk, we introduce revolutionary real-time content-aware video quality improvement methods for live applications that keep the added latency very low.

 

 

 

Hadi Amirpour is a postdoctoral researcher at the University of Klagenfurt. He received his B.Sc. degrees in Electrical and Biomedical Engineering, and he pursued his M.Sc. in Electrical Engineering. He got his Ph.D. in computer science from the University of Klagenfurt in 2022. He was involved in the project EmergIMG, a Portuguese consortium on emerging imaging technologies, funded by the Portuguese funding agency and H2020. Currently, he is working on the ATHENA project in cooperation with its industry partner Bitmovin. His research interests are image processing and compression, video processing and compression, quality of assessment, emerging 3D imaging technology, and medical image analysis.

,

ARTICONF project review was a great sucssess

The project partners reunited at @itecmmc for a final project review. Thank you Horizon2020 @EU_Commission it has been an honour to collaborate for the Future Hyper-connected Sociality.

ARTICONF project review

@UvA_Amsterdam
@mscdigsoc
@UOhrid
@MOGTechnologies
@AgiliaCenter
@vialog_io
@bitYogaAS
@itec

, ,

Paper accepted: CADLAD: Device-aware Bitrate Ladder Construction for HTTP Adaptive Streaming

18th International Conference on Network and Service Management (CNSM 2022)

Thessaloniki, Greece | 31 October – 4 November 2022

Conference Website

Minh Nguyen (Alpen-Adria-Universität Klagenfurt, Austria), Babak Taraghi (Alpen-Adria-Universität Klagenfurt, Austria), Abdelhak Bentaleb (National University of Singapore, Singapore), Roger Zimmermann (National University of Singapore, Singapore), and Christian Timmerer (Alpen-Adria-Universität Klagenfurt, Austria)

Abstract: Considering network conditions, video content, and viewer device type/screen resolution to construct a bitrate ladder is necessary to deliver the best Quality of Experience (QoE).
A large-screen device like a TV needs a high bitrate with high resolution to provide good visual quality, whereas a small one like a phone requires a low bitrate with low resolution. In
addition, encoding high-quality levels at the server side while the network is unable to deliver them causes unnecessary cost for the content provider. Recently, the Common Media Client Data (CMCD) standard has been proposed, which defines the data that is collected at the client and sent to the server with its HTTP requests. This data is useful in log analysis, quality of service/experience monitoring and delivery improvements.

cadlad

 

In this paper, we introduce a CMCD-Aware per-Device bitrate LADder construction (CADLAD) that leverages CMCD to address the above issues. CADLAD comprises components at both client and server sides. The client calculates the top bitrate (tb) — a CMCD parameter to indicate the highest bitrate that can be rendered at the client — and sends it to the server together with its device type and screen resolution. The server decides on a suitable bitrate ladder, whose maximum bitrate and resolution are based on CMCD parameters, to the client device with the purpose of providing maximum QoE while minimizing delivered data. CADLAD has two versions to work in Video on
Demand (VoD) and live streaming scenarios. Our CADLAD is client agnostic; hence, it can work with any players and ABR algorithms at the client. The experimental results show that CADLAD is able to increase the QoE by 2.6x while saving 71% of delivered data, compared to an existing bitrate ladder of an available video dataset. We implement our idea within CAdViSE — an open-source testbed for reproducibility.

 

,

Paper accepted: Hybrid P2P-CDN Architecture for Live Video Streaming: An Online Learning Approach

IEEE Global Communications Conference (GLOBECOM)

December 4-8, 2022 |Rio de Janeiro, Brazil
Conference Website

Authors: Reza Farahani (Alpen-Adria-Universität Klagenfurt, Austria), Abdelhak Bentaleb (National University of Singapore, Singapore), Ekrem Cetinkaya (Alpen-Adria-Universität Klagenfurt, Austria), Christian Timmerer (Alpen-Adria-Universität Klagenfurt, Austria), Roger Zimmermann (National University of Singapore, Singapore), and Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt, Austria)

Abstract: a cost-effective, scalable, and flexible architecture that supports low latency and high-quality live video streaming is still a challenge for Over-The-Top (OTT) service providers. To cope with this issue, this paper leverages Peer-to-Peer (P2P), Content Delivery Network (CDN), edge computing, Network Function Virtualization (NFV), and distributed video transcoding paradigms to introduce a hybRId P2P-CDN arcHiTecture for livE video stReaming (RICHTER). We first introduce RICHTER’s multi-layer architecture and design an action tree that considers all feasible resources provided by peers, edge, and CDN servers for serving peer requests with minimum latency and maximum quality. We then formulate the problem as an optimization model executed at the edge of the network. We present an Online Learning (OL) approach that leverages an unsupervised Self Organizing Map (SOM) to (i) alleviate the time complexity issue of the optimization model and (ii) make it a suitable solution for large-scale scenarios by enabling decisions for groups of requests instead of for single requests. Finally, we implement the RICHTER framework, conduct our experiments on a large-scale cloud-based testbed including 350 HAS players, and compare its effectiveness with baseline systems. The experimental results illustrate that RICHTER outperforms baseline schemes in terms of users’ Quality of Experience (QoE), latency, and network utilization, by at least 59%, 39%, and 70%, respectively.

Vignesh V Menon
,

Video Coding Enhancements for HTTP Adaptive Streaming

ACM Multimedia Conference – Doctoral Symposium Track

Lisbon, Portugal | 10-14 October 2022

Vignesh V Menon (Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität Klagenfurt)

Abstract: Rapid growth in multimedia streaming traffic over the Internet motivates the research and further investigation of the video coding performance of such services in terms of speed and Quality of Experience (QoE). HTTP Adaptive Streaming (HAS) is today’s de-facto standard to deliver clients the highest possible video quality. In HAS, the same video content is encoded at multiple bitrates, resolutions, framerates, and coding formats called representations. This study aims to (i) provide fast and compression-efficient multi-bitrate, multi-resolution representations, (ii) provide fast and compression-efficient multi-codec representations, (iii) improve the encoding efficiency of Video on Demand (VoD) streaming using content-adaptive encoding optimizations, and (iv) provide encoding schemes with optimizations per-title for live streaming applications to decrease the storage or delivery costs or/and increase QoE.

The ideal video compression system for HAS envisioned in this doctoral study.

 

Hadi

Paper accepted: Secure Reversible Data Hiding in Encrypted Images based on Classification Encryption Difference

Secure Reversible Data Hiding in Encrypted Images based on Classification Encryption Difference

IEEE 24th Workshop on MultiMedia Signal Processing (MMSP)

September 26-28, 2022 | Shanghai, China

Authors: Lingfeng Qu (Southwest Jiaotong University),  Hadi Amirpour (Alpen-Adria-Universität Klagenfurt),   Mohammad Ghanbari (University of Essex, UK), and Christian Timmerer (Alpen-Adria-Universität Klagenfurt), Hongjie He (Southwest Jiaotong University)


Abstract: This paper introduces an algorithm to improve the security, efficiency, and
embedding capacity of reversible data hiding in encrypted images (RDH-EI). It is based on classification encryption difference and adaptive fixed-length coding. Firstly, the prediction error image is obtained, the difference with a bin value greater than the encryption threshold in the difference histogram is found, and it is further modified to obtain the embedding threshold range. Then, under the condition of ensuring that the difference inside and outside the threshold range is not confused, the difference within the threshold is only scrambled, and the difference outside the threshold is scrambled and mod encrypted. After obtaining the encrypted image, an adaptive difference fixed-length coding method is proposed to encode and compress the differences within the threshold. The secret data is embedded in the multiple most significant bits of the encoded difference. Experimental results show that the embedding capacity of the proposed algorithm is improved compared with the state-of-the-art algorithm.

Interns at ATHENA (Summer 2022)

In July-August 2022, the ATHENA Christian Doppler Laboratory hosted four interns working on the following topics:

  • Fabio Zinner: A Study and Evaluation on HTTP Adaptive Video Streaming using Mininet
  • Moritz Pecher: Dataset Creation and HAS Basics
  • Per-Luca Thalmann: Codec-war: is it necessary? Welcome to the multi-codec world
  • Georg Kelih: Server Client Simulator for QoE with practical Implementation

At the end of their internships, they presented their works and achieved results, and received official certificates from the university. We believe the joint work with them was beneficial for both the laboratory and the interns. We would like to thank the interns for their genuine interest, productive work, and excellent feedback about our laboratory.

Fabio Zinner: “In my four weeks, I had an amazingly practical and theoretical experience which is very important for my future practical and academic line of work! It was great and fascinating working with Python, Mininet, Linux, FFMpeg, Gpac, Iperf, etc. I really liked working with ATHENA, and the experience I gathered was exceptional. Also, I am very happy that I had Reza Farahani as my supervisor!”

Per-Luca Thalmann: “I really enjoyed my 4 weeks at ATHENA. At first, I had to read a lot of articles and papers to get a basic understanding of Video Codecs and encoding. As I started my Main Project, which evaluated the performance of modern codecs with different video complexities, I noticed that everything I had read before was useful to progress faster towards my end goal. After I got the results of my script, which ran for over a week, I also noticed some outcomes which were not expected. Basically, that older codecs get at some very specific settings higher Scores than their successor. Whenever I got stuck or had any questions, my supervisor, Vignesh, helped me. I did not only improve my technical knowledge, but I also got a lot of insights into how research works, what is the motivation of research and also about the process for scientific research.”

Georg Kelih: “I worked by Athena as an Intern for a month and got the tasks to build a simulator which simulates the server-client communication (ABR, bitrate ladder, resource allocation) and shows the results in a graph and a Server Client script where the server runs on the local host and the client requests segments and plays them using python-vlc
My daily routine was pretty chill, not only we had only 30 hours to work, but also the programming was quite fun and challenging. So my day looked something like this I stand up go to work play a game of round table soccer and then start to work start Visual Studio Code and write the code I thought about yesterday hope that it runs, but it shows you just a few error messages start debugging then notice that it’s already time to eat something and that I am hungry, eat something and find finally after your lunch the silly error I made think about the new implementation and better ways to solve something and then it is already time to go, so you go to the Strandbad to swim a round and then drive home. Something like this, my daily routine looked like. For me, I think it was a bit too chill for my taste because I like the stress of a 40-hour week especially when I only work during my holidays.
But the rest was absolutely nice, especially that here by Athena are so many people from different countries is pretty cool. For myself, I learned not many new skills, but I found out about many new Linux tools and how to find information even more efficiently.”