Distributed and Parallel Systems

Title: Matching-based Scheduling of Asynchronous Data Processing Workflows on the Computing Continuum

Heidelberg, Germany | September 6-9, 2022

https://clustercomp.org/2022/

Authors: Narges Mehran, Zahra Najafabadi Samani, Dragi Kimovski, Radu Prodan

Abstract: Today’s distributed computing infrastructures encompass complex workflows for real-time data gathering, transferring, storage, and processing, quickly overwhelming centralized cloud centers. Recently, the computing continuum that federates the Cloud services with emerging Fog and Edge devices represents a relevant alternative for supporting the next-generation data processing workflows. However, eminent challenges in automating data processing across the computing continuum still exist, such as scheduling heterogeneous devices across the Cloud, Fog, and Edge layers. We propose a new scheduling algorithm called C3-MATCH, based on matching theory principles, involving two sets of players negotiating different utility functions: 1) workflow microservices that prefer computing devices with lower data processing and queuing times; 2) computing continuum devices that prefer microservices with corresponding resource requirements and less data transmission time. We evaluate C3-MATCH using real-world road sign inspection and sentiment analysis workflows on a federated computing continuum across four Cloud, Fog, and Edge providers. Our combined simulation and real execution results reveal that C3-MATCH achieves up to 67% lower completion time compared to three state-of-the-art methods.

Title: An Elastic and Traffic-Aware Scheduler for Distributed Data Stream Processing in Heterogeneous Clusters

Authors: Hamid Hadian, Mohammadreza Farrokh, Mohsen Sharifi, and Ali Jafari

Abstract:

Existing Data Stream Processing (DSP) systems perform poorly while encountering heavy workloads, particularly on clustered set of (heterogeneous) computers. Elasticity and changing application parallelism degree can limit the performance degradation in the face of varying workloads that negatively impact the overall application response time. Elasticity can be achieved by operator scaling, i.e., by replication and relocation of operators at runtime. However, scaling decisions at runtime is challenging, since it first increases the overall communication overhead between operators and secondly changes any initial scheduling that could lead to a non-optimal scheduling plan. In this paper, we investigate the problem of elasticity and scaling decisions and propose a DSP system called ER-Storm. To curb communication overhead, we propose a new 3-step mechanism for replication and relocation of operators upon detecting a bottleneck operator that overutilizes a worker node. The other challenge is to select the proper worker nodes to host relocated operators. By discretizing the input workload, we model the relocation of operators between worker nodes at runtime through a scalable Markov Decision Process (MDP) and use a model-free notion of reinforcement learning (Q-Learning) to find optimal solutions. We have implemented our propositions on the Apache Storm version 2.1.0. Our experimental results show that ER-Storm reduces the average topology response time by 20 to 60 percent based on the rate of input workload (low or high) compared to the R-Storm scheduler and the Online-Scheduler of Storm.

https://open-research-europe.ec.europa.eu/collections/cloud-based-technologies/about

Results of collaborative work in the ADAPT between Austira (FFG) and China (CAS) accteped at flagship conference of IEEE Intelligent Transportation Systems Society

Title: Hybrid On/Off Blockchain Approach for Vehicle Data Management, Processing and Visualization Exemplified by the ADAPT Platform

Authors: Aso Validi, Vladislav Kashansky, Jihed Khiari, Hamid Hadian, Radu Pordan, Juanjuan Li, Fei-Yue Wang, Cristina Olaverri-Monreal

Abstract: Hybrid on/off-blockchain vehicle data management approaches have received a lot of attention in recent years. However, there are various technical challenges remained to deal with. In this paper we relied on real-world data from Austria to investigate the effects of connectivity on the transport of personal protective equipment. We proposed a three-step mechanism to process, simulate, and store/visualize aggregated vehicle datasets together with a formal pipeline process workflow model. To this end, we implemented a hybrid blockchain platform based on the hyperledger fabric and Gluster file systems. The obtained results demonstrated efficiency and stability for both hyperledger fabric and gluster file system, ability of the both on/off-blockchain mechanisms to meet the platform’s quality of service requirements.

Short description: The Intel4EC workshop aims to bring together researchers, developers and practitioners from academia and industry to present their experiences, results and research progress covering architectural designs, methods and applications of AI/ML-enabled Edge-Cloud operations and services. By bringing together these research topics, Intel4EC looks forward to help the community define open standards, AI/ML benchmarks that contribute to experiment reproducibility and systematize the complete management pipeline for a myriad of Cloud-Edge operations.

Organisers: Nishant Saurabh, Zhiming Zhao and Dragi Kimovski

Website and call for papers: https://www.intel4ec-workshop.nl/

An EU funding programme enabling researchers to set up their own interdisciplinary research networks in Europe and beyond. #COSTactions

 

From 16 to 19 May 2022, general Co-Chair, Radu Prodan, successfully presented the relevance of Cloud-Edge-IoT at CCGrid2022 conference in Taormina, Italy.

Find out more here.

This years´s “Lange Nacht der Forschung” took place on May 20, 2022. The LNDF is Austria´s most significant national research event to present the accomplishments to the broad public. ITEC was represented by three stations and involved in the station of Computer Games and Engineering, and it was a fantastic experience for everyone! We tried to make our research easily understandable for everyone.

 

Prof. Radu Prodan

Radu Prodan talks about “Democratic Trustworthy News in the Social Continuum: the ARTICONF Appeoach” Live streaming at the Digital Day in Messina.