Skip to Main content Skip to Navigation
Conference papers

A Multi-Metric Adaptive Stream Processing System

Abstract : Stream processing systems (SPS) have to deal with highly dynamic scenarios where its adaptation is mandatory in order to accomplish realistic applications requirements. In this work, we propose a new adaptive SPS for real-time processing that, based on input data rate variation, dynamically adapts the number of active operator replicas. Our SPS extends Storm by pre-allocating, for each operator, a set of inactive replicas which are activated (or deactivated) when necessary without the Storm reconfiguration cost. We exploit the MAPE model and define a new metric that aggregates the value of multiple metrics to dynamically changes the number of replicas of an operator. We deploy our SPS over Google Cloud Platform and results confirm that our metric can tolerate highly dynamic conditions, improving resource usage while preserving high throughput and low latency.
Complete list of metadata
Contributor : Pierre Sens Connect in order to contact the contributor
Submitted on : Friday, January 7, 2022 - 10:42:45 AM
Last modification on : Wednesday, June 8, 2022 - 12:50:07 PM
Long-term archiving on: : Friday, April 8, 2022 - 6:35:28 PM


Files produced by the author(s)


  • HAL Id : hal-03516376, version 1


Daniel Wladdimiro, Luciana Arantes, Pierre Sens, Nicolas Hidalgo. A Multi-Metric Adaptive Stream Processing System. NCA 2021 - 20th IEEE International Symposium on Network Computing and Applications, Nov 2021, Cambridge, Boston, United States. ⟨hal-03516376⟩



Record views


Files downloads