Skip to Main content Skip to Navigation
Conference papers

A Multi-Metric Adaptive Stream Processing System

Abstract : Stream processing systems (SPS) have to deal with highly dynamic scenarios where its adaptation is mandatory in order to accomplish realistic applications requirements. In this work, we propose a new adaptive SPS for real-time processing that, based on input data rate variation, dynamically adapts the number of active operator replicas. Our SPS extends Storm by pre-allocating, for each operator, a set of inactive replicas which are activated (or deactivated) when necessary without the Storm reconfiguration cost. We exploit the MAPE model and define a new metric that aggregates the value of multiple metrics to dynamically changes the number of replicas of an operator. We deploy our SPS over Google Cloud Platform and results confirm that our metric can tolerate highly dynamic conditions, improving resource usage while preserving high throughput and low latency.
Complete list of metadata

https://hal.inria.fr/hal-03516376
Contributor : Pierre Sens Connect in order to contact the contributor
Submitted on : Friday, January 7, 2022 - 10:42:45 AM
Last modification on : Wednesday, June 8, 2022 - 12:50:07 PM
Long-term archiving on: : Friday, April 8, 2022 - 6:35:28 PM

File

Paper___A_Multi_Metric_Adaptiv...
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03516376, version 1

Citation

Daniel Wladdimiro, Luciana Arantes, Pierre Sens, Nicolas Hidalgo. A Multi-Metric Adaptive Stream Processing System. NCA 2021 - 20th IEEE International Symposium on Network Computing and Applications, Nov 2021, Cambridge, Boston, United States. ⟨hal-03516376⟩

Share

Metrics

Record views

38

Files downloads

67