TY  - JOUR
A1  - Kruse, Sebastian
A1  - Kaoudi, Zoi
A1  - Contreras-Rojas, Bertty
A1  - Chawla, Sanjay
A1  - Naumann, Felix
A1  - Quiane-Ruiz, Jorge-Arnulfo
T1  - RHEEMix in the data jungle
BT  - a cost-based optimizer for cross-platform systems
JF  - The VLDB Journal
N2  - Data analytics are moving beyond the limits of a single platform. In this paper, we present the cost-based optimizer of Rheem, an open-source cross-platform system that copes with these new requirements. The optimizer allocates the subtasks of data analytic tasks to the most suitable platforms. Our main contributions are: (i) a mechanism based on graph transformations to explore alternative execution strategies; (ii) a novel graph-based approach to determine efficient data movement plans among subtasks and platforms; and (iii) an efficient plan enumeration algorithm, based on a novel enumeration algebra. We extensively evaluate our optimizer under diverse real tasks. We show that our optimizer can perform tasks more than one order of magnitude faster when using multiple platforms than when using a single platform.
KW  - Cross-platform
KW  - Polystore
KW  - Query optimization
KW  - Data processing
Y1  - 2020
U6  - https://doi.org/10.1007/s00778-020-00612-x
SN  - 1066-8888
SN  - 0949-877X
VL  - 29
IS  - 6
SP  - 1287
EP  - 1310
PB  - Springer
CY  - Berlin
ER  - 
TY  - GEN
A1  - Kruse, Sebastian
A1  - Kaoudi, Zoi
A1  - Contreras-Rojas, Bertty
A1  - Chawla, Sanjay
A1  - Naumann, Felix
A1  - Quiané-Ruiz, Jorge-Arnulfo
T1  - RHEEMix in the data jungle
BT  - a cost-based optimizer for cross-platform systems
T2  - Zweitveröffentlichungen der Universität Potsdam : Reihe der Digital Engineering Fakultät
N2  - Data analytics are moving beyond the limits of a single platform. In this paper, we present the cost-based optimizer of Rheem, an open-source cross-platform system that copes with these new requirements. The optimizer allocates the subtasks of data analytic tasks to the most suitable platforms. Our main contributions are: (i) a mechanism based on graph transformations to explore alternative execution strategies; (ii) a novel graph-based approach to determine efficient data movement plans among subtasks and platforms; and (iii) an efficient plan enumeration algorithm, based on a novel enumeration algebra. We extensively evaluate our optimizer under diverse real tasks. We show that our optimizer can perform tasks more than one order of magnitude faster when using multiple platforms than when using a single platform.
T3  - Zweitveröffentlichungen der Universität Potsdam : Reihe der Digital Engineering Fakultät - 22 
KW  - cross-platform
KW  - polystore
KW  - query optimization
KW  - data processing
Y1  - 2020
U6  - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus4-519443
IS  - 6
ER  - 
TY  - GEN
A1  - Kruse, Sebastian
A1  - Kaoudi, Zoi
A1  - Quiane-Ruiz, Jorge-Arnulfo
A1  - Chawla, Sanjay
A1  - Naumann, Felix
A1  - Contreras-Rojas, Bertty
T1  - Optimizing Cross-Platform Data Movement
T2  - 2019 IEEE 35th International Conference on Data Engineering (ICDE)
N2  - Data analytics are moving beyond the limits of a single data processing platform. A cross-platform query optimizer is necessary to enable applications to run their tasks over multiple platforms efficiently and in a platform-agnostic manner. For the optimizer to be effective, it must consider data movement costs across different data processing platforms. In this paper, we present the graph-based data movement strategy used by RHEEM, our open-source cross-platform system. In particular, we (i) model the data movement problem as a new graph problem, which we prove to be NP-hard, and (ii) propose a novel graph exploration algorithm, which allows RHEEM to discover multiple hidden opportunities for cross-platform data processing.
Y1  - 2019
SN  - 978-1-5386-7474-1
SN  - 978-1-5386-7475-8
U6  - https://doi.org/10.1109/ICDE.2019.00162
SN  - 1084-4627
SN  - 1063-6382
SP  - 1642
EP  - 1645
PB  - IEEE
CY  - New York
ER  -