TY  - GEN
A1  - Kruse, Sebastian
A1  - Kaoudi, Zoi
A1  - Quiane-Ruiz, Jorge-Arnulfo
A1  - Chawla, Sanjay
A1  - Naumann, Felix
A1  - Contreras-Rojas, Bertty
T1  - Optimizing Cross-Platform Data Movement
T2  - 2019 IEEE 35th International Conference on Data Engineering (ICDE)
N2  - Data analytics are moving beyond the limits of a single data processing platform. A cross-platform query optimizer is necessary to enable applications to run their tasks over multiple platforms efficiently and in a platform-agnostic manner. For the optimizer to be effective, it must consider data movement costs across different data processing platforms. In this paper, we present the graph-based data movement strategy used by RHEEM, our open-source cross-platform system. In particular, we (i) model the data movement problem as a new graph problem, which we prove to be NP-hard, and (ii) propose a novel graph exploration algorithm, which allows RHEEM to discover multiple hidden opportunities for cross-platform data processing.
Y1  - 2019
SN  - 978-1-5386-7474-1
SN  - 978-1-5386-7475-8
U6  - https://doi.org/10.1109/ICDE.2019.00162
SN  - 1084-4627
SN  - 1063-6382
SP  - 1642
EP  - 1645
PB  - IEEE
CY  - New York
ER  -