Optimizing Cross-Platform Data Movement
- Data analytics are moving beyond the limits of a single data processing platform. A cross-platform query optimizer is necessary to enable applications to run their tasks over multiple platforms efficiently and in a platform-agnostic manner. For the optimizer to be effective, it must consider data movement costs across different data processing platforms. In this paper, we present the graph-based data movement strategy used by RHEEM, our open-source cross-platform system. In particular, we (i) model the data movement problem as a new graph problem, which we prove to be NP-hard, and (ii) propose a novel graph exploration algorithm, which allows RHEEM to discover multiple hidden opportunities for cross-platform data processing.
| Verfasserangaben: | Sebastian KruseORCiDGND, Zoi Kaoudi, Jorge-Arnulfo Quiane-RuizORCiD, Sanjay ChawlaORCiDGND, Felix NaumannORCiDGND, Bertty Contreras-Rojas |
|---|---|
| DOI: | https://doi.org/10.1109/ICDE.2019.00162 |
| ISBN: | 978-1-5386-7474-1 |
| ISBN: | 978-1-5386-7475-8 |
| ISSN: | 1084-4627 |
| ISSN: | 1063-6382 |
| Titel des übergeordneten Werks (Englisch): | 2019 IEEE 35th International Conference on Data Engineering (ICDE) |
| Verlag: | IEEE |
| Verlagsort: | New York |
| Publikationstyp: | Sonstiges |
| Sprache: | Englisch |
| Jahr der Erstveröffentlichung: | 2019 |
| Erscheinungsjahr: | 2019 |
| Datum der Freischaltung: | 05.05.2021 |
| Seitenanzahl: | 4 |
| Erste Seite: | 1642 |
| Letzte Seite: | 1645 |
| Organisationseinheiten: | Digital Engineering Fakultät / Hasso-Plattner-Institut für Digital Engineering GmbH |
| DDC-Klassifikation: | 0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 000 Informatik, Informationswissenschaft, allgemeine Werke |
| Peer Review: | Referiert |
