MaeSTrO: A Mobile App for Style Transfer Orchestration using Neural Networks
- Mobile expressive rendering gained increasing popularity among users seeking casual creativity by image stylization and supports the development of mobile artists as a new user group. In particular, neural style transfer has advanced as a core technology to emulate characteristics of manifold artistic styles. However, when it comes to creative expression, the technology still faces inherent limitations in providing low-level controls for localized image stylization. This work enhances state-of-the-art neural style transfer techniques by a generalized user interface with interactive tools to facilitate a creative and localized editing process. Thereby, we first propose a problem characterization representing trade-offs between visual quality, run-time performance, and user control. We then present MaeSTrO, a mobile app for orchestration of neural style transfer techniques using iterative, multi-style generative and adaptive neural networks that can be locally controlled by on-screen painting metaphors. At this, first user testsMobile expressive rendering gained increasing popularity among users seeking casual creativity by image stylization and supports the development of mobile artists as a new user group. In particular, neural style transfer has advanced as a core technology to emulate characteristics of manifold artistic styles. However, when it comes to creative expression, the technology still faces inherent limitations in providing low-level controls for localized image stylization. This work enhances state-of-the-art neural style transfer techniques by a generalized user interface with interactive tools to facilitate a creative and localized editing process. Thereby, we first propose a problem characterization representing trade-offs between visual quality, run-time performance, and user control. We then present MaeSTrO, a mobile app for orchestration of neural style transfer techniques using iterative, multi-style generative and adaptive neural networks that can be locally controlled by on-screen painting metaphors. At this, first user tests indicate different levels of satisfaction for the implemented techniques and interaction design.…
Author details: | Max ReimannORCiD, Mandy Klingbeil, Sebastian Pasewaldt, Amir SemmoGND, Matthias TrappORCiDGND, Jürgen Roland Friedrich DöllnerORCiDGND |
---|---|
DOI: | https://doi.org/10.1109/CW.2018.00016 |
ISBN: | 978-1-5386-7315-7 |
Title of parent work (English): | International Conference on Cyberworlds (CW) |
Publisher: | IEEE |
Place of publishing: | New York |
Editor(s): | A Sourina Sourin |
Publication type: | Other |
Language: | English |
Date of first publication: | 2018/12/27 |
Publication year: | 2018 |
Release date: | 2022/02/22 |
Tag: | non-photorealistic rendering; style transfer |
Number of pages: | 8 |
First page: | 9 |
Last Page: | 16 |
Funding institution: | Federal Ministry of Education and Research (BMBF), GermanyFederal Ministry of Education & Research (BMBF) [01IS15041] |
Organizational units: | Digital Engineering Fakultät / Hasso-Plattner-Institut für Digital Engineering GmbH |
DDC classification: | 0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 000 Informatik, Informationswissenschaft, allgemeine Werke |
Peer review: | Referiert |