TY  - JOUR
A1  - Reimann, Max
A1  - Klingbeil, Mandy
A1  - Pasewaldt, Sebastian
A1  - Semmo, Amir
A1  - Trapp, Matthias
A1  - Döllner, Jürgen Roland Friedrich
T1  - Locally controllable neural style transfer on mobile devices
JF  - The Visual Computer
N2  - Mobile expressive rendering gained increasing popularity among users seeking casual creativity by image stylization and supports the development of mobile artists as a new user group. In particular, neural style transfer has advanced as a core technology to emulate characteristics of manifold artistic styles. However, when it comes to creative expression, the technology still faces inherent limitations in providing low-level controls for localized image stylization. In this work, we first propose a problem characterization of interactive style transfer representing a trade-off between visual quality, run-time performance, and user control. We then present MaeSTrO, a mobile app for orchestration of neural style transfer techniques using iterative, multi-style generative and adaptive neural networks that can be locally controlled by on-screen painting metaphors. At this, we enhance state-of-the-art neural style transfer techniques by mask-based loss terms that can be interactively parameterized by a generalized user interface to facilitate a creative and localized editing process. We report on a usability study and an online survey that demonstrate the ability of our app to transfer styles at improved semantic plausibility.
KW  - Non-photorealistic rendering
KW  - Style transfer
KW  - Neural networks
KW  - Mobile devices
KW  - Interactive control
KW  - Expressive rendering
Y1  - 2019
U6  - https://doi.org/10.1007/s00371-019-01654-1
SN  - 0178-2789
SN  - 1432-2315
VL  - 35
IS  - 11
SP  - 1531
EP  - 1547
PB  - Springer
CY  - New York
ER  - 
TY  - JOUR
A1  - Stojanovic, Vladeta
A1  - Trapp, Matthias
A1  - Richter, Rico
A1  - Döllner, Jürgen Roland Friedrich
T1  - Service-oriented semantic enrichment of indoor point clouds using octree-based multiview classification
JF  - Graphical Models
N2  - The use of Building Information Modeling (BIM) for Facility Management (FM) in the Operation and Maintenance (O&M) stages of the building life-cycle is intended to bridge the gap between operations and digital data, but lacks the functionality of assessing the state of the built environment due to non-automated generation of associated semantics. 3D point clouds can be used to capture the physical state of the built environment, but also lack these associated semantics. A prototypical implementation of a service-oriented architecture for classification of indoor point cloud scenes of office environments is presented, using multiview classification. The multiview classification approach is tested using a retrained Convolutional Neural Network (CNN) model - Inception V3. The presented approach for classifying common office furniture objects (chairs, sofas and desks), contained in 3D point cloud scans, is tested and evaluated. The results show that the presented approach can classify common office furniture up to an acceptable degree of accuracy, and is suitable for quick and robust semantics approximation - based on RGB (red, green and blue color channel) cubemap images of the octree partitioned areas of the 3D point cloud scan. Additional methods for web-based 3D visualization, editing and annotation of point clouds are also discussed. Using the described approach, captured scans of indoor environments can be semantically enriched using object annotations derived from multiview classification results. Furthermore, the presented approach is suited for semantic enrichment of lower resolution indoor point clouds acquired using commodity mobile devices.
KW  - Semantic enrichment
KW  - 3D point clouds
KW  - Multiview classification
KW  - Service-oriented
KW  - Indoor environments
Y1  - 2019
U6  - https://doi.org/10.1016/j.gmod.2019.101039
SN  - 1524-0703
SN  - 1524-0711
VL  - 105
PB  - Elsevier
CY  - San Diego
ER  -