TY - JOUR A1 - Shekhar, Sumit A1 - Reimann, Max A1 - Mayer, Maximilian A1 - Semmo, Amir A1 - Pasewaldt, Sebastian A1 - Döllner, Jürgen A1 - Trapp, Matthias T1 - Interactive photo editing on smartphones via intrinsic decomposition JF - Computer graphics forum : journal of the European Association for Computer Graphics N2 - Intrinsic decomposition refers to the problem of estimating scene characteristics, such as albedo and shading, when one view or multiple views of a scene are provided. The inverse problem setting, where multiple unknowns are solved given a single known pixel-value, is highly under-constrained. When provided with correlating image and depth data, intrinsic scene decomposition can be facilitated using depth-based priors, which nowadays is easy to acquire with high-end smartphones by utilizing their depth sensors. In this work, we present a system for intrinsic decomposition of RGB-D images on smartphones and the algorithmic as well as design choices therein. Unlike state-of-the-art methods that assume only diffuse reflectance, we consider both diffuse and specular pixels. For this purpose, we present a novel specularity extraction algorithm based on a multi-scale intensity decomposition and chroma inpainting. At this, the diffuse component is further decomposed into albedo and shading components. We use an inertial proximal algorithm for non-convex optimization (iPiano) to ensure albedo sparsity. Our GPU-based visual processing is implemented on iOS via the Metal API and enables interactive performance on an iPhone 11 Pro. Further, a qualitative evaluation shows that we are able to obtain high-quality outputs. Furthermore, our proposed approach for specularity removal outperforms state-of-the-art approaches for real-world images, while our albedo and shading layer decomposition is faster than the prior work at a comparable output quality. Manifold applications such as recoloring, retexturing, relighting, appearance editing, and stylization are shown, each using the intrinsic layers obtained with our method and/or the corresponding depth data. KW - CCS Concepts KW - center dot Computing KW - methodologie KW - Image-based rendering KW - Image KW - processing KW - Computational photography Y1 - 2021 U6 - https://doi.org/10.1111/cgf.142650 SN - 0167-7055 SN - 1467-8659 VL - 40 SP - 497 EP - 510 PB - Blackwell CY - Oxford ER - TY - JOUR A1 - Reimann, Max A1 - Buchheim, Benito A1 - Semmo, Amir A1 - Döllner, Jürgen A1 - Trapp, Matthias T1 - Controlling strokes in fast neural style transfer using content transforms JF - The Visual Computer N2 - Fast style transfer methods have recently gained popularity in art-related applications as they make a generalized real-time stylization of images practicable. However, they are mostly limited to one-shot stylizations concerning the interactive adjustment of style elements. In particular, the expressive control over stroke sizes or stroke orientations remains an open challenge. To this end, we propose a novel stroke-adjustable fast style transfer network that enables simultaneous control over the stroke size and intensity, and allows a wider range of expressive editing than current approaches by utilizing the scale-variance of convolutional neural networks. Furthermore, we introduce a network-agnostic approach for style-element editing by applying reversible input transformations that can adjust strokes in the stylized output. At this, stroke orientations can be adjusted, and warping-based effects can be applied to stylistic elements, such as swirls or waves. To demonstrate the real-world applicability of our approach, we present StyleTune, a mobile app for interactive editing of neural style transfers at multiple levels of control. Our app allows stroke adjustments on a global and local level. It furthermore implements an on-device patch-based upsampling step that enables users to achieve results with high output fidelity and resolutions of more than 20 megapixels. Our approach allows users to art-direct their creations and achieve results that are not possible with current style transfer applications. Y1 - 2022 U6 - https://doi.org/10.1007/s00371-022-02518-x SN - 0178-2789 SN - 1432-2315 VL - 38 IS - 12 SP - 4019 EP - 4033 PB - Springer CY - New York ER - TY - JOUR A1 - Pasewaldt, Sebastian A1 - Semmo, Amir A1 - Trapp, Matthias A1 - Döllner, Jürgen T1 - Multi-perspective 3D panoramas JF - International journal of geographical information science N2 - This article presents multi-perspective 3D panoramas that focus on visualizing 3D geovirtual environments (3D GeoVEs) for navigation and exploration tasks. Their key element, a multi-perspective view (MPV), seamlessly combines what is seen from multiple viewpoints into a single image. This approach facilitates the presentation of information for virtual 3D city and landscape models, particularly by reducing occlusions, increasing screen-space utilization, and providing additional context within a single image. We complement MPVs with cartographic visualization techniques to stylize features according to their semantics and highlight important or prioritized information. When combined, both techniques constitute the core implementation of interactive, multi-perspective 3D panoramas. They offer a large number of effective means for visual communication of 3D spatial information, a high degree of customization with respect to cartographic design, and manifold applications in different domains. We discuss design decisions of 3D panoramas for the exploration of and navigation in 3D GeoVEs. We also discuss a preliminary user study that indicates that 3D panoramas are a promising approach for navigation systems using 3D GeoVEs. KW - multi-perspective visualization KW - panorama KW - focus plus context visualization KW - 3D geovirtual environments KW - cartographic design Y1 - 2014 U6 - https://doi.org/10.1080/13658816.2014.922686 SN - 1365-8816 SN - 1362-3087 VL - 28 IS - 10 SP - 2030 EP - 2051 PB - Routledge, Taylor & Francis Group CY - Abingdon ER - TY - JOUR A1 - Hildebrandt, Dieter A1 - Döllner, Jürgen T1 - Service-oriented, standards-based 3D geovisualization : potential and challenges N2 - The application of the architectural concept of service-oriented architectures (SOA) in combination with open standards when building distributed, 3D geovisualization systems offers the potential to cover and take advantage of the opportunities and demands created by the rise of ubiquitous computer networks and the Internet as well as to overcome prevalent interoperability barriers. In this paper, based on a literature study and our own experiences, we discuss the potential and challenges that arise when building standards-based, distributed systems according to the SOA paradigm for 3D geovisualization, with a particular focus on 3D geovirtual environments and virtual 3D city models. First, we briefly introduce fundamentals of the SOA paradigm, identify requirements for service-oriented 3D geovisualization systems, and present an architectural framework that relates SOA concepts, geovisualization concepts, and standardization proposals by the Open Geospatial Consortium in a common frame of reference. Next, we discuss the potential and challenges driven by the SOA paradigm on four different levels of abstraction, namely service fundamentals, service composition, interaction services, performance, and overarching aspects, and we discuss those driven by standardization. We further exemplify and substantiate the discussion in the scope of a case study and the image-based provisioning of and interaction with visual representations of remote virtual 3D city models. Y1 - 2010 UR - http://www.sciencedirect.com/science/journal/01989715 U6 - https://doi.org/10.1016/j.compenvurbsys.2010.05.003 SN - 0198-9715 ER -