TY - GEN A1 - Schneider, Oliver A1 - Shigeyama, Jotaro A1 - Kovacs, Robert A1 - Roumen, Thijs Jan A1 - Marwecki, Sebastian A1 - Böckhoff, Nico A1 - Glöckner, Daniel Amadeus Johannes A1 - Bounama, Jonas A1 - Baudisch, Patrick T1 - DualPanto BT - a haptic device that enables blind users to continuously interact with virtual worlds T2 - UIST '18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology N2 - We present a new haptic device that enables blind users to continuously track the absolute position of moving objects in spatial virtual environments, as is the case in sports or shooter games. Users interact with DualPanto by operating the me handle with one hand and by holding on to the it handle with the other hand. Each handle is connected to a pantograph haptic input/output device. The key feature is that the two handles are spatially registered with respect to each other. When guiding their avatar through a virtual world using the me handle, spatial registration enables users to track moving objects by having the device guide the output hand. This allows blind players of a 1-on-1 soccer game to race for the ball or evade an opponent; it allows blind players of a shooter game to aim at an opponent and dodge shots. In our user study, blind participants reported very high enjoyment when using the device to play (6.5/7). KW - Haptics KW - force-feedback KW - accessibility KW - blind KW - visually impaired KW - gaming Y1 - 2018 SN - 978-1-4503-5948-1 U6 - https://doi.org/10.1145/3242587.3242604 SP - 877 EP - 887 PB - Association for Computing Machinery CY - New York ER - TY - JOUR A1 - Seemüller, Anna A1 - Fiehler, Katja A1 - Rösler, Frank T1 - Unimodal and crossmodal working memory representations of visual and kinesthetic movement trajectories JF - Acta psychologica : international journal of psychonomics N2 - The present study investigated whether visual and kinesthetic stimuli are stored as multisensory or modality-specific representations in unimodal and crossmodal working memory tasks. To this end, angle-shaped movement trajectories were presented to 16 subjects in delayed matching-to-sample tasks either visually or kinesthetically during encoding and recognition. During the retention interval, a secondary visual or kinesthetic interference task was inserted either immediately or with a delay after encoding. The modality of the interference task interacted significantly with the encoding modality. After visual encoding, memory was more impaired by a visual than by a kinesthetic secondary task, while after kinesthetic encoding the pattern was reversed. The time when the secondary task had to be performed interacted with the encoding modality as well. For visual encoding, memory was more impaired, when the secondary task had to be performed at the beginning of the retention interval. In contrast, memory after kinesthetic encoding was more affected, when the secondary task was introduced later in the retention interval. The findings suggest that working memory traces are maintained in a modality-specific format characterized by distinct consolidation processes that take longer after kinesthetic than after visual encoding. KW - Visual representations KW - Kinesthetic representations KW - Vision KW - Haptics KW - Dual-task interference Y1 - 2011 U6 - https://doi.org/10.1016/j.actpsy.2010.09.014 SN - 0001-6918 VL - 136 IS - 1 SP - 52 EP - 59 PB - Elsevier CY - Amsterdam ER - TY - JOUR A1 - Seemüller, Anna A1 - Müller, E. M. A1 - Rösler, Frank T1 - EEG-power and -coherence changes in a unimodal and a crossmodal working memory task with visual and kinesthetic stimuli JF - International journal of psychophysiology N2 - We investigated EEG-power and EEG-coherence changes in a unimodal and a crossmodal matching-to-sample working memory task with either visual or kinesthetic stimuli. Angle-shaped trajectories were used as stimuli presented either as a moving dot on a screen or as a passive movement of a haptic device. Effects were evaluated during the different phases of encoding, maintenance, and recognition. Alpha power was modulated during encoding by the stimulus modality, and in crossmodal conditions during encoding and maintenance by the expected modality of the upcoming test stimulus. These power modulations were observed over modality-specific cortex regions. Systematic changes of coherence for crossmodal compared to unimodal tasks were not observed during encoding and maintenance but only during recognition. There, coherence in the theta-band increased between electrode sites over left central and occipital cortex areas in the crossmodal compared to the unimodal conditions. The results underline the importance of modality-specific representations and processes in unimodal and crossmodal working memory tasks. Crossmodal recognition of visually and kinesthetically presented object features seems to be related to a direct interaction of somatosensory/motor and visual cortex regions by means of long-range synchronization in the theta-band and such interactions seem to take place at the beginning of the recognition phase, i.e. when crossmodal transfer is actually necessary. KW - Visual representations KW - Kinesthetic representations KW - Synchronization KW - Crossmodal matching KW - Haptics Y1 - 2012 U6 - https://doi.org/10.1016/j.ijpsycho.2011.10.009 SN - 0167-8760 VL - 83 IS - 1 SP - 87 EP - 95 PB - Elsevier CY - Amsterdam ER -