TY - GEN A1 - Konak, Orhan A1 - Wegner, Pit A1 - Arnrich, Bert T1 - IMU-Based Movement Trajectory Heatmaps for Human Activity Recognition T2 - Postprints der Universität Potsdam : Reihe der Digital Engineering Fakultät N2 - Recent trends in ubiquitous computing have led to a proliferation of studies that focus on human activity recognition (HAR) utilizing inertial sensor data that consist of acceleration, orientation and angular velocity. However, the performances of such approaches are limited by the amount of annotated training data, especially in fields where annotating data is highly time-consuming and requires specialized professionals, such as in healthcare. In image classification, this limitation has been mitigated by powerful oversampling techniques such as data augmentation. Using this technique, this work evaluates to what extent transforming inertial sensor data into movement trajectories and into 2D heatmap images can be advantageous for HAR when data are scarce. A convolutional long short-term memory (ConvLSTM) network that incorporates spatiotemporal correlations was used to classify the heatmap images. Evaluation was carried out on Deep Inertial Poser (DIP), a known dataset composed of inertial sensor data. The results obtained suggest that for datasets with large numbers of subjects, using state-of-the-art methods remains the best alternative. However, a performance advantage was achieved for small datasets, which is usually the case in healthcare. Moreover, movement trajectories provide a visual representation of human activities, which can help researchers to better interpret and analyze motion patterns. T3 - Zweitveröffentlichungen der Universität Potsdam : Reihe der Digital Engineering Fakultät - 4 KW - human activity recognition KW - image processing KW - machine learning KW - sensor data Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus4-487799 IS - 4 ER - TY - JOUR A1 - Konak, Orhan A1 - Wegner, Pit A1 - Arnrich, Bert T1 - IMU-Based Movement Trajectory Heatmaps for Human Activity Recognition JF - Sensors N2 - Recent trends in ubiquitous computing have led to a proliferation of studies that focus on human activity recognition (HAR) utilizing inertial sensor data that consist of acceleration, orientation and angular velocity. However, the performances of such approaches are limited by the amount of annotated training data, especially in fields where annotating data is highly time-consuming and requires specialized professionals, such as in healthcare. In image classification, this limitation has been mitigated by powerful oversampling techniques such as data augmentation. Using this technique, this work evaluates to what extent transforming inertial sensor data into movement trajectories and into 2D heatmap images can be advantageous for HAR when data are scarce. A convolutional long short-term memory (ConvLSTM) network that incorporates spatiotemporal correlations was used to classify the heatmap images. Evaluation was carried out on Deep Inertial Poser (DIP), a known dataset composed of inertial sensor data. The results obtained suggest that for datasets with large numbers of subjects, using state-of-the-art methods remains the best alternative. However, a performance advantage was achieved for small datasets, which is usually the case in healthcare. Moreover, movement trajectories provide a visual representation of human activities, which can help researchers to better interpret and analyze motion patterns. KW - human activity recognition KW - image processing KW - machine learning KW - sensor data Y1 - 2020 U6 - https://doi.org/10.3390/s20247179 SN - 1424-8220 VL - 20 IS - 24 PB - MDPI CY - Basel ER - TY - JOUR A1 - Konak, Orhan A1 - van de Water, Robin A1 - Döring, Valentin A1 - Fiedler, Tobias A1 - Liebe, Lucas A1 - Masopust, Leander A1 - Postnov, Kirill A1 - Sauerwald, Franz A1 - Treykorn, Felix A1 - Wischmann, Alexander A1 - Gjoreski, Hristijan A1 - Luštrek, Mitja A1 - Arnrich, Bert T1 - HARE BT - unifying the human activity recognition engineering workflow JF - Sensors N2 - Sensor-based human activity recognition is becoming ever more prevalent. The increasing importance of distinguishing human movements, particularly in healthcare, coincides with the advent of increasingly compact sensors. A complex sequence of individual steps currently characterizes the activity recognition pipeline. It involves separate data collection, preparation, and processing steps, resulting in a heterogeneous and fragmented process. To address these challenges, we present a comprehensive framework, HARE, which seamlessly integrates all necessary steps. HARE offers synchronized data collection and labeling, integrated pose estimation for data anonymization, a multimodal classification approach, and a novel method for determining optimal sensor placement to enhance classification results. Additionally, our framework incorporates real-time activity recognition with on-device model adaptation capabilities. To validate the effectiveness of our framework, we conducted extensive evaluations using diverse datasets, including our own collected dataset focusing on nursing activities. Our results show that HARE’s multimodal and on-device trained model outperforms conventional single-modal and offline variants. Furthermore, our vision-based approach for optimal sensor placement yields comparable results to the trained model. Our work advances the field of sensor-based human activity recognition by introducing a comprehensive framework that streamlines data collection and classification while offering a novel method for determining optimal sensor placement. KW - human activity recognition KW - multimodal classification KW - privacy preservation KW - real-time classification KW - sensor placement Y1 - 2023 U6 - https://doi.org/10.3390/s23239571 SN - 1424-8220 VL - 23 IS - 23 PB - MDPI CY - Basel ER -