TY - JOUR A1 - Stober, Sebastian A1 - Sternin, Avital T1 - Decoding music perception and imagination using deep-learning techniques JF - Signal processing and machine learning for brain-machine interfaces N2 - Deep learning is a sub-field of machine learning that has recently gained substantial popularity in various domains such as computer vision, automatic speech recognition, natural language processing, and bioinformatics. Deep-learning techniques are able to learn complex feature representations from raw signals and thus also have potential to improve signal processing in the context of brain-computer interfaces (BCIs). However, they typically require large amounts of data for training - much more than what can often be provided with reasonable effort when working with brain activity recordings of any kind. In order to still leverage the power of deep-learning techniques with limited available data, special care needs to be taken when designing the BCI task, defining the structure of the deep model, and choosing the training method. This chapter presents example approaches for the specific scenario of music-based brain-computer interaction through electroencephalography - in the hope that these will prove to be valuable in different settings as well. We explain important decisions for the design of the BCI task and their impact on the models and training techniques that can be used. Furthermore, we present and compare various pre-training techniques that aim to improve the signal-to-noise ratio. Finally, we discuss approaches to interpret the trained models. Y1 - 2018 SN - 978-1-78561-399-9 SN - 978-1-78561-398-2 U6 - https://doi.org/10.1049/PBCE114E VL - 114 SP - 271 EP - 299 PB - Institution of Engineering and Technology CY - London ER - TY - GEN A1 - Low, Thomas A1 - Hentschel, Christian A1 - Stober, Sebastian A1 - Sack, Harald A1 - Nürnberger, Andreas ED - Amsaleg, Laurent ED - Guðmundsson, Gylfi Þór ED - Gurrin, Cathal ED - Jónsson, Björn Þór ED - Satoh, Shin'ichi T1 - Exploring large movie collections BT - comparing visual berrypicking and traditional browsing T2 - Lecture notes in computer science N2 - We compare Visual Berrypicking, an interactive approach allowing users to explore large and highly faceted information spaces using similarity-based two-dimensional maps, with traditional browsing techniques. For large datasets, current projection methods used to generate maplike overviews suffer from increased computational costs and a loss of accuracy resulting in inconsistent visualizations. We propose to interactively align inexpensive small maps, showing local neighborhoods only, which ideally creates the impression of panning a large map. For evaluation, we designed a web-based prototype for movie exploration and compared it to the web interface of The Movie Database (TMDb) in an online user study. Results suggest that users are able to effectively explore large movie collections by hopping from one neighborhood to the next. Additionally, due to the projection of movie similarities, interesting links between movies can be found more easily, and thus, compared to browsing serendipitous discoveries are more likely. KW - Exploratory interfaces KW - Media retrieval KW - Multidimensional scaling KW - User study Y1 - 2016 SN - 978-3-319-51814-5 SN - 978-3-319-51813-8 U6 - https://doi.org/10.1007/978-3-319-51814-5_17 SN - 0302-9743 SN - 1611-3349 VL - 10133 SP - 198 EP - 208 PB - Springer CY - Cham ER - TY - GEN A1 - Ofner, Andre A1 - Stober, Sebastian T1 - Hybrid variational predictive coding as a bridge between human and artificial cognition T2 - ALIFE 2019: The 2019 Conference on Artificial Life N2 - Predictive coding and its generalization to active inference offer a unified theory of brain function. The underlying predictive processing paradigmhas gained significant attention in artificial intelligence research for its representation learning and predictive capacity. Here, we suggest that it is possible to integrate human and artificial generative models with a predictive coding network that processes sensations simultaneously with the signature of predictive coding found in human neuroimaging data. We propose a recurrent hierarchical predictive coding model that predicts low-dimensional representations of stimuli, electroencephalogram and physiological signals with variational inference. We suggest that in a shared environment, such hybrid predictive coding networks learn to incorporate the human predictive model in order to reduce prediction error. We evaluate the model on a publicly available EEG dataset of subjects watching one-minute long video excerpts. Our initial results indicate that the model can be trained to predict visual properties such as the amount, distance and motion of human subjects in videos. Y1 - 2019 SP - 68 EP - 69 PB - MIT Press CY - Cambridge ER - TY - JOUR A1 - Stober, Sebastian T1 - Model-based frameworks for user adapted information exploration BT - an overview JF - Companion technology : a paradigm shift in human-technology interaction Y1 - 2017 SN - 978-3-319-43664-7 VL - Cham SP - 37 EP - 56 PB - Springer ER - TY - JOUR A1 - Bridwell, David A. A1 - Cavanagh, James F. A1 - Collins, Anne G. E. A1 - Nunez, Michael D. A1 - Srinivasan, Ramesh A1 - Stober, Sebastian A1 - Calhoun, Vince D. T1 - Moving Beyond ERP Components BT - a selective review of approaches to integrate EEG and behavior JF - Frontiers in human neuroscienc N2 - Relationships between neuroimaging measures and behavior provide important clues about brain function and cognition in healthy and clinical populations. While electroencephalography (EEG) provides a portable, low cost measure of brain dynamics, it has been somewhat underrepresented in the emerging field of model-based inference. We seek to address this gap in this article by highlighting the utility of linking EEG and behavior, with an emphasis on approaches for EEG analysis that move beyond focusing on peaks or "components" derived from averaging EEG responses across trials and subjects (generating the event-related potential, ERP). First, we review methods for deriving features from EEG in order to enhance the signal within single-trials. These methods include filtering based on user-defined features (i.e., frequency decomposition, time-frequency decomposition), filtering based on data-driven properties (i.e., blind source separation, BSS), and generating more abstract representations of data (e.g., using deep learning). We then review cognitive models which extract latent variables from experimental tasks, including the drift diffusion model (DDM) and reinforcement learning (RL) approaches. Next, we discuss ways to access associations among these measures, including statistical models, data-driven joint models and cognitive joint modeling using hierarchical Bayesian models (HBMs). We think that these methodological tools are likely to contribute to theoretical advancements, and will help inform our understandings of brain dynamics that contribute to moment-to-moment cognitive function. KW - EEG KW - ERP KW - blind source separation KW - partial least squares KW - canonical correlations analysis KW - representational similarity analysis KW - deep learning KW - hierarchical Bayesian model Y1 - 2018 U6 - https://doi.org/10.3389/fnhum.2018.00106 SN - 1662-5161 VL - 12 PB - Frontiers Research Foundation CY - Lausanne ER - TY - GEN A1 - Bridwell, David A. A1 - Cavanagh, James F. A1 - Collins, Anne G. E. A1 - Nunez, Michael D. A1 - Srinivasan, Ramesh A1 - Stober, Sebastian A1 - Calhoun, Vince D. T1 - Moving beyond ERP components BT - a selective review of approaches to integrate EEG and behavior T2 - Postprints der Universität Potsdam : Humanwissenschaftliche Reihe N2 - Relationships between neuroimaging measures and behavior provide important clues about brain function and cognition in healthy and clinical populations. While electroencephalography (EEG) provides a portable, low cost measure of brain dynamics, it has been somewhat underrepresented in the emerging field of model-based inference. We seek to address this gap in this article by highlighting the utility of linking EEG and behavior, with an emphasis on approaches for EEG analysis that move beyond focusing on peaks or "components" derived from averaging EEG responses across trials and subjects (generating the event-related potential, ERP). First, we review methods for deriving features from EEG in order to enhance the signal within single-trials. These methods include filtering based on user-defined features (i.e., frequency decomposition, time-frequency decomposition), filtering based on data-driven properties (i.e., blind source separation, BSS), and generating more abstract representations of data (e.g., using deep learning). We then review cognitive models which extract latent variables from experimental tasks, including the drift diffusion model (DDM) and reinforcement learning (RL) approaches. Next, we discuss ways to access associations among these measures, including statistical models, data-driven joint models and cognitive joint modeling using hierarchical Bayesian models (HBMs). We think that these methodological tools are likely to contribute to theoretical advancements, and will help inform our understandings of brain dynamics that contribute to moment-to-moment cognitive function. T3 - Zweitveröffentlichungen der Universität Potsdam : Humanwissenschaftliche Reihe - 656 KW - EEG KW - ERP KW - blind source separation KW - partial least squares KW - canonical correlations analysis KW - representational similarity analysis KW - deep learning KW - hierarchical Bayesian model Y1 - 2020 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus4-459667 SN - 1866-8364 IS - 656 ER - TY - JOUR A1 - Stober, Sebastian T1 - Toward Studying Music Cognition with Information Retrieval Techniques BT - Lessons Learned from the OpenMIIR Initiative JF - Frontiers in psychology N2 - As an emerging sub-field of music information retrieval (MIR), music imagery information retrieval (MIIR) aims to retrieve information from brain activity recorded during music cognition–such as listening to or imagining music pieces. This is a highly inter-disciplinary endeavor that requires expertise in MIR as well as cognitive neuroscience and psychology. The OpenMIIR initiative strives to foster collaborations between these fields to advance the state of the art in MIIR. As a first step, electroencephalography (EEG) recordings of music perception and imagination have been made publicly available, enabling MIR researchers to easily test and adapt their existing approaches for music analysis like fingerprinting, beat tracking or tempo estimation on this new kind of data. This paper reports on first results of MIIR experiments using these OpenMIIR datasets and points out how these findings could drive new research in cognitive neuroscience. KW - music cognition KW - music perception KW - music information retrieval KW - deep learning KW - representation learning Y1 - 2017 U6 - https://doi.org/10.3389/fpsyg.2017.01255 SN - 1664-1078 VL - 8 PB - Frontiers Research Foundation CY - Lausanne ER - TY - GEN A1 - Stober, Sebastian T1 - Toward Studying Music Cognition with Information Retrieval Techniques BT - Lessons Learned from the OpenMIIR Initiative N2 - As an emerging sub-field of music information retrieval (MIR), music imagery information retrieval (MIIR) aims to retrieve information from brain activity recorded during music cognition–such as listening to or imagining music pieces. This is a highly inter-disciplinary endeavor that requires expertise in MIR as well as cognitive neuroscience and psychology. The OpenMIIR initiative strives to foster collaborations between these fields to advance the state of the art in MIIR. As a first step, electroencephalography (EEG) recordings of music perception and imagination have been made publicly available, enabling MIR researchers to easily test and adapt their existing approaches for music analysis like fingerprinting, beat tracking or tempo estimation on this new kind of data. This paper reports on first results of MIIR experiments using these OpenMIIR datasets and points out how these findings could drive new research in cognitive neuroscience. T3 - Zweitveröffentlichungen der Universität Potsdam : Humanwissenschaftliche Reihe - 347 KW - deep learning KW - music cognition KW - music information retrieval KW - music perception KW - representation learning Y1 - 2017 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus4-402762 ER - TY - JOUR A1 - Stober, Sebastian T1 - Toward Studying Music Cognition with Information Retrieval Techniques: Lessons Learned from the OpenMIIR Initiative JF - Frontiers in psychology N2 - As an emerging sub-field of music information retrieval (MIR), music imagery information retrieval (MIIR) aims to retrieve information from brain activity recorded during music cognition-such as listening to or imagining music pieces. This is a highly interdisciplinary endeavor that requires expertise in MIR as well as cognitive neuroscience and psychology. The OpenMIIR initiative strives to foster collaborations between these fields to advance the state of the art in MIIR. As a first step, electroencephalography (EEG) recordings ofmusic perception and imagination have beenmade publicly available, enabling MIR researchers to easily test and adapt their existing approaches for music analysis like fingerprinting, beat tracking or tempo estimation on this new kind of data. This paper reports on first results of MIIR experiments using these OpenMIIR datasets and points out how these findings could drive new research in cognitive neuroscience. KW - music cognition KW - music perception KW - music information retrieval KW - deep learning KW - representation learning Y1 - 2017 U6 - https://doi.org/10.3389/fpsyg.2017.01255 SN - 1664-1078 VL - 8 PB - Frontiers Research Foundation CY - Lausanne ER -