DOAJ gelistet
Refine
Has Fulltext
- no (31)
Document Type
- Article (31)
Language
- English (31)
Is part of the Bibliography
- yes (31)
Keywords
- German (3)
- exhaustivity (3)
- eye-tracking (2)
- face (2)
- Akan (1)
- Bayesian inference (1)
- Bayesian models (1)
- Bayesian parameter estimation (1)
- Cognitive emotional (1)
- Conversation Analysis (1)
Institute
- Department Linguistik (31) (remove)
The ZuCo benchmark on cross-subject reading task classification with EEG and eye-tracking data
(2023)
We present a new machine learning benchmark for reading task classification with the goal of advancing EEG and eye-tracking research at the intersection between computational language processing and cognitive neuroscience. The benchmark task consists of a cross-subject classification to distinguish between two reading paradigms: normal reading and task-specific reading. The data for the benchmark is based on the Zurich Cognitive Language Processing Corpus (ZuCo 2.0), which provides simultaneous eye-tracking and EEG signals from natural reading of English sentences. The training dataset is publicly available, and we present a newly recorded hidden testset. We provide multiple solid baseline methods for this task and discuss future improvements. We release our code and provide an easy-to-use interface to evaluate new approaches with an accompanying public leaderboard: .
We argue for a perspective on bilingual heritage speakers as native speakers of both their languages and present results from a large-scale, cross-linguistic study that took such a perspective and approached bilinguals and monolinguals on equal grounds.
We targeted comparable language use in bilingual and monolingual speakers, crucially covering broader repertoires than just formal language. A main database was the open-access RUEG corpus, which covers comparable informal vs. formal and spoken vs. written productions by adolescent and adult bilinguals with heritage-Greek, -Russian, and -Turkish in Germany and the United States and with heritage-German in the United States, and matching data from monolinguals in Germany, the United States, Greece, Russia, and Turkey. Our main results lie in three areas.
(1) We found non-canonical patterns not only in bilingual, but also in monolingual speakers, including patterns that have so far been considered absent from native grammars, in domains of morphology, syntax, intonation, and pragmatics.
(2) We found a degree of lexical and morphosyntactic inter-speaker variability in monolinguals that was sometimes higher than that of bilinguals, further challenging the model of the streamlined native speaker.
(3) In majority language use, non-canonical patterns were dominant in spoken and/or informal registers, and this was true for monolinguals and bilinguals. In some cases, bilingual speakers were leading quantitatively. In heritage settings where the language was not part of formal schooling, we found tendencies of register leveling, presumably due to the fact that speakers had limited access to formal registers of the heritage language.
Our findings thus indicate possible quantitative differences and different register distributions rather than distinct grammatical patterns in bilingual and monolingual speakers. This supports the integration of heritage speakers into the native-speaker continuum. Approaching heritage speakers from this perspective helps us to better understand the empirical data and can shed light on language variation and change in native grammars.
Furthermore, our findings for monolinguals lead us to reconsider the state-of-the art on majority languages, given recurring evidence for non-canonical patterns that deviate from what has been assumed in the literature so far, and might have been attributed to bilingualism had we not included informal and spoken registers in monolinguals and bilinguals alike.
Stimulus data and experimental design for a self-paced reading study on emoji-word substitutions
(2022)
This data paper presents the experimental design and stimuli from an online self-paced reading study on the processing of emojis substituting lexically ambiguous nouns. We recorded reading times for the target ambiguous nouns and for emojis depicting either the intended target referent or a contextually inappropriate homophonous noun. Furthermore, we recorded comprehension accuracy, demographics and a self-assessment of the participants' emoji usage frequency. The data includes all stimuli used, the raw data, the full JavaScript code for the online experiment, as well as Python and R code for the data analysis. We believe that our dataset may give important insights related to the comprehension mechanisms involved in the cognitive processing of emojis. For interpretation and discussion of the experiment, please see the original article entitled "The processing of emoji-word substitutions: A self-paced-reading study".
How tone, intonation and emotion shape the development of infants' fundamental frequency perception
(2022)
Fundamental frequency (integral(0)), perceived as pitch, is the first and arguably most salient auditory component humans are exposed to since the beginning of life.
It carries multiple linguistic (e.g., word meaning) and paralinguistic (e.g., speakers' emotion) functions in speech and communication.
The mappings between these functions and integral(0) features vary within a language and differ cross-linguistically. For instance, a rising pitch can be perceived as a question in English but a lexical tone in Mandarin. Such variations mean that infants must learn the specific mappings based on their respective linguistic and social environments.
To date, canonical theoretical frameworks and most empirical studies do not view or consider the multi-functionality of integral(0), but typically focus on individual functions. More importantly, despite the eventual mastery of integral(0) in communication, it is unclear how infants learn to decompose and recognize these overlapping functions carried by integral(0). In this paper, we review the symbioses and synergies of the lexical, intonational, and emotional functions that can be carried by integral(0) and are being acquired throughout infancy.
On the basis of our review, we put forward the Learnability Hypothesis that infants decompose and acquire multiple integral(0) functions through native/environmental experiences. Under this hypothesis, we propose representative cases such as the synergy scenario, where infants use visual cues to disambiguate and decompose the different integral(0) functions. Further, viable ways to test the scenarios derived from this hypothesis are suggested across auditory and visual modalities.
Discovering how infants learn to master the diverse functions carried by integral(0) can increase our understanding of linguistic systems, auditory processing and communication functions.
In this study, we investigated the cognitive-emotional interplay by measuring the effects of executive competition (Pessoa, 2013), i.e., how inhibitory control is influenced when emotional information is encountered. Sixty-three children (8 to 9 years of age) participated in an inhibition task (central task) accompanied by happy, sad, or neutral emoticons (displayed in the periphery). Typical interference effects were found in the main task for speed and accuracy, but in general, these effects were not additionally modulated by the peripheral emoticons indicating that processing of the main task exhausted the limited capacity such that interference from the task-irrelevant, peripheral information did not show (Pessoa, 2013). Further analyses revealed that the magnitude of interference effects depended on the order of congruency conditions: when incongruent conditions preceded congruent ones, there was greater interference. This effect was smaller in sad conditions, and particularly so at the beginning of the experiment. These findings suggest that the bottom-up perception of task-irrelevant emotional information influenced the top-down process of inhibitory control among children in the sad condition when processing demands were particularly high. We discuss if the salience and valence of the emotional stimuli as well as task demands are the decisive characteristics that modulate the strength of this relation.
Pivots revisited
(2021)
The term "pivot" usually refers to two overlapping syntactic units such that the completion of the first unit simultaneously launches the second. In addition, pivots are generally said to be characterized by the smooth prosodic integration of their syntactic parts. This prosodic integration is typically achieved by prosodic-phonetic matching of the pivot components. As research on such turns in a range of languages has illustrated, speakers routinely deploy pivots so as to be able to continue past a point of possible turn completion, in the service of implementing some additional or revised action. This article seeks to build on, and complement, earlier research by exploring two issues in more detail as follows: (1) what exactly do pivotal turn extensions accomplish on the action dimension, and (2) what role does prosodic-phonetic packaging play in this? We will show that pivot constructions not only exhibit various degrees of prosodic-phonetic (non-)integration, i.e., differently strong cesuras, but that they can be ordered on a continuum, and that this cline maps onto the relationship of the actions accomplished by the components of the pivot construction. While tighter prosodic-phonetic integration, i.e., weak(er) cesuring, co-occurs with post-pivot actions whose relationship to that of the pre-pivot tends to be rather retrospective in character, looser prosodic-phonetic integration, i.e., strong(er) cesuring, is associated with a more prospective orientation of the post-pivot's action. These observations also raise more general questions with regard to the analysis of action.
I can see it in your eyes
(2021)
Over the past years, extensive research has been dedicated to developing robust platforms and data-driven dialog models to support long-term human-robot interactions. However, little is known about how people's perception of robots and engagement with them develop over time and how these can be accurately assessed through implicit and continuous measurement techniques. In this paper, we explore this by involving participants in three interaction sessions with multiple days of zero exposure in between. Each session consists of a joint task with a robot as well as two short social chats with it before and after the task. We measure participants' gaze patterns with a wearable eye-tracker and gauge their perception of the robot and engagement with it and the joint task using questionnaires. Results disclose that aversion of gaze in a social chat is an indicator of a robot's uncanniness and that the more people gaze at the robot in a joint task, the worse they perform. In contrast with most HRI literature, our results show that gaze toward an object of shared attention, rather than gaze toward a robotic partner, is the most meaningful predictor of engagement in a joint task. Furthermore, the analyses of gaze patterns in repeated interactions disclose that people's mutual gaze in a social chat develops congruently with their perceptions of the robot over time. These are key findings for the HRI community as they entail that gaze behavior can be used as an implicit measure of people's perception of robots in a social chat and of their engagement and task performance in a joint task.
In this paper we examine the effect of uncertainty on readers' predictions about meaning. In particular, we were interested in how uncertainty might influence the likelihood of committing to a specific sentence meaning. We conducted two event-related potential (ERP) experiments using particle verbs such as turn down and manipulated uncertainty by constraining the context such that readers could be either highly certain about the identity of a distant verb particle, such as turn the bed [...] down, or less certain due to competing particles, such as turn the music [...] up/down. The study was conducted in German, where verb particles appear clause-finally and may be separated from the verb by a large amount of material. We hypothesised that this separation would encourage readers to predict the particle, and that high certainty would make prediction of a specific particle more likely than lower certainty. If a specific particle was predicted, this would reflect a strong commitment to sentence meaning that should incur a higher processing cost if the prediction is wrong. If a specific particle was less likely to be predicted, commitment should be weaker and the processing cost of a wrong prediction lower. If true, this could suggest that uncertainty discourages predictions via an unacceptable cost-benefit ratio. However, given the clear predictions made by the literature, it was surprisingly unclear whether the uncertainty manipulation affected the two ERP components studied, the N400 and the PNP. Bayes factor analyses showed that evidence for our a priori hypothesised effect sizes was inconclusive, although there was decisive evidence against a priori hypothesised effect sizes larger than 1 mu Vfor the N400 and larger than 3 mu V for the PNP. We attribute the inconclusive finding to the properties of verb-particle dependencies that differ from the verb-noun dependencies in which the N400 and PNP are often studied.
Discourse Prominence and Antecedent MisRetrieval during Native and Non-Native Pronoun Resolution
(2022)
Previous studies on non-native (L2) anaphor resolution suggest that L2 comprehenders are guided more strongly by discourse-level cues compared to native (L1) comprehenders. Here we examine whether and how a grammatically inappropriate antecedent’s discourse status affects the likelihood of it being considered during L1 and L2 pronoun resolution. We used an interference paradigm to examine how the extrasentential discourse impacts the resolution of German object pronouns. In an eye-tracking-during-reading experiment we examined whether an elaborated local antecedent ruled out by binding Condition B would be mis-retrieved during pronoun resolution, and whether initially introducing this antecedent as the discourse topic would affect the chances of it being mis-retrieved. While both participant groups rejected the inappropriate antecedent in an offline questionnaire irrespective of its discourse prominence, their real-time processing patterns differed. L1 speakers initially mis-retrieved the inappropriate antecedent regardless of its contextual prominence. L1 Russian/L2 German speakers, in contrast, were affected by the antecedent’s discourse status, considering it only when it was discourse-new but not when it had previously been introduced as the discourse topic. Our findings show that L2 comprehenders are highly sensitive to discourse dynamics such as topic shifts, supporting the claim that discourse-level cues are more strongly weighted during L2 compared to L1 processing.
When researchers carry out a null hypothesis significance test, it is tempting to assume that a statistically significant result lowers Prob(H0), the probability of the null hypothesis being true. Technically, such a statement is meaningless for various reasons: e.g., the null hypothesis does not have a probability associated with it. However, it is possible to relax certain assumptions to compute the posterior probability Prob(H0) under repeated sampling. We show in a step-by-step guide that the intuitively appealing belief, that Prob(H0) is low when significant results have been obtained under repeated sampling, is in general incorrect and depends greatly on: (a) the prior probability of the null being true; (b) type-I error rate, (c) type-II error rate, and (d) replication of a result. Through step-by-step simulations using open-source code in the R System of Statistical Computing, we show that uncertainty about the null hypothesis being true often remains high despite a significant result. To help the reader develop intuitions about this common misconception, we provide a Shiny app (https://danielschad.shinyapps.io/probnull/). We expect that this tutorial will help researchers better understand and judge results from null hypothesis significance tests.