Refine
Year of publication
Document Type
- Article (21)
- Postprint (9)
- Doctoral Thesis (6)
Is part of the Bibliography
- yes (36)
Keywords
- eye-tracking (36) (remove)
Institute
- Department Linguistik (21)
- Strukturbereich Kognitionswissenschaften (4)
- Department Psychologie (3)
- Department Sport- und Gesundheitswissenschaften (2)
- Department Erziehungswissenschaft (1)
- Department Grundschulpädagogik (1)
- Hasso-Plattner-Institut für Digital Engineering GmbH (1)
- Humanwissenschaftliche Fakultät (1)
- Institut für Romanistik (1)
- Strukturbereich Bildungswissenschaften (1)
The ZuCo benchmark on cross-subject reading task classification with EEG and eye-tracking data
(2023)
We present a new machine learning benchmark for reading task classification with the goal of advancing EEG and eye-tracking research at the intersection between computational language processing and cognitive neuroscience. The benchmark task consists of a cross-subject classification to distinguish between two reading paradigms: normal reading and task-specific reading. The data for the benchmark is based on the Zurich Cognitive Language Processing Corpus (ZuCo 2.0), which provides simultaneous eye-tracking and EEG signals from natural reading of English sentences. The training dataset is publicly available, and we present a newly recorded hidden testset. We provide multiple solid baseline methods for this task and discuss future improvements. We release our code and provide an easy-to-use interface to evaluate new approaches with an accompanying public leaderboard: .
Thematic role assignment and word order preferences in the child language acquisition of Tagalog
(2018)
A critical task in daily communications is identifying who did what to whom in an utterance, or assigning the thematic roles agent and patient in a sentence. This dissertation is concerned with Tagalog-speaking children’s use of word order and morphosyntactic markers for thematic role assignment. It aims to explain children’s difficulties in interpreting sentences with a non-canonical order of arguments (i.e., patient-before-agent) by testing the predictions of the following accounts: the frequency account (Demuth, 1989), the Competition model (MacWhinney & Bates, 1989), and the incremental processing account (Trueswell & Gleitman, 2004). Moreover, the experiments in this dissertation test the influence of a word order strategy in a language like Tagalog, where the thematic roles are always unambiguous in a sentence, due to its verb-initial order and its voice-marking system. In Tagalog’s voice-marking system, the inflection on the verb indicates the thematic role of the noun marked by 'ang.' First, the possible basis for a word order strategy in Tagalog was established using a sentence completion experiment given to adults and 5- and 7-year-old children (Chapter 2) and a child-directed speech corpus analysis (Chapter 3). In general, adults and children showed an agent-before-patient preference, although adults’ preference was also affected by sentence voice. Children’s comprehension was then examined through a self-paced listening and picture verification task (Chapter 3) and an eye-tracking and picture selection task (Chapter 4), where word order (agent-initial or patient-initial) and voice (agent voice or patient voice) were manipulated. Offline (i.e., accuracy) and online (i.e., listening times, looks to the target) measures revealed that 5- and 7-year-old Tagalog-speaking children had a bias to interpret the first noun as the agent. Additionally, the use of word order and morphosyntactic markers was found to be modulated by voice. In the agent voice, children relied more on a word order strategy; while in the patient voice, they relied on the morphosyntactic markers. These results are only partially explained by the accounts being tested in this dissertation. Instead, the findings support computational accounts of incremental word prediction and learning such as Chang, Dell, & Bock’s (2006) model.
It is well established in language acquisition research that monolingual children and adult second language learners misinterpret sentences with the universal quantifier every and make quantifier-spreading errors that are attributed to a preference for a match in number between two sets of objects. The present Visual World eye-tracking study tested bilingual heritage Russian–English adults and investigated how they interpret of sentences like Every alligator lies in a bathtub in both languages. Participants performed a sentence–picture verification task while their eye movements were recorded. Pictures showed three pairs of alligators in bathtubs and two extra objects: elephants (Control condition), bathtubs (Overexhaustive condition), or alligators (Underexhaustive condition). Monolingual adults performed at ceiling in all conditions. Heritage language (HL) adults made 20% q-spreading errors, but only in the Overexhaustive condition, and when they made an error they spent more time looking at the two extra bathtubs during the Verb region. We attribute q-spreading in HL speakers to cognitive overload caused by the necessity to integrate conflicting sources of information, i.e. the spoken sentences in their weaker, heritage, language and attention-demanding visual context, that differed with respect to referential salience.
It is well established in language acquisition research that monolingual children and adult second language learners misinterpret sentences with the universal quantifier every and make quantifier-spreading errors that are attributed to a preference for a match in number between two sets of objects. The present Visual World eye-tracking study tested bilingual heritage Russian-English adults and investigated how they interpret of sentences like Every alligator lies in a bathtub in both languages. Participants performed a sentence-picture verification task while their eye movements were recorded. Pictures showed three pairs of alligators in bathtubs and two extra objects: elephants (Control condition), bathtubs (Overexhaustive condition), or alligators (Underexhaustive condition). Monolingual adults performed at ceiling in all conditions. Heritage language (HL) adults made 20% q-spreading errors, but only in the Overexhaustive condition, and when they made an error they spent more time looking at the two extra bathtubs during the Verb region. We attribute q-spreading in HL speakers to cognitive overload caused by the necessity to integrate conflicting sources of information, i.e. the spoken sentences in their weaker, heritage, language and attention-demanding visual context, that differed with respect to referential salience.
Background: Comprehension of non-canonical sentences is frequently characterised by chance level performance in people with aphasia (PWA). Chance level performance has been interpreted as guessing, but online data does not support this rendering. It is still not clear whether the incorrect sentence processing is guided by the compensatory strategies that PWA might employ to overcome linguistic difficulties.Aims: We aim to study to what extent people with non-fluent aphasia are aware of their sentence comprehension deficits.Methods & Procedures: This study combined offline and online data to investigate the effect of word order and error-awareness on sentence comprehension in a group of PWA and non-brain damaged (NBD) participants. The offline tasks involved auditory sentence picture-matching immediately followed by a confidence rating (CR). Participants were asked to judge the perceived correctness of their previous answer. Online data consisted of eye-tracking.Outcomes & Results: Replicating previous findings, PWA had significantly worse comprehension of Theme-Agent order compared to Agent-Theme order sentences. Controls showed ceiling level sentence comprehension. CR was a poor predictor of response accuracy in PWA, but moderate-good in NBD. A total of 6.8% of judgements were classified as guessing by PWA. Post hoc gaze data analysis indicated that CR was a predictor of the fixation pattern during the presentation of the linguistic stimuli.Conclusions: Results suggest that PWA were mostly unaware of their sentence comprehension errors and did not consciously employ strategies to compensate for their difficulties.
In an eye-tracking study we tested the hypothesis that comprehension is facilitated by a match between the order of the verb and its arguments in a sentence and the order of the actual sensorimotor interaction with these objects (for example, in the phrase put the bag into the box, the order of the arguments corresponds to the order of motor actions: take the bag, put it into the box) could facilitate comprehension of such constructions. We tested 40 native Russian speakers in a visual world sentence-picture matching task. In prepositional constructions, there was no difference between conditions that matched or mismatched sensorimotor stereotypes, whereas in instrumental constructions, sensorimotor stereotypes facilitated comprehension.