TY - JOUR A1 - Matuschek, Hannes A1 - Kliegl, Reinhold A1 - Vasishth, Shravan A1 - Baayen, Harald R. A1 - Bates, Douglas T1 - Balancing Type I error and power in linear mixed models JF - Journal of memory and language N2 - Linear mixed-effects models have increasingly replaced mixed-model analyses of variance for statistical inference in factorial psycholinguistic experiments. Although LMMs have many advantages over ANOVA, like ANOVAs, setting them up for data analysis also requires some care. One simple option, when numerically possible, is to fit the full variance covariance structure of random effects (the maximal model; Barr, Levy, Scheepers & Tily, 2013), presumably to keep Type I error down to the nominal a in the presence of random effects. Although it is true that fitting a model with only random intercepts may lead to higher Type I error, fitting a maximal model also has a cost: it can lead to a significant loss of power. We demonstrate this with simulations and suggest that for typical psychological and psycholinguistic data, higher power is achieved without inflating Type I error rate if a model selection criterion is used to select a random effect structure that is supported by the data. (C) 2017 The Authors. Published by Elsevier Inc. KW - Power KW - Linear mixed effect model KW - Hypothesis testing Y1 - 2017 U6 - https://doi.org/10.1016/j.jml.2017.01.001 SN - 0749-596X SN - 1096-0821 VL - 94 SP - 305 EP - 315 PB - Elsevier CY - San Diego ER - TY - JOUR A1 - Baayen, Harald R. A1 - Vasishth, Shravan A1 - Kliegl, Reinhold A1 - Bates, Douglas T1 - The cave of shadows: Addressing the human factor with generalized additive mixed models JF - Journal of memory and language KW - Generalized additive mixed models KW - Within-experiment adaptation KW - Autocorrelation KW - Experimental time series KW - Confirmatory versus exploratory data analysis KW - Model selection Y1 - 2017 U6 - https://doi.org/10.1016/j.jml.2016.11.006 SN - 0749-596X SN - 1096-0821 VL - 94 SP - 206 EP - 234 PB - Elsevier CY - San Diego ER - TY - JOUR A1 - Engelmann, Felix A1 - Vasishth, Shravan A1 - Engbert, Ralf A1 - Kliegl, Reinhold T1 - A framework for modeling the interaction of syntactic processing and eye movement control JF - Topics in cognitive science N2 - We explore the interaction between oculomotor control and language comprehension on the sentence level using two well-tested computational accounts of parsing difficulty. Previous work (Boston, Hale, Vasishth, & Kliegl, 2011) has shown that surprisal (Hale, 2001; Levy, 2008) and cue-based memory retrieval (Lewis & Vasishth, 2005) are significant and complementary predictors of reading time in an eyetracking corpus. It remains an open question how the sentence processor interacts with oculomotor control. Using a simple linking hypothesis proposed in Reichle, Warren, and McConnell (2009), we integrated both measures with the eye movement model EMMA (Salvucci, 2001) inside the cognitive architecture ACT-R (Anderson et al., 2004). We built a reading model that could initiate short Time Out regressions (Mitchell, Shen, Green, & Hodgson, 2008) that compensate for slow postlexical processing. This simple interaction enabled the model to predict the re-reading of words based on parsing difficulty. The model was evaluated in different configurations on the prediction of frequency effects on the Potsdam Sentence Corpus. The extension of EMMA with postlexical processing improved its predictions and reproduced re-reading rates and durations with a reasonable fit to the data. This demonstration, based on simple and independently motivated assumptions, serves as a foundational step toward a precise investigation of the interaction between high-level language processing and eye movement control. KW - Sentence comprehension KW - Eye movements KW - Reading KW - Parsing difficulty KW - Working memory KW - Surprisal KW - Computational modeling Y1 - 2013 U6 - https://doi.org/10.1111/tops.12026 SN - 1756-8757 VL - 5 IS - 3 SP - 452 EP - 474 PB - Wiley-Blackwell CY - Hoboken ER - TY - JOUR A1 - Boston, Marisa Ferrara A1 - Halbe, John T. A1 - Vasishth, Shravan A1 - Kliegl, Reinhold T1 - Parallel processing and entence comprehension difficulty N2 - Eye fixation durations during normal reading correlate with processing difficulty, but the specific cognitive mechanisms reflected in these measures are not well understood. This study finds support in German readers' eye fixations for two distinct difficulty metrics: surprisal, which reflects the change in probabilities across syntactic analyses as new words are integrated; and retrieval, which quantifies comprehension difficulty in terms of working memory constraints. We examine the predictions of both metrics using a family of dependency parsers indexed by an upper limit on the number of candidate syntactic analyses they retain at successive words. Surprisal models all fixation measures and regression probability. By contrast, retrieval does not model any measure in serial processing. As more candidate analyses are considered in parallel at each word, retrieval can account for the same measures as surprisal. This pattern suggests an important role for ranked parallelism in theories of sentence comprehension. Y1 - 2011 UR - http://www.tandfonline.com/doi/full/10.1080/01690965.2010.492228 U6 - https://doi.org/10.1080/01690965.2010.492228 ER - TY - JOUR A1 - Boston, Marisa Ferrara A1 - Hale, John A1 - Kliegl, Reinhold A1 - Patil, Umesh A1 - Vasishth, Shravan T1 - Parsing costs as predictors of reading difficulty : an evaluation using the Potsdam Sentence Corpus Y1 - 2008 UR - http://www.jemr.org/ SN - 1995-8692 ER - TY - JOUR A1 - Boston, Marisa Ferrara A1 - Hale, John A1 - Kliegl, Reinhold A1 - Vasishth, Shravan T1 - Surprising parser actions and reading difficulty Y1 - 2008 ER - TY - JOUR A1 - Boston, Marisa Ferrara A1 - Hale, John T. A1 - Vasishth, Shravan A1 - Kliegl, Reinhold T1 - Parallel processing and sentence comprehension difficulty JF - Language and cognitive processes N2 - Eye fixation durations during normal reading correlate with processing difficulty, but the specific cognitive mechanisms reflected in these measures are not well understood. This study finds support in German readers' eye fixations for two distinct difficulty metrics: surprisal, which reflects the change in probabilities across syntactic analyses as new words are integrated; and retrieval, which quantifies comprehension difficulty in terms of working memory constraints. We examine the predictions of both metrics using a family of dependency parsers indexed by an upper limit on the number of candidate syntactic analyses they retain at successive words. Surprisal models all fixation measures and regression probability. By contrast, retrieval does not model any measure in serial processing. As more candidate analyses are considered in parallel at each word, retrieval can account for the same measures as surprisal. This pattern suggests an important role for ranked parallelism in theories of sentence comprehension. KW - Reading KW - Parsing KW - Computer model KW - Corpus Y1 - 2011 U6 - https://doi.org/10.1080/01690965.2010.492228 SN - 0169-0965 VL - 26 IS - 3 SP - 301 EP - 349 PB - Wiley CY - Hove ER - TY - JOUR A1 - Nicenboim, Bruno A1 - Vasishth, Shravan A1 - Gattei, Carolina A1 - Sigman, Mariano A1 - Kliegl, Reinhold T1 - Working memory differences in long-distance dependency resolution JF - Frontiers in psychology N2 - There is a wealth of evidence showing that increasing the distance between an argument and its head leads to more processing effort, namely, locality effects: these are usually associated with constraints in working memory (DLT: Gibson, 2000: activation-based model: Lewis and Vasishth, 2005). In SOV languages, however, the opposite effect has been found: antilocality (see discussion in Levy et al., 2013). Antilocality effects can be explained by the expectation based approach as proposed by Levy (2008) or by the activation-based model of sentence processing as proposed by Lewis and Vasishth (2005). We report an eye-tracking and a self-paced reading study with sentences in Spanish together with measures of individual differences to examine the distinction between expectation- and memory based accounts, and within memory-based accounts the further distinction between DLT and the activation-based model. The experiments show that (i) antilocality effects as predicted by the expectation account appear only for high-capacity readers; (ii) increasing dependency length by interposing material that modifies the head of the dependency (the verb) produces stronger facilitation than increasing dependency length with material that does not modify the head; this is in agreement with the activation-based model but not with the expectation account; and (iii) a possible outcome of memory load on low-capacity readers is the increase in regressive saccades (locality effects as predicted by memory-based accounts) or, surprisingly, a speedup in the self-paced reading task; the latter consistent with good-enough parsing (Ferreira et al., 2002). In sum, the study suggests that individual differences in working memory capacity play a role in dependency resolution, and that some of the aspects of dependency resolution can be best explained with the activation-based model together with a prediction component. KW - locality KW - antilocality KW - working memory capacity KW - individual differences KW - Spanish KW - activation KW - DLT KW - expectation Y1 - 2015 U6 - https://doi.org/10.3389/fpsyg.2015.00312 SN - 1664-1078 VL - 6 PB - Frontiers Research Foundation CY - Lausanne ER - TY - GEN A1 - Boston, Marisa Ferrara A1 - Hale, John A1 - Kliegl, Reinhold A1 - Patil, Umesh A1 - Vasishth, Shravan T1 - Parsing costs as predictors of reading difficulty: An evaluation using the Potsdam Sentence Corpus N2 - The surprisal of a word on a probabilistic grammar constitutes a promising complexity metric for human sentence comprehension difficulty. Using two different grammar types, surprisal is shown to have an effect on fixation durations and regression probabilities in a sample of German readers’ eye movements, the Potsdam Sentence Corpus. A linear mixed-effects model was used to quantify the effect of surprisal while taking into account unigram and bigram frequency, word length, and empirically-derived word predictability; the so-called “early” and “late” measures of processing difficulty both showed an effect of surprisal. Surprisal is also shown to have a small but statistically non-significant effect on empirically-derived predictability itself. This work thus demonstrates the importance of including parsing costs as a predictor of comprehension difficulty in models of reading, and suggests that a simple identification of syntactic parsing costs with early measures and late measures with durations of post-syntactic events may be difficult to uphold. T3 - Zweitveröffentlichungen der Universität Potsdam : Humanwissenschaftliche Reihe - paper 253 Y1 - 2008 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus-57139 ER - TY - JOUR A1 - von der Malsburg, Titus Raban A1 - Kliegl, Reinhold A1 - Vasishth, Shravan T1 - Determinants of Scanpath Regularity in Reading JF - Cognitive science : a multidisciplinary journal of anthropology, artificial intelligence, education, linguistics, neuroscience, philosophy, psychology ; journal of the Cognitive Science Society N2 - Scanpaths have played an important role in classic research on reading behavior. Nevertheless, they have largely been neglected in later research perhaps due to a lack of suitable analytical tools. Recently, von der Malsburg and Vasishth (2011) proposed a new measure for quantifying differences between scanpaths and demonstrated that this measure can recover effects that were missed with the traditional eyetracking measures. However, the sentences used in that study were difficult to process and scanpath effects accordingly strong. The purpose of the present study was to test the validity, sensitivity, and scope of applicability of the scanpath measure, using simple sentences that are typically read from left to right. We derived predictions for the regularity of scanpaths from the literature on oculomotor control, sentence processing, and cognitive aging and tested these predictions using the scanpath measure and a large database of eye movements. All predictions were confirmed: Sentences with short words and syntactically more difficult sentences elicited more irregular scanpaths. Also, older readers produced more irregular scanpaths than younger readers. In addition, we found an effect that was not reported earlier: Syntax had a smaller influence on the eye movements of older readers than on those of young readers. We discuss this interaction of syntactic parsing cost with age in terms of shifts in processing strategies and a decline of executive control as readers age. Overall, our results demonstrate the validity and sensitivity of the scanpath measure and thus establish it as a productive and versatile tool for reading research. KW - Eye movements KW - Reading KW - Scanpaths KW - Language understanding KW - Oculo-motor control KW - Individual differences KW - Aging KW - Development Y1 - 2015 U6 - https://doi.org/10.1111/cogs.12208 SN - 0364-0213 SN - 1551-6709 VL - 39 IS - 7 SP - 1675 EP - 1703 PB - Wiley-Blackwell CY - Hoboken ER - TY - GEN A1 - Boston, Marisa Ferrara A1 - Hale, John T. A1 - Vasishth, Shravan A1 - Kliegl, Reinhold T1 - Parallel processing and sentence comprehension difficulty N2 - Eye fixation durations during normal reading correlate with processing difficulty but the specific cognitive mechanisms reflected in these measures are not well understood. This study finds support in German readers’ eyefixations for two distinct difficulty metrics: surprisal, which reflects the change in probabilities across syntactic analyses as new words are integrated, and retrieval, which quantifies comprehension difficulty in terms of working memory constraints. We examine the predictions of both metrics using a family of dependency parsers indexed by an upper limit on the number of candidate syntactic analyses they retain at successive words. Surprisal models all fixation measures and regression probability. By contrast, retrieval does not model any measure in serial processing. As more candidate analyses are considered in parallel at each word, retrieval can account for the same measures as surprisal. This pattern suggests an important role for ranked parallelism in theories of sentence comprehension. T3 - Zweitveröffentlichungen der Universität Potsdam : Humanwissenschaftliche Reihe - paper 252 Y1 - 2011 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus-57159 ER - TY - JOUR A1 - Nicenboim, Bruno A1 - Vasishth, Shravan A1 - Gattei, Carolina A1 - Sigman, Mariano A1 - Kliegl, Reinhold T1 - Working memory differences in long-distance dependency resolution JF - Frontiers in psychology N2 - There is a wealth of evidence showing that increasing the distance between an argument and its head leads to more processing effort, namely, locality effects; these are usually associated with constraints in working memory (DLT: Gibson, 2000; activation-based model: Lewis and Vasishth, 2005). In SOV languages, however, the opposite effect has been found: antilocality (see discussion in Levy et al., 2013). Antilocality effects can be explained by the expectation-based approach as proposed by Levy (2008) or by the activation-based model of sentence processing as proposed by Lewis and Vasishth (2005). We report an eye-tracking and a self-paced reading study with sentences in Spanish together with measures of individual differences to examine the distinction between expectation- and memory-based accounts, and within memory-based accounts the further distinction between DLT and the activation-based model. The experiments show that (i) antilocality effects as predicted by the expectation account appear only for high-capacity readers; (ii) increasing dependency length by interposing material that modifies the head of the dependency (the verb) produces stronger facilitation than increasing dependency length with material that does not modify the head; this is in agreement with the activation-based model but not with the expectation account; and (iii) a possible outcome of memory load on low-capacity readers is the increase in regressive saccades (locality effects as predicted by memory-based accounts) or, surprisingly, a speedup in the self-paced reading task; the latter consistent with good-enough parsing (Ferreira et al., 2002). In sum, the study suggests that individual differences in working memory capacity play a role in dependency resolution, and that some of the aspects of dependency resolution can be best explained with the activation-based model together with a prediction component. KW - locality KW - antilocality KW - working memory capacity KW - individual differences KW - Spanish KW - activation KW - DLT KW - expectation Y1 - 2015 U6 - https://doi.org/10.3389/fpsyg.2015.00312 SN - 1664-1078 VL - 6 IS - 312 PB - Frontiers Research Foundation CY - Lausanne ER - TY - JOUR A1 - Schad, Daniel A1 - Vasishth, Shravan A1 - Hohenstein, Sven A1 - Kliegl, Reinhold T1 - How to capitalize on a priori contrasts in linear (mixed) models BT - a tutorial JF - Journal of memory and language N2 - Factorial experiments in research on memory, language, and in other areas are often analyzed using analysis of variance (ANOVA). However, for effects with more than one numerator degrees of freedom, e.g., for experimental factors with more than two levels, the ANOVA omnibus F-test is not informative about the source of a main effect or interaction. Because researchers typically have specific hypotheses about which condition means differ from each other, a priori contrasts (i.e., comparisons planned before the sample means are known) between specific conditions or combinations of conditions are the appropriate way to represent such hypotheses in the statistical model. Many researchers have pointed out that contrasts should be "tested instead of, rather than as a supplement to, the ordinary 'omnibus' F test" (Hays, 1973, p. 601). In this tutorial, we explain the mathematics underlying different kinds of contrasts (i.e., treatment, sum, repeated, polynomial, custom, nested, interaction contrasts), discuss their properties, and demonstrate how they are applied in the R System for Statistical Computing (R Core Team, 2018). In this context, we explain the generalized inverse which is needed to compute the coefficients for contrasts that test hypotheses that are not covered by the default set of contrasts. A detailed understanding of contrast coding is crucial for successful and correct specification in linear models (including linear mixed models). Contrasts defined a priori yield far more useful confirmatory tests of experimental hypotheses than standard omnibus F-tests. Reproducible code is available from https://osf.io/7ukf6/. KW - contrasts KW - null hypothesis significance testing KW - linear models KW - a priori KW - hypotheses Y1 - 2019 U6 - https://doi.org/10.1016/j.jml.2019.104038 SN - 0749-596X SN - 1096-0821 VL - 110 PB - Elsevier CY - San Diego ER - TY - GEN A1 - Nicenboim, Bruno A1 - Vasishth, Shravan A1 - Gattei, Carolina A1 - Sigman, Mariano A1 - Kliegl, Reinhold T1 - Working memory differences in long-distance dependency resolution N2 - There is a wealth of evidence showing that increasing the distance between an argument and its head leads to more processing effort, namely, locality effects; these are usually associated with constraints in working memory (DLT: Gibson, 2000; activation-based model: Lewis and Vasishth, 2005). In SOV languages, however, the opposite effect has been found: antilocality (see discussion in Levy et al., 2013). Antilocality effects can be explained by the expectation-based approach as proposed by Levy (2008) or by the activation-based model of sentence processing as proposed by Lewis and Vasishth (2005). We report an eye-tracking and a self-paced reading study with sentences in Spanish together with measures of individual differences to examine the distinction between expectation- and memory-based accounts, and within memory-based accounts the further distinction between DLT and the activation-based model. The experiments show that (i) antilocality effects as predicted by the expectation account appear only for high-capacity readers; (ii) increasing dependency length by interposing material that modifies the head of the dependency (the verb) produces stronger facilitation than increasing dependency length with material that does not modify the head; this is in agreement with the activation-based model but not with the expectation account; and (iii) a possible outcome of memory load on low-capacity readers is the increase in regressive saccades (locality effects as predicted by memory-based accounts) or, surprisingly, a speedup in the self-paced reading task; the latter consistent with good-enough parsing (Ferreira et al., 2002). In sum, the study suggests that individual differences in working memory capacity play a role in dependency resolution, and that some of the aspects of dependency resolution can be best explained with the activation-based model together with a prediction component. T3 - Zweitveröffentlichungen der Universität Potsdam : Humanwissenschaftliche Reihe - paper 273 KW - locality KW - antilocality KW - working memory capacity KW - individual differences KW - Spanish KW - activation KW - DLT KW - expectation Y1 - 2015 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus4-75694 ER -