Statistical Methods for Linguistic Research: Foundational Ideas - Part I
- We present the fundamental ideas underlying statistical hypothesis testing using the frequentist framework. We start with a simple example that builds up the one-sample t-test from the beginning, explaining important concepts such as the sampling distribution of the sample mean, and the iid assumption. Then, we examine the meaning of the p-value in detail and discuss several important misconceptions about what a p-value does and does not tell us. This leads to a discussion of Type I, II error and power, and Type S and M error. An important conclusion from this discussion is that one should aim to carry out appropriately powered studies. Next, we discuss two common issues that we have encountered in psycholinguistics and linguistics: running experiments until significance is reached and the ‘garden-of-forking-paths’ problem discussed by Gelman and others. The best way to use frequentist methods is to run appropriately powered studies, check model assumptions, clearly separate exploratory data analysis from planned comparisons decidedWe present the fundamental ideas underlying statistical hypothesis testing using the frequentist framework. We start with a simple example that builds up the one-sample t-test from the beginning, explaining important concepts such as the sampling distribution of the sample mean, and the iid assumption. Then, we examine the meaning of the p-value in detail and discuss several important misconceptions about what a p-value does and does not tell us. This leads to a discussion of Type I, II error and power, and Type S and M error. An important conclusion from this discussion is that one should aim to carry out appropriately powered studies. Next, we discuss two common issues that we have encountered in psycholinguistics and linguistics: running experiments until significance is reached and the ‘garden-of-forking-paths’ problem discussed by Gelman and others. The best way to use frequentist methods is to run appropriately powered studies, check model assumptions, clearly separate exploratory data analysis from planned comparisons decided upon before the study was run, and always attempt to replicate results.…
Author details: | Shravan VasishthORCiDGND, Bruno NicenboimORCiDGND |
---|---|
DOI: | https://doi.org/10.1111/lnc3.12201 |
ISSN: | 1749-818X |
Title of parent work (English): | Language and linguistics compass |
Publisher: | Wiley-Blackwell |
Place of publishing: | Hoboken |
Publication type: | Article |
Language: | English |
Year of first publication: | 2016 |
Publication year: | 2016 |
Release date: | 2020/03/22 |
Volume: | 10 |
Number of pages: | 21 |
First page: | 349 |
Last Page: | 369 |
Organizational units: | Humanwissenschaftliche Fakultät / Strukturbereich Kognitionswissenschaften / Department Psychologie |
Peer review: | Referiert |
Institution name at the time of the publication: | Humanwissenschaftliche Fakultät / Exzellenzbereich Kognitionswissenschaften |
Humanwissenschaftliche Fakultät / Institut für Psychologie |