TY  - JOUR
A1  - Vasishth, Shravan
A1  - Mertzen, Daniela
A1  - Jaeger, Lena A.
A1  - Gelman, Andrew
T1  - The statistical significance filter leads to overoptimistic expectations of replicability
JF  - Journal of memory and language
N2  - It is well-known in statistics (e.g., Gelman & Carlin, 2014) that treating a result as publishable just because the p-value is less than 0.05 leads to overoptimistic expectations of replicability. These effects get published, leading to an overconfident belief in replicability. We demonstrate the adverse consequences of this statistical significance filter by conducting seven direct replication attempts (268 participants in total) of a recent paper (Levy & Keller, 2013). We show that the published claims are so noisy that even non-significant results are fully compatible with them. We also demonstrate the contrast between such small-sample studies and a larger-sample study; the latter generally yields a less noisy estimate but also a smaller effect magnitude, which looks less compelling but is more realistic. We reiterate several suggestions from the methodology literature for improving current practices.
KW  - Type M error
KW  - Replicability
KW  - Surprisal
KW  - Expectation
KW  - Locality
KW  - Bayesian data analysis
KW  - Parameter estimation
Y1  - 2018
U6  - https://doi.org/10.1016/j.jml.2018.07.004
SN  - 0749-596X
SN  - 1096-0821
VL  - 103
SP  - 151
EP  - 175
PB  - Elsevier
CY  - San Diego
ER  - 
TY  - JOUR
A1  - Schlägel, Ulrike E.
A1  - Lewis, Mark A.
T1  - Robustness of movement models: can models bridge the gap between temporal scales of data sets and behavioural processes?
JF  - Journal of mathematical biology
KW  - Animal movement
KW  - Sampling rate
KW  - Resource selection
KW  - GPS data
KW  - Parameter estimation
KW  - Markov model
Y1  - 2016
U6  - https://doi.org/10.1007/s00285-016-1005-5
SN  - 0303-6812
SN  - 1432-1416
VL  - 73
SP  - 1691
EP  - 1726
PB  - Springer
CY  - Heidelberg
ER  - 
TY  - JOUR
A1  - Schlägel, Ulrike E.
A1  - Lewis, Mark A.
T1  - A framework for analyzing the robustness of movement models to variable step discretization
JF  - Journal of mathematical biology
N2  - When sampling animal movement paths, the frequency at which location measurements are attempted is a critical feature for data analysis. Important quantities derived from raw data, e.g. travel distance or sinuosity, can differ largely based on the temporal resolution of the data. Likewise, when movement models are fitted to data, parameter estimates have been demonstrated to vary with sampling rate. Thus, biological statements derived from such analyses can only be made with respect to the resolution of the underlying data, limiting extrapolation of results and comparison between studies. To address this problem, we investigate whether there are models that are robust against changes in temporal resolution. First, we propose a mathematically rigorous framework, in which we formally define robustness as a model property. We then use the framework for a thorough assessment of a range of basic random walk models, in which we also show how robustness relates to other probabilistic concepts. While we found robustness to be a strong condition met by few models only, we suggest a new method to extend models so as to make them robust. Our framework provides a new systematic, mathematically founded approach to the question if, and how, sampling rate of movement paths affects statistical inference.
KW  - Animal movement
KW  - Random walk
KW  - Sampling rate
KW  - Discretization
KW  - GPS data
KW  - Parameter estimation
Y1  - 2016
U6  - https://doi.org/10.1007/s00285-016-0969-5
SN  - 0303-6812
SN  - 1432-1416
VL  - 73
SP  - 815
EP  - 845
PB  - Springer
CY  - Heidelberg
ER  -