Refine
Has Fulltext
- no (2)
Document Type
- Article (2)
Language
- English (2)
Is part of the Bibliography
- yes (2)
Keywords
- Assessment (1)
- Programme for International Student (1)
- academic performance (1)
- acceleration (1)
- complex survey designs (1)
- educational large-scale assessments (1)
- elementary school (1)
- grade skipping (1)
- individual (1)
- meta-analysis (1)
- participant data (1)
- propensity score matching (1)
Institute
- Department Erziehungswissenschaft (2) (remove)
Descriptive analyses of socially important or theoretically interesting phenomena and trends are a vital component of research in the behavioral, social, economic, and health sciences.
Such analyses yield reliable results when using representative individual participant data (IPD) from studies with complex survey designs, including educational large-scale assessments (ELSAs) or social, health, and economic survey and panel studies. The meta-analytic integration of these results offers unique and novel research opportunities to provide strong empirical evidence of the consistency and generalizability of important phenomena and trends.
Using ELSAs as an example, this tutorial offers methodological guidance on how to use the two-stage approach to IPD meta-analysis to account for the statistical challenges of complex survey designs (e.g., sampling weights, clustered and missing IPD), first, to conduct descriptive analyses (Stage 1), and second, to integrate results with three-level meta-analytic and meta-regression models to take into account dependencies among effect sizes (Stage 2).
The two-stage approach is illustrated with IPD on reading achievement from the Programme for International Student Assessment (PISA). We demonstrate how to analyze and integrate standardized mean differences (e.g., gender differences), correlations (e.g., with students' socioeconomic status [SES]), and interactions between individual characteristics at the participant level (e.g., the interaction between gender and SES) across several PISA cycles.
All the datafiles and R scripts we used are available online. Because complex social, health, or economic survey and panel studies share many methodological features with ELSAs, the guidance offered in this tutorial is also helpful for synthesizing research evidence from these studies.
Using German data, we examined the effects of one specific type of acceleration-grade skipping-on academic performance. Prior research on the effects of acceleration has suffered from methodological restrictions, especially due to a lack of appropriate comparison groups and a priori measurements. For this reason, propensity score matching was applied in this analysis to minimize selection bias due to observed confounding variables. Various types of matching were attempted, and, in consideration of balancing the covariates, full matching was the final choice. We used data from the Berlin ELEMENT Study, analyzing, after matching, the information of 81 students who had skipped a grade over the course of elementary school and up to 1,668 nonaccelerated students who attended the same grade level as the accelerated students. Measurements took place 3 times between the 4th and 6th grades, including the assessment of reading, spelling, and mathematics performance. After matching, the results of between-group comparisons regarding performance indices showed no significant effects of skipping a grade, other than a small positive effect found on spelling performance. Theoretical implications and methodological limitations are discussed.