Refine
Is part of the Bibliography
- yes (8)
Keywords
- large-scale assessment (3)
- motivation (3)
- Instructional quality (2)
- affect (2)
- design parameters (2)
- intraclass correlation (2)
- learning styles (2)
- multilevel models (2)
- propensity score matching (2)
- student achievement (2)
Institute
Für die Analyse der Unterrichtsqualität von Schulen durch Schülerurteile sollten drei Voraussetzungen erfüllt sein: (1) eine angemessene Übereinstimmung der Schülerurteile innerhalb der Schulen, (2) systematische Variabilität der Schülerurteile zwischen Schulen, (3) ein ausreichendes Maß an Reliabilität der aggregierten Urteile. Diese Studie untersucht mit internationalen PISA-Daten (Zyklen 2000–2012; 81 Länder, über 55.300 Schulen, über 1,3 Millionen 15-Jährige), inwiefern dies für Indikatoren der Qualitätsdimensionen des Unterrichts (Klassenführung, Kognitive Aktivierung, Konstruktive Unterstützung) zutrifft. Dafür bestimmten wir das Übereinstimmungsmaß rWG(J) sowie die Intraklassenkorrelationen ICC(1) und ICC(2). Es zeigte sich, dass (1) die Mehrzahl der Unterrichtsmerkmale eine moderate oder starke Übereinstimmung in Schulen aufwies, (2) sich Unterrichtsmerkmale aus Sicht der Schülerschaft systematisch zwischen Schulen unterschieden, jedoch (3) die Reliabilität der aggregierten Schülerurteile in vielen Ländern nicht ausreichte. Die Ergebnisse diskutieren wir vor dem Hintergrund von Konventionen zur Beurteilung der Übereinstimmung, Variabilität und Reliabilität auf Schulebene.
Despite the fact that grade retention is now seen as controversial in many quarters, it remains common practice in numerous countries. Previous research on the effects of grade retention on student development has, however, generated ambiguous results, particularly in terms of motivational outcomes. This ambiguity has been attributed in part to a lack of high-quality studies including a longitudinal design, a suitable comparison group, and adequate statistical control of preretention differences. Based on longitudinal data of N = 3,288 German students over 3 years of secondary school, we examined differences in their academic self-concept, scholarly interests, learning motivation, and achievement motivation between those being retained in the 6th grade (n = 61) and those of the same age being promoted annually. To account for confounding variables, we applied full propensity score matching on baseline measures of the dependent variables, as well as various other covariates that have been found to be associated with the risk of retention (e.g., cognitive ability, academic performance, and family background variables). Results reveal a steep decline in students’ academic self-concept, interests, and learning motivation during the last months spent in the original class, just before retention. For those measures that were available, negative effects were still partly significant after 1 year, but had diminished 2 years after grade retention. Contrary to predictions suggested by the big-fish-little-pond effect, we found no positive effects of retention on students’ academic self-concept.
Between-School Variation in Students' Achievement, Motivation, Affect, and Learning Strategies
(2017)
To plan group-randomized trials where treatment conditions are assigned to schools, researchers need design parameters that provide information about between-school differences in outcomes as well as the amount of variance that can be explained by covariates at the student (L1) and school (L2) levels. Most previous research has offered these parameters for U.S. samples and for achievement as the outcome. This paper and the online supplementary materials provide design parameters for 81 countries in three broad outcome categories (achievement, affect and motivation, and learning strategies) for domain-general and domain-specific (mathematics, reading, and science) measures. Sociodemographic characteristics were used as covariates. Data from representative samples of 15-year-old students stemmed from five cycles of the Programme for International Student Assessment (PISA; total number of students/schools: 1,905,147/70,098). Between-school differences as well as the amount of variance explained at L1 and L2 varied widely across countries and educational outcomes, demonstrating the limited generalizability of design parameters across these dimensions. The use of the design parameters to plan group-randomized trials is illustrated.
Quality of mathematics education has gained significant attention in educational politics and among educators as mathematics advances the foundations of analytical thinking necessary to excel in today’s knowledge-based economy. Recent research on instructional quality has focused on students’ development of competencies. Competency-based instruction is believed to be an effective approach to instruction as it is closely aligned to educational standards. We use data from the National Assessment Study 2012 in Germany and apply the theory of planned behavior to determine what motivates mathematics teachers (n = 1660) to take a competency-based approach to instruction. Results indicate that competencies outlined in the educational standards are a tangible element of current mathematics instruction. Within the framework of this study, we identified teachers’ perceived behavior control as the strongest determinant of taking a competency-based approach to instruction. We conclude that advancement of competency-based instruction depends on teachers’ beliefs about their professional resources.
Prior research has shown that quantity of schooling affects the development of intelligence in childhood and adolescence. However, it is still debated whether other aspects of schooling-such as ability tracking or, more generally, school quality-can also influence intelligence. In this study, the authors analyzed intelligence gains in academic- and vocational-track schools in Germany, testing for differential effects of school quality (academic vs. vocational track) on psychometric intelligence. Longitudinal data were obtained from a sample of N = 1,038 Grade 7 and 10 students in 49 schools. A nonverbal reasoning test was used as an indicator of general psychometric intelligence, and relevant psychological and social background variables were included in the analyses. Propensity score matching was used to control for selection bias. Results showed a positive effect of attending the academic track.
Equally able students have lower academic self-concepts in high-achieving classrooms than in low-achieving classrooms. This highly general and robust frame of reference effect is widely known as the Big-Fish-Little-Pond Effect (BFLPE; Marsh, 1987). This study contributes to research aiming to identify moderators of the BFLPE by investigating the effects of students' personality (i.e. Big Five traits and narcissism). Multilevel structural equation modeling was used to test the moderator hypotheses, drawing on data from a large sample of N= 4973 upper secondary track students (M age = 19.57). Consistent with a priori predictions, the negative effect of school-average achievement (the BFLPE) interacted significantly with narcissism. Students high in narcissism experienced smaller BFLPEs than did students with low or average levels of narcissism. The statistically significant effect for neuroticism acted in the opposite direction. The study illustrates how personality moderates frame of reference effects that are central to self-concept formation.
Between-school variation in students' achievement, motivation, affect, and learning strategies
(2017)
To plan group-randomized trials where treatment conditions are assigned to schools, researchers need design parameters that provide information about between-school differences in outcomes as well as the amount of variance that can be explained by covariates at the student (L1) and school (L2) levels. Most previous research has offered these parameters for U.S. samples and for achievement as the outcome. This paper and the online supplementary materials provide design parameters for 81 countries in three broad outcome categories (achievement, affect and motivation, and learning strategies) for domain-general and domain-specific (mathematics, reading, and science) measures. Sociodemographic characteristics were used as covariates. Data from representative samples of 15-year-old students stemmed from five cycles of the Programme for International Student Assessment (PISA; total number of students/schools: 1,905,147/70,098). Between-school differences as well as the amount of variance explained at L1 and L2 varied widely across countries and educational outcomes, demonstrating the limited generalizability of design parameters across these dimensions. The use of the design parameters to plan group-randomized trials is illustrated.
To plan cluster-randomized trials with sufficient statistical power to detect intervention effects on student achievement, researchers need multilevel design parameters, including measures of between-classroom and between-school differences and the amounts of variance explained by covariates at the student, classroom, and school level. Previous research has mostly been conducted in the United States, focused on two-level designs, and limited to core achievement domains (i.e., mathematics, science, reading). Using representative data of students attending grades 1-12 from three German longitudinal large-scale assessments (3,963 <= N <= 14,640), we used three- and two-level latent (covariate) models to provide design parameters and corresponding standard errors for a broad array of domain-specific (e.g., mathematics, science, verbal skills) and domain-general (e.g., basic cognitive functions) achievement outcomes. Three covariate sets were applied comprising (a) pretest scores, (b) sociodemographic characteristics, and (c) their combination. Design parameters varied considerably as a function of the hierarchical level, achievement outcome, and grade level. Our findings demonstrate the need to strive for an optimal fit between design parameters and target research context. We illustrate the application of design parameters in power analyses.