TY - JOUR A1 - Wenger, Marina A1 - Lüdtke, Oliver A1 - Brunner, Martin T1 - Übereinstimmung, Variabilität und Reliabilität von Schülerurteilen zur Unterrichtsqualität auf Schulebene T1 - Interrater agreement, variability and reliability of student ratings of instructional quality at the school-level BT - Ergebnisse aus 81 Ländern BT - Results from 81 countries JF - Zeitschrift für Erziehungswissenschaft N2 - Für die Analyse der Unterrichtsqualität von Schulen durch Schülerurteile sollten drei Voraussetzungen erfüllt sein: (1) eine angemessene Übereinstimmung der Schülerurteile innerhalb der Schulen, (2) systematische Variabilität der Schülerurteile zwischen Schulen, (3) ein ausreichendes Maß an Reliabilität der aggregierten Urteile. Diese Studie untersucht mit internationalen PISA-Daten (Zyklen 2000–2012; 81 Länder, über 55.300 Schulen, über 1,3 Millionen 15-Jährige), inwiefern dies für Indikatoren der Qualitätsdimensionen des Unterrichts (Klassenführung, Kognitive Aktivierung, Konstruktive Unterstützung) zutrifft. Dafür bestimmten wir das Übereinstimmungsmaß rWG(J) sowie die Intraklassenkorrelationen ICC(1) und ICC(2). Es zeigte sich, dass (1) die Mehrzahl der Unterrichtsmerkmale eine moderate oder starke Übereinstimmung in Schulen aufwies, (2) sich Unterrichtsmerkmale aus Sicht der Schülerschaft systematisch zwischen Schulen unterschieden, jedoch (3) die Reliabilität der aggregierten Schülerurteile in vielen Ländern nicht ausreichte. Die Ergebnisse diskutieren wir vor dem Hintergrund von Konventionen zur Beurteilung der Übereinstimmung, Variabilität und Reliabilität auf Schulebene. N2 - Using student ratings to assess instructional quality of schools should fulfill three requirements: (1)an appropriate level of inter-rater agreement within schools, (2)systematic variance of student ratings between schools, (3)an adequate reliability level of aggregated student ratings. Using international PISA-data (2000-2012; 81countries, over 55,300 schools, over 1.3million 15-year olds) this study investigated how these requirements were met regarding indicators of instructional quality (classroom management, cognitive activation, individual learning support). We computed the interrater agreement index r(WG(J)), as well as the intraclass correlations ICC(1) and ICC(2). Our results showed that (1)student ratings demonstrated amoderate or strong level of agreement for most indicators of instructional quality and (2)instructional quality assessed by students varied systematically between schools. Yet, (3)reliability of aggregated student ratings was not sufficient in many countries. We discuss these results regarding conventions to evaluate agreement, variability, and reliability of student ratings at the school level. KW - Instructional quality KW - Student ratings KW - PISA Y1 - 2018 U6 - https://doi.org/10.1007/s11618-018-0813-3 SN - 1434-663X SN - 1862-5215 VL - 21 IS - 5 SP - 929 EP - 950 PB - Springer CY - Wiesbaden ER - TY - JOUR A1 - Lenski, Anna Eva A1 - Richter, Dirk A1 - Lüdtke, Oliver T1 - Using the theory of planned behavior to predict teachers’ likelihood of taking a competency-based approach to instruction JF - European Journal of Psychology of Education N2 - Quality of mathematics education has gained significant attention in educational politics and among educators as mathematics advances the foundations of analytical thinking necessary to excel in today’s knowledge-based economy. Recent research on instructional quality has focused on students’ development of competencies. Competency-based instruction is believed to be an effective approach to instruction as it is closely aligned to educational standards. We use data from the National Assessment Study 2012 in Germany and apply the theory of planned behavior to determine what motivates mathematics teachers (n = 1660) to take a competency-based approach to instruction. Results indicate that competencies outlined in the educational standards are a tangible element of current mathematics instruction. Within the framework of this study, we identified teachers’ perceived behavior control as the strongest determinant of taking a competency-based approach to instruction. We conclude that advancement of competency-based instruction depends on teachers’ beliefs about their professional resources. KW - Instructional quality KW - Educational standards KW - Mathematics instruction KW - Large-scale assessment KW - Theory of planned behavior Y1 - 2017 U6 - https://doi.org/10.1007/s10212-017-0356-7 SN - 0256-2928 SN - 1878-5174 VL - 34 IS - 1 SP - 169 EP - 186 PB - Springer CY - New York ER - TY - JOUR A1 - Kretschmann, Julia A1 - Vock, Miriam A1 - Lüdtke, Oliver A1 - Jansen, Malte A1 - Gronostaj, Anna T1 - Effects of grade retention on students’ motivation: A longitudinal study over 3 years of secondary school JF - The journal of educational psychology N2 - Despite the fact that grade retention is now seen as controversial in many quarters, it remains common practice in numerous countries. Previous research on the effects of grade retention on student development has, however, generated ambiguous results, particularly in terms of motivational outcomes. This ambiguity has been attributed in part to a lack of high-quality studies including a longitudinal design, a suitable comparison group, and adequate statistical control of preretention differences. Based on longitudinal data of N = 3,288 German students over 3 years of secondary school, we examined differences in their academic self-concept, scholarly interests, learning motivation, and achievement motivation between those being retained in the 6th grade (n = 61) and those of the same age being promoted annually. To account for confounding variables, we applied full propensity score matching on baseline measures of the dependent variables, as well as various other covariates that have been found to be associated with the risk of retention (e.g., cognitive ability, academic performance, and family background variables). Results reveal a steep decline in students’ academic self-concept, interests, and learning motivation during the last months spent in the original class, just before retention. For those measures that were available, negative effects were still partly significant after 1 year, but had diminished 2 years after grade retention. Contrary to predictions suggested by the big-fish-little-pond effect, we found no positive effects of retention on students’ academic self-concept. KW - grade retention KW - secondary school KW - academic self-concept KW - motivation KW - propensity score matching Y1 - 2019 U6 - https://doi.org/10.1037/edu0000353 SN - 0022-0663 SN - 1939-2176 VL - 111 IS - 8 SP - 1432 EP - 1446 PB - American Psychological Association CY - Washington ER - TY - JOUR A1 - Jonkmann, Kathrin A1 - Becker, Michael A1 - Marsh, Herbert W. A1 - Lüdtke, Oliver A1 - Trautwein, Ulrich T1 - Personality traits moderate the Big-Fish-Little-Pond Effect of academic self-concept JF - Learning and individual differences N2 - Equally able students have lower academic self-concepts in high-achieving classrooms than in low-achieving classrooms. This highly general and robust frame of reference effect is widely known as the Big-Fish-Little-Pond Effect (BFLPE; Marsh, 1987). This study contributes to research aiming to identify moderators of the BFLPE by investigating the effects of students' personality (i.e. Big Five traits and narcissism). Multilevel structural equation modeling was used to test the moderator hypotheses, drawing on data from a large sample of N= 4973 upper secondary track students (M age = 19.57). Consistent with a priori predictions, the negative effect of school-average achievement (the BFLPE) interacted significantly with narcissism. Students high in narcissism experienced smaller BFLPEs than did students with low or average levels of narcissism. The statistically significant effect for neuroticism acted in the opposite direction. The study illustrates how personality moderates frame of reference effects that are central to self-concept formation. KW - Big-Fish-Little-Pond Effect KW - Academic self-concept KW - Moderation KW - Personality Y1 - 2012 U6 - https://doi.org/10.1016/j.lindif.2012.07.020 SN - 1041-6080 VL - 22 IS - 6 SP - 736 EP - 746 PB - Elsevier CY - Amsterdam ER - TY - JOUR A1 - Brunner, Martin A1 - Keller, Ulrich A1 - Wenger, Marina A1 - Fischbach, Antoine A1 - Lüdtke, Oliver T1 - Between-School Variation in Students' Achievement, Motivation, Affect, and Learning Strategies BT - Results from 81 Countries for Planning Group-Randomized Trials in Education JF - Journal of research on educational effectiveness / Society for Research on Educational Effectiveness (SREE) N2 - To plan group-randomized trials where treatment conditions are assigned to schools, researchers need design parameters that provide information about between-school differences in outcomes as well as the amount of variance that can be explained by covariates at the student (L1) and school (L2) levels. Most previous research has offered these parameters for U.S. samples and for achievement as the outcome. This paper and the online supplementary materials provide design parameters for 81 countries in three broad outcome categories (achievement, affect and motivation, and learning strategies) for domain-general and domain-specific (mathematics, reading, and science) measures. Sociodemographic characteristics were used as covariates. Data from representative samples of 15-year-old students stemmed from five cycles of the Programme for International Student Assessment (PISA; total number of students/schools: 1,905,147/70,098). Between-school differences as well as the amount of variance explained at L1 and L2 varied widely across countries and educational outcomes, demonstrating the limited generalizability of design parameters across these dimensions. The use of the design parameters to plan group-randomized trials is illustrated. KW - student achievement KW - motivation KW - affect KW - learning styles KW - intraclass correlation KW - large-scale assessment KW - multilevel models KW - design parameters Y1 - 2017 U6 - https://doi.org/10.1080/19345747.2017.1375584 SN - 1934-5747 SN - 1934-5739 VL - 11 IS - 3 SP - 452 EP - 478 PB - Routledge, Taylor & Francis Group CY - Abingdon ER - TY - GEN A1 - Brunner, Martin A1 - Keller, Ulrich A1 - Wenger, Marina A1 - Fischbach, Antoine A1 - Lüdtke, Oliver T1 - Between-school variation in students' achievement, motivation, affect, and learning strategies BT - results from 81 countries for planning group-randomized trials in education T2 - Postprints der Universität Potsdam : Humanwissenschaftliche Reihe N2 - To plan group-randomized trials where treatment conditions are assigned to schools, researchers need design parameters that provide information about between-school differences in outcomes as well as the amount of variance that can be explained by covariates at the student (L1) and school (L2) levels. Most previous research has offered these parameters for U.S. samples and for achievement as the outcome. This paper and the online supplementary materials provide design parameters for 81 countries in three broad outcome categories (achievement, affect and motivation, and learning strategies) for domain-general and domain-specific (mathematics, reading, and science) measures. Sociodemographic characteristics were used as covariates. Data from representative samples of 15-year-old students stemmed from five cycles of the Programme for International Student Assessment (PISA; total number of students/schools: 1,905,147/70,098). Between-school differences as well as the amount of variance explained at L1 and L2 varied widely across countries and educational outcomes, demonstrating the limited generalizability of design parameters across these dimensions. The use of the design parameters to plan group-randomized trials is illustrated. T3 - Zweitveröffentlichungen der Universität Potsdam : Humanwissenschaftliche Reihe - 465 KW - student achievement KW - motivation KW - affect KW - learning styles KW - intraclass correlation KW - large-scale assessment KW - multilevel models KW - design parameters Y1 - 2018 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus4-412662 IS - 465 ER - TY - JOUR A1 - Becker, Michael A1 - Lüdtke, Oliver A1 - Trautwein, Ulrich A1 - Köller, Olaf A1 - Baumert, Jürgen T1 - The differential effects of school racking on psychometric intelligence Do academic-track schools make students smarter? JF - The journal of educational psychology N2 - Prior research has shown that quantity of schooling affects the development of intelligence in childhood and adolescence. However, it is still debated whether other aspects of schooling-such as ability tracking or, more generally, school quality-can also influence intelligence. In this study, the authors analyzed intelligence gains in academic- and vocational-track schools in Germany, testing for differential effects of school quality (academic vs. vocational track) on psychometric intelligence. Longitudinal data were obtained from a sample of N = 1,038 Grade 7 and 10 students in 49 schools. A nonverbal reasoning test was used as an indicator of general psychometric intelligence, and relevant psychological and social background variables were included in the analyses. Propensity score matching was used to control for selection bias. Results showed a positive effect of attending the academic track. KW - school quality KW - intelligence KW - cognitive development KW - longitudinal studies KW - propensity score matching Y1 - 2012 U6 - https://doi.org/10.1037/a0027608 SN - 0022-0663 VL - 104 IS - 3 SP - 682 EP - 699 PB - American Psychological Association CY - Washington ER -