TY - JOUR A1 - Brunner, Martin A1 - Keller, Ulrich A1 - Wenger, Marina A1 - Fischbach, Antoine A1 - Lüdtke, Oliver T1 - Between-School Variation in Students' Achievement, Motivation, Affect, and Learning Strategies BT - Results from 81 Countries for Planning Group-Randomized Trials in Education JF - Journal of research on educational effectiveness / Society for Research on Educational Effectiveness (SREE) N2 - To plan group-randomized trials where treatment conditions are assigned to schools, researchers need design parameters that provide information about between-school differences in outcomes as well as the amount of variance that can be explained by covariates at the student (L1) and school (L2) levels. Most previous research has offered these parameters for U.S. samples and for achievement as the outcome. This paper and the online supplementary materials provide design parameters for 81 countries in three broad outcome categories (achievement, affect and motivation, and learning strategies) for domain-general and domain-specific (mathematics, reading, and science) measures. Sociodemographic characteristics were used as covariates. Data from representative samples of 15-year-old students stemmed from five cycles of the Programme for International Student Assessment (PISA; total number of students/schools: 1,905,147/70,098). Between-school differences as well as the amount of variance explained at L1 and L2 varied widely across countries and educational outcomes, demonstrating the limited generalizability of design parameters across these dimensions. The use of the design parameters to plan group-randomized trials is illustrated. KW - student achievement KW - motivation KW - affect KW - learning styles KW - intraclass correlation KW - large-scale assessment KW - multilevel models KW - design parameters Y1 - 2017 U6 - https://doi.org/10.1080/19345747.2017.1375584 SN - 1934-5747 SN - 1934-5739 VL - 11 IS - 3 SP - 452 EP - 478 PB - Routledge, Taylor & Francis Group CY - Abingdon ER - TY - GEN A1 - Brunner, Martin A1 - Keller, Ulrich A1 - Wenger, Marina A1 - Fischbach, Antoine A1 - Lüdtke, Oliver T1 - Between-school variation in students' achievement, motivation, affect, and learning strategies BT - results from 81 countries for planning group-randomized trials in education T2 - Postprints der Universität Potsdam : Humanwissenschaftliche Reihe N2 - To plan group-randomized trials where treatment conditions are assigned to schools, researchers need design parameters that provide information about between-school differences in outcomes as well as the amount of variance that can be explained by covariates at the student (L1) and school (L2) levels. Most previous research has offered these parameters for U.S. samples and for achievement as the outcome. This paper and the online supplementary materials provide design parameters for 81 countries in three broad outcome categories (achievement, affect and motivation, and learning strategies) for domain-general and domain-specific (mathematics, reading, and science) measures. Sociodemographic characteristics were used as covariates. Data from representative samples of 15-year-old students stemmed from five cycles of the Programme for International Student Assessment (PISA; total number of students/schools: 1,905,147/70,098). Between-school differences as well as the amount of variance explained at L1 and L2 varied widely across countries and educational outcomes, demonstrating the limited generalizability of design parameters across these dimensions. The use of the design parameters to plan group-randomized trials is illustrated. T3 - Zweitveröffentlichungen der Universität Potsdam : Humanwissenschaftliche Reihe - 465 KW - student achievement KW - motivation KW - affect KW - learning styles KW - intraclass correlation KW - large-scale assessment KW - multilevel models KW - design parameters Y1 - 2018 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus4-412662 IS - 465 ER - TY - JOUR A1 - Schmidt, Isabelle A1 - Brunner, Martin A1 - Keller, Lena A1 - Scherrer, Vsevolod A1 - Wollschlager, Rachel A1 - Baudson, Tanja Gabriele A1 - Preckel, Franzis T1 - Profile formation of academic self-concept in elementary school students in grades 1 to 4 JF - PLoS one Y1 - 2017 U6 - https://doi.org/10.1371/journal.pone.0177854 SN - 1932-6203 VL - 12 PB - PLoS CY - San Fransisco ER - TY - JOUR A1 - Wenger, Marina A1 - Lüdtke, Oliver A1 - Brunner, Martin T1 - Übereinstimmung, Variabilität und Reliabilität von Schülerurteilen zur Unterrichtsqualität auf Schulebene T1 - Interrater agreement, variability and reliability of student ratings of instructional quality at the school-level BT - Ergebnisse aus 81 Ländern BT - Results from 81 countries JF - Zeitschrift für Erziehungswissenschaft N2 - Für die Analyse der Unterrichtsqualität von Schulen durch Schülerurteile sollten drei Voraussetzungen erfüllt sein: (1) eine angemessene Übereinstimmung der Schülerurteile innerhalb der Schulen, (2) systematische Variabilität der Schülerurteile zwischen Schulen, (3) ein ausreichendes Maß an Reliabilität der aggregierten Urteile. Diese Studie untersucht mit internationalen PISA-Daten (Zyklen 2000–2012; 81 Länder, über 55.300 Schulen, über 1,3 Millionen 15-Jährige), inwiefern dies für Indikatoren der Qualitätsdimensionen des Unterrichts (Klassenführung, Kognitive Aktivierung, Konstruktive Unterstützung) zutrifft. Dafür bestimmten wir das Übereinstimmungsmaß rWG(J) sowie die Intraklassenkorrelationen ICC(1) und ICC(2). Es zeigte sich, dass (1) die Mehrzahl der Unterrichtsmerkmale eine moderate oder starke Übereinstimmung in Schulen aufwies, (2) sich Unterrichtsmerkmale aus Sicht der Schülerschaft systematisch zwischen Schulen unterschieden, jedoch (3) die Reliabilität der aggregierten Schülerurteile in vielen Ländern nicht ausreichte. Die Ergebnisse diskutieren wir vor dem Hintergrund von Konventionen zur Beurteilung der Übereinstimmung, Variabilität und Reliabilität auf Schulebene. N2 - Using student ratings to assess instructional quality of schools should fulfill three requirements: (1)an appropriate level of inter-rater agreement within schools, (2)systematic variance of student ratings between schools, (3)an adequate reliability level of aggregated student ratings. Using international PISA-data (2000-2012; 81countries, over 55,300 schools, over 1.3million 15-year olds) this study investigated how these requirements were met regarding indicators of instructional quality (classroom management, cognitive activation, individual learning support). We computed the interrater agreement index r(WG(J)), as well as the intraclass correlations ICC(1) and ICC(2). Our results showed that (1)student ratings demonstrated amoderate or strong level of agreement for most indicators of instructional quality and (2)instructional quality assessed by students varied systematically between schools. Yet, (3)reliability of aggregated student ratings was not sufficient in many countries. We discuss these results regarding conventions to evaluate agreement, variability, and reliability of student ratings at the school level. KW - Instructional quality KW - Student ratings KW - PISA Y1 - 2018 U6 - https://doi.org/10.1007/s11618-018-0813-3 SN - 1434-663X SN - 1862-5215 VL - 21 IS - 5 SP - 929 EP - 950 PB - Springer CY - Wiesbaden ER - TY - JOUR A1 - Gärtner, Holger A1 - Brunner, Martin T1 - Once good teaching, always good teaching? BT - the differential stability of student perceptions of teaching quality JF - Educational Assessment, Evaluation and Accountability N2 - In many countries, students are asked about their perceptions of teaching in order to make decisions about the further development of teaching practices on the basis of this feedback. The stability of this measurement of teaching quality is a prerequisite for the ability to generalize the results to other teaching situations. The present study aims to expand the extant empirical body of knowledge on the effects of situational factors on the stability of students’ perceptions of teaching quality. Therefore, we investigate whether the degree of stability is moderated by three situational factors: time between assessments, subjects taught by teachers, and students’ grade levels. To this end, we analyzed data from a web-based student feedback system. The study involved 497 teachers, each of whom conducted two student surveys. We examined the differential stability of student perceptions of 16 teaching constructs that were operationalized as latent correlations between aggregated student perceptions of the same teacher’s teaching. Testing metric invariance indicated that student ratings provided measures of teaching constructs that were invariant across time, subjects, and grade levels. Stability was moderated to some extent by grade level but not by subjects taught nor time spacing between surveys. The results provide evidence of the extent to which situational factors may affect the stability of student perceptions of teaching constructs. The generalizability of the students’ feedback results to other teaching situations is discussed. KW - Stability KW - Student perception KW - Instruction KW - Generalizability KW - Situation Y1 - 2018 U6 - https://doi.org/10.1007/s11092-018-9277-5 SN - 1874-8597 SN - 1874-8600 VL - 30 IS - 2 SP - 159 EP - 182 PB - Springer CY - Heidelberg ER - TY - JOUR A1 - Heyder, Anke A1 - Brunner, Martin T1 - Teachers' aptitude beliefs as a predictor of helplessness in low-achieving students BT - Commonalities and differences between academic domains JF - Learning and individual differences N2 - Low-achieving students are at risk of experiencing a pattern of emotional, motivational, and cognitive deficits called school-related helplessness if they attribute their low achievement to low aptitude. Teachers' beliefs about the causes of students' low achievement are important sources of attributional information for students. In a sample of 2117 German ninth-graders attending the lowest track, 118 math and 129 German-language teachers, we tested whether teachers' beliefs about the extent to which aptitude causes achievement moderated the achievement-helplessness relation in students and whether there were differences between math and German. Multilevel analyses revealed that low prior achievement predicted higher helplessness in both subjects but the effect was stronger in math than in German. Teachers' beliefs amplified the achievement-helplessness relation in math but not in German. Results are discussed regarding domain-specific epistemological beliefs, and implications for research and practice are derived. KW - Helplessness KW - Teacher beliefs KW - Aptitude KW - Attribution theory KW - Domain differences Y1 - 2018 U6 - https://doi.org/10.1016/j.lindif.2018.01.015 SN - 1041-6080 SN - 1873-3425 VL - 62 SP - 118 EP - 127 PB - Elsevier CY - Amsterdam ER - TY - JOUR A1 - Thoren, Katharina A1 - Brunner, Martin T1 - Flächendeckende Implementation des Jahrgangsübergreifenden Lernens BT - Welche Typen gibt es und zeigen diese Unterschiede in der Schul- und Unterrichtsqualität? JF - Zeitschrift für Erziehungswissenschaft N2 - Bildungspolitische Reformen unterscheiden sich in der Breite, Tiefe und Nachhaltigkeit, mit der sie realisiert werden. Der vorliegende Beitrag beschäftigt sich mit diesem Thema am Beispiel der Umsetzung des Jahrgangsübergreifenden Lernens (JÜL) in Berlin. JÜL war eine der zentralen Innovationen bei der Neugestaltung des Schulanfangs. Vor diesem Hintergrund behandelt die erste Teilstudie, wie JÜL an Schulen in den Schuljahren 2007/08 bis 2015/16 implementiert wurde. Es wurden Daten der Berliner Schulstatistik zu einem Längsschnitt auf Schulebene zusammengefasst (N = 356). Latente Profilanalysen identifizieren sechs Implementationstypen, die sich in Zeitpunkt und Dauer der Umsetzung von JÜL unterscheiden. Hierbei diente der Anteil der JÜL-Klassen an den Klassen der Schulanfangsphase als Indikator. Die zweite Teilstudie analysiert Unterschiede in der Schul- und Unterrichtsqualität auf Grundlage von Daten aus der Berliner Schulinspektion (N = 282). Mittels Varianzanalysen (ANOVA) zeigen sich a) Unterschiede zugunsten der Schulen, die frühzeitig und dauerhaft JÜL umsetzten und b) Unterschiede zugunsten der Schulen, die in ihren JÜL-Klassen drei – im Vergleich zu zwei – Jahrgänge zusammenfassen. KW - Educational reform KW - Implementation success KW - Longitudinal analyses KW - Mixed-age learning Y1 - 2018 U6 - https://doi.org/10.1007/s11618-018-0841-z SN - 1434-663X SN - 1862-5215 VL - 22 IS - 2 SP - 279 EP - 300 PB - Springer CY - Wiesbaden ER - TY - JOUR A1 - Schmidt, Isabelle A1 - Brunner, Martin A1 - Preckel, Franzis T1 - Effects of achievement differences for internal/external frame of reference model investigations BT - A test of robustness of findings over diverse student samples JF - British journal of educational psychology N2 - Background Achievement in math and achievement in verbal school subjects are more strongly correlated than the respective academic self-concepts. The internal/external frame of reference model (I/E model; Marsh, 1986, Am. Educ. Res. J., 23, 129) explains this finding by social and dimensional comparison processes. We investigated a key assumption of the model that dimensional comparisons mainly depend on the difference in achievement between subjects. We compared correlations between subject-specific self-concepts of groups of elementary and secondary school students with or without achievement differences in the respective subjects. Aims The main goals were (1) to show that effects of dimensional comparisons depend to a large degree on the existence of achievement differences between subjects, (2) to demonstrate the generalizability of findings over different grade levels and self-concept scales, and (3) to test a rarely used correlation comparison approach (CCA) for the investigation of I/E model assumptions. Samples We analysed eight German elementary and secondary school student samples (grades 3–8) from three independent studies (Ns 326–878). Method Correlations between math and German self-concepts of students with identical grades in the respective subjects were compared with the correlation of self-concepts of students having different grades using Fisher's Z test for independent samples. Results In all samples, correlations between math self-concept and German self-concept were higher for students having identical grades than for students having different grades. Differences in median correlations had small effect sizes for elementary school students and moderate effect sizes for secondary school students. Conclusions Findings generalized over grades and indicated a developmental aspect in self-concept formation. The CCA complements investigations within I/E-research. KW - academic self-concept KW - frame of reference KW - elementary school students KW - dimensional comparisons KW - internal/external frame-of-reference model Y1 - 2017 U6 - https://doi.org/10.1111/bjep.12198 SN - 0007-0998 SN - 2044-8279 VL - 88 IS - 4 SP - 513 EP - 528 PB - Wiley CY - Hoboken ER - TY - JOUR A1 - Brandt, Naemi D. A1 - Becker, Michael A1 - Tetzner, Julia A1 - Brunner, Martin A1 - Kuhl, Poldi A1 - Maaz, Kai T1 - Personality across the lifespan exploring measurement invariance of a short Big Five Inventory from ages 11 to 84 JF - European journal of psychological assessment N2 - Personality is a relevant predictor for important life outcomes across the entire lifespan. Although previous studies have suggested the comparability of the measurement of the Big Five personality traits across adulthood, the generalizability to childhood is largely unknown. The present study investigated the structure of the Big Five personality traits assessed with the Big Five Inventory-SOEP Version (BFI-S; SOEP = Socio-Economic Panel) across a broad age range spanning 11-84 years. We used two samples of N = 1,090 children (52% female, M-age = 11.87) and N = 18,789 adults (53% female, M-age = 51.09), estimating a multigroup CFA analysis across four age groups (late childhood: 11-14 years; early adulthood: 17-30 years; middle adulthood: 31-60 years; late adulthood: 61-84 years). Our results indicated the comparability of the personality trait metric in terms of general factor structure, loading patterns, and the majority of intercepts across all age groups. Therefore, the findings suggest both a reliable assessment of the Big Five personality traits with the BFI-S even in late childhood and a vastly comparable metric across age groups. KW - personality traits KW - measurement invariance KW - ESEM KW - lifespan KW - late KW - childhood Y1 - 2018 U6 - https://doi.org/10.1027/1015-5759/a000490 SN - 1015-5759 SN - 2151-2426 VL - 36 IS - 1 SP - 162 EP - 173 PB - Hogrefe CY - Göttingen ER - TY - JOUR A1 - Levy, Jessica A1 - Brunner, Martin A1 - Keller, Ulrich A1 - Fischbach, Antoine T1 - Methodological issues in value-added modeling: an international review from 26 countries JF - Educational Assessment, Evaluation and Accountability N2 - Value-added (VA) modeling can be used to quantify teacher and school effectiveness by estimating the effect of pedagogical actions on students’ achievement. It is gaining increasing importance in educational evaluation, teacher accountability, and high-stakes decisions. We analyzed 370 empirical studies on VA modeling, focusing on modeling and methodological issues to identify key factors for improvement. The studies stemmed from 26 countries (68% from the USA). Most studies applied linear regression or multilevel models. Most studies (i.e., 85%) included prior achievement as a covariate, but only 2% included noncognitive predictors of achievement (e.g., personality or affective student variables). Fifty-five percent of the studies did not apply statistical adjustments (e.g., shrinkage) to increase precision in effectiveness estimates, and 88% included no model diagnostics. We conclude that research on VA modeling can be significantly enhanced regarding the inclusion of covariates, model adjustment and diagnostics, and the clarity and transparency of reporting. What is the added value from attending a certain school or being taught by a certain teacher? To answer this question, the value-added (VA) model was developed. In this model, the actual achievement attained by students attending a certain school or being taught by a certain teacher is juxtaposed with the achievement that is expected for students with the same background characteristics (e.g., pretest scores). To this end, the VA model can be used to compute a VA score for each school or teacher, respectively. If actual achievement is better than expected achievement, there is a positive effect (i.e., a positive VA score) of attending a certain school or being taught by a certain teacher. In other words, VA models have been developed to “make fair comparisons of the academic progress of pupils in different settings” (Tymms 1999, p. 27). Their aim is to operationalize teacher or school effectiveness objectively. Specifically, VA models are often used for accountability purposes and high-stakes decisions (e.g., to allocate financial or personal resources to schools or even to decide which teachers should be promoted or discharged). Consequently, VA modeling is a highly political topic, especially in the USA, where many states have implemented VA or VA-based models for teacher evaluation (Amrein-Beardsley and Holloway 2017; Kurtz 2018). However, this use for high-stakes decisions is highly controversial and researchers seem to disagree concerning the question if VA scores should be used for decision-making (Goldhaber 2015). For a more exhaustive discussion of the use of VA models for accountability reasons, see, for example, Scherrer (2011). Given the far-reaching impact of VA scores, it is surprising that there is scarcity of systematic reviews of how VA scores are computed, evaluated, and how this research is reported. To this end, we review 370 empirical studies from 26 countries to rigorously examine several key issues in VA modeling, involving (a) the statistical model (e.g., linear regression, multilevel model) that is used, (b) model diagnostics and reported statistical parameters that are used to evaluate the quality of the VA model, (c) the statistical adjustments that are made to overcome methodological challenges (e.g., measurement error of the outcome variables), and (d) the covariates (e.g., pretest scores, students’ sociodemographic background) that are used when estimating expected achievement. All this information is critical for meeting the transparency standards defined by the American Educational Research Association (AERA 2006). Transparency is vital for educational research in general and especially for highly consequential research, such as VA modeling. First, transparency is highly relevant for researchers. The clearer the description of the model, the easier it is to build upon the knowledge of previous research and to safeguard the potential for replicating previous results. Second, because decisions that are based on VA scores affect teachers’ lives and schools’ futures, not only educational agents but also the general public should be able to comprehend how these scores are calculated to allow for public scrutiny. Specifically, given that VA scores can have devastating consequences on teachers’ lives and on the students they teach, transparency is particularly important to evaluate the chosen methodology to compute VA models for a certain purpose. Such evaluations are essential to answer the question to what extent the quality of VA scores allows to base far-reaching decisions on these scores for accountability purposes. KW - Value-added modeling KW - Literature review KW - Primary and secondary education KW - Teacher effectiveness KW - School effectiveness Y1 - 2019 U6 - https://doi.org/10.1007/s11092-019-09303-w SN - 1874-8597 SN - 1874-8600 VL - 31 IS - 3 SP - 257 EP - 287 PB - Springer CY - Heidelberg ER - TY - JOUR A1 - Thoren, Katharina A1 - Hannover, Bettina A1 - Brunner, Martin T1 - Empirische Arbeit: Welche Schulen machen mit beim Jahrgangsübergreifenden Lernen? T1 - Which Schools Engage in Mixed-age Learning? BT - Demografische und leistungsbezogene Merkmale unterschiedlich reformbereiter Grundschulen in Berlin BT - Demographic and Achievement Related Characteristics of Schools According to Their Reform Efforts JF - Psychologie in Erziehung und Unterricht : Zeitschrift für Forschung und Praxis N2 - Im Schuljahr 2008/09 war Jahrgangsübergreifendes Lernen (JÜL) in der Berliner Schuleingangsphase verpflichtend eingeführt worden. Doch nicht alle Schulen übernahmen diese Reform. In dieser Studie untersuchen wir, inwiefern Schulen sich in Abhängigkeit davon, wie schnell und umfassend sie JÜL implementiert hatten, in Merkmalen ihrer Schülerschaft voneinander unterscheiden. Wir nahmen an, dass mit dem Ziel von JÜL, Heterogenität produktiv für das Lernen zu nutzen, die Reform für solche Schulen besonders attraktiv war, die eine heterogene Schülerschaft haben. Heterogenität wurde über die Anteile von Kindern mit (a) nichtdeutscher Erstsprache und (b) Lernmittelzuzahlungsbefreiung operationalisiert. Weiter wurde untersucht, ob sich die Leistungen der Kinder in Deutsch und Mathematik zwischen den Schulen unterschieden. Die Ergebnisse zeigen erwartungsgemäß, dass Schulen mit einer heterogenen Schülerschaft JÜL schnell und nachhaltig implementierten. Im zeitlichen Verlauf ließen sich, nach Kontrolle der Heterogenität der Schülerschaft, keine Leistungsunterschiede zwischen den Schulen feststellen. Die Ergebnisse werden hinsichtlich der Frage diskutiert, unter welchen Voraussetzungen Schulen Reformen implementieren und wie sich JÜL auf Bildungsergebnisse auswirken kann. KW - Mixed-age learning KW - implementation of school reform KW - openness to reform KW - ethnic student composition KW - socioeconomic student composition KW - Jahrgangsübergreifendes Lernen KW - Implementation von Schulreformen KW - Reformbereitschaft KW - Zusammensetzung Schülerschaft hinsichtlich Erstsprache KW - Zusammensetzung Schülerschaft hinsichtlich Lernmittelzuzahlungsbefreiung Y1 - 2019 U6 - https://doi.org/10.2378/peu2019.art03d SN - 0342-183X VL - 66 IS - 1 SP - 19 EP - 32 PB - Reinhardt CY - München ER - TY - JOUR A1 - Hasl, Andrea A1 - Kretschmann, Julia A1 - Richter, Dirk A1 - Voelkle, Manuel A1 - Brunner, Martin T1 - Investigating Core Assumptions of the "American Dream": Historical Related to Key Life Outcomes in Adulthood JF - Psychology and aging N2 - The present study examines how historical changes in the U.S. socioeconomic environment in the 20th century may have affected core assumptions of the "American Dream." Specifically, the authors examined whether such changes modulated the extent to which adolescents' intelligence (IQ), their grade point average (GPA), and their parents' socioeconomic status (SES) could predict key life outcomes in adulthood about 20 years later. The data stemmed from two representative U.S. birth cohorts of 15- and 16-year-olds who were born in the early 1960s (N = 3,040) and 1980s (N = 3,524) and who participated in the National Longitudinal Surveys of Youth (NLSY). Cohort differences were analyzed with respect to differences in average relations by means of multiple and logistic regression and for specific points in each outcome distribution by means of quantile regressions. In both cohorts, IQ, GPA, and parental SES predicted important educational, occupational, and health-related life-outcomes about 20 years later. Across historical time, the predictive utility of adolescent IQ and parental SES remained stable for the most part. Yet, the combined effects of social-ecological and socioeconomic changes may have increased the predictive utility (that is, the regression weights) of adolescent GPA for educational, occupational, and health outcomes over time for individuals who were born in the 1980s. Theoretical implications concerning adult development, aging, and late life inequality are discussed. (PsycINFO Database Record. KW - cohort differences KW - intelligence KW - grade point average KW - socioeconomic status KW - life span research Y1 - 2019 U6 - https://doi.org/10.1037/pag0000392 SN - 0882-7974 SN - 1939-1498 VL - 34 IS - 8 SP - 1055 EP - 1076 PB - American Psychological Association CY - Washington ER - TY - JOUR A1 - Breit, Moritz Lion A1 - Brunner, Martin A1 - Preckel, Franzis T1 - General intelligence and specific cognitive abilities in adolescence BT - tests of age differentiation, ability differentiation, and their interaction in two large samples JF - Developmental psychology N2 - Differentiation of intelligence refers to changes in the structure of intelligence that depend on individuals' level of general cognitive ability (ability differentiation hypothesis) or age (developmental differentiation hypothesis). The present article aimed to investigate ability differentiation, developmental differentiation, and their interaction with nonlinear factor analytic models in 2 studies. Study 1 was comprised of a nationally representative sample of 7,127 U.S. students (49.4% female; M-age = 14.51, SD = 1.42, range = 12.08-17.00) who completed the computerized adaptive version of the Armed Service Vocational Aptitude Battery. Study 2 analyzed the norming sample of the Berlin Intelligence Structure Test with 1,506 German students (44% female; M-age = 14.54, SD = 1.35, range = 10.00-18.42). Results of Study 1 supported the ability differentiation hypothesis but not the developmental differentiation hypothesis. Rather, the findings pointed to age-dedifferentiation (i.e., higher correlations between different abilities with increasing age). There was evidence for an interaction between age and ability differentiation, with greater ability differentiation found for older adolescents. Study 2 provided little evidence for ability differentiation but largely replicated the findings for age dedifferentiation and the interaction between age and ability differentiation. The present results provide insight into the complex dynamics underlying the development of intelligence structure during adolescence. Implications for the assessment of intelligence are discussed. KW - intelligence KW - ability differentiation KW - age differentiation KW - nonlinear KW - factor analysis KW - adolescence Y1 - 2020 U6 - https://doi.org/10.1037/dev0000876 SN - 0012-1649 SN - 1939-0599 VL - 56 IS - 2 SP - 364 EP - 384 PB - American Psychological Association CY - Washington ER - TY - JOUR A1 - Levy, Jessica A1 - Mussack, Dominic A1 - Brunner, Martin A1 - Keller, Ulrich A1 - Cardoso-Leite, Pedro A1 - Fischbach, Antoine T1 - Contrasting classical and machine learning approaches in the estimation of value-added scores in large-scale educational data JF - Frontiers in psychology N2 - There is no consensus on which statistical model estimates school value-added (VA) most accurately. To date, the two most common statistical models used for the calculation of VA scores are two classical methods: linear regression and multilevel models. These models have the advantage of being relatively transparent and thus understandable for most researchers and practitioners. However, these statistical models are bound to certain assumptions (e.g., linearity) that might limit their prediction accuracy. Machine learning methods, which have yielded spectacular results in numerous fields, may be a valuable alternative to these classical models. Although big data is not new in general, it is relatively new in the realm of social sciences and education. New types of data require new data analytical approaches. Such techniques have already evolved in fields with a long tradition in crunching big data (e.g., gene technology). The objective of the present paper is to competently apply these "imported" techniques to education data, more precisely VA scores, and assess when and how they can extend or replace the classical psychometrics toolbox. The different models include linear and non-linear methods and extend classical models with the most commonly used machine learning methods (i.e., random forest, neural networks, support vector machines, and boosting). We used representative data of 3,026 students in 153 schools who took part in the standardized achievement tests of the Luxembourg School Monitoring Program in grades 1 and 3. Multilevel models outperformed classical linear and polynomial regressions, as well as different machine learning models. However, it could be observed that across all schools, school VA scores from different model types correlated highly. Yet, the percentage of disagreements as compared to multilevel models was not trivial and real-life implications for individual schools may still be dramatic depending on the model type used. Implications of these results and possible ethical concerns regarding the use of machine learning methods for decision-making in education are discussed. KW - value-added modeling KW - school effectiveness KW - machine learning KW - model KW - comparison KW - longitudinal data Y1 - 2020 U6 - https://doi.org/10.3389/fpsyg.2020.02190 SN - 1664-1078 VL - 11 PB - Frontiers Research Foundation CY - Lausanne ER - TY - JOUR A1 - Wenger, Marina A1 - Gärtner, Holger A1 - Brunner, Martin T1 - To what extent are characteristics of a school's student body, instructional quality, school quality, and school achievement interrelated? JF - School effectiveness and school improvement N2 - The aim of educational policy should be to provide a good education to all students. Thus, a key question arises regarding the extent to which key characteristics of school composition (proportion of students with migration background, socioeconomic status [SES], prior school achievement, and achievement heterogeneity), instructional quality, school quality, and later school achievement are interrelated. The present study addressed this research question by examining school inspection data, official school statistics, and large-scale achievement data from all primary schools in Berlin, Germany (N = 343). The results of correlation and path analyses showed that school composition (average SES, average prior school achievement) predicted components of instructional quality (SES: classroom management, cognitive activation; achievement: cognitive activation, individual learning support). The relation between school composition characteristics and most components of school quality was close to zero. Contrary to our expectations, only the effect of school SES on later achievement was mediated by instructional quality. KW - school composition KW - instructional quality KW - school quality Y1 - 2020 U6 - https://doi.org/10.1080/09243453.2020.1754243 SN - 0924-3453 SN - 1744-5124 VL - 31 IS - 4 SP - 548 EP - 575 PB - Routledge, Taylor & Francis Group CY - Abingdon ER - TY - JOUR A1 - Richter, Eric A1 - Brunner, Martin A1 - Richter, Dirk T1 - Teacher educators’ task perception and its relationship to professional identity and teaching practice JF - Teaching and teacher education : an international journal of research and studies N2 - We assessed teacher educators? task perception and investigated its relationship with components of their professional identity and their teaching practice. Using data from 145 teacher educators, two different task perceptions were found: transmitters and facilitators. Teacher educators who were categorized as facilitator tend to demonstrate higher levels of self-efficacy, job satisfaction, constructivist beliefs about teaching and learning and use more effective teaching strategies. The findings demonstrate that teaching practices of teacher educators are rooted in their professional identity. ? 2021 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). KW - Teacher educator KW - Professional identity KW - Professional development KW - Teacher learning KW - Teacher education Y1 - 2021 U6 - https://doi.org/10.1016/j.tate.2021.103303 SN - 0742-051X VL - 101 PB - Elsevier CY - Amsterdam ER - TY - JOUR A1 - Breit, Moritz Lion A1 - Brunner, Martin A1 - Preckel, Franzis T1 - Age and ability differentiation in children BT - a review and empirical investigation JF - Developmental psychology N2 - Differentiation hypotheses concern changes in the structural organization of cognitive abilities that depend on the level of general intelligence (ability differentiation) or age (developmental differentiation). Part 1 of this article presents a review of the literature on ability and developmental differentiation effects in children, revealing the need for studies that examine both effects simultaneously in this age group with appropriate statistical methods. Part 2 presents an empirical study in which nonlinear factor analytic models were applied to the standardization sample (N = 2,619 German elementary schoolchildren; 48% female; age: M = 8.8 years, SD = 1.2, range 6-12 years) of the THINK 1-4 intelligence test to investigate ability differentiation, developmental differentiation, and their interaction. The sample was nationally representative regarding age, gender, urbanization, and geographic location of residence but not regarding parents' education and migration background (overrepresentation of children with more educated parents, underrepresentation of children with migration background). The results showed no consistent evidence for the presence of differentiation effects or their interaction. Instead, different patterns were observed for figural, numerical, and verbal reasoning. Implications for the construction of intelligence tests, the assessment of intelligence in children, and for theories of cognitive development are discussed. KW - intelligence KW - ability differentiation KW - age differentiation KW - SLODR KW - childhood Y1 - 2021 U6 - https://doi.org/10.1037/dev0001147 SN - 0012-1649 SN - 1939-0599 VL - 57 IS - 3 SP - 325 EP - 346 PB - American Psychological Association CY - Richmond, Va. [u.a.] ER - TY - JOUR A1 - Keller, Lena A1 - Preckel, Franzis A1 - Brunner, Martin T1 - Nonlinear relations between achievement and academic self-concepts in elementary and secondary school BT - an integrative data analysis across 13 countries JF - Journal of educational psychology / American Psychological Association N2 - It is well-documented that academic achievement is associated with students' self-perceptions of their academic abilities, that is, their academic self-concepts. However, low-achieving students may apply self-protective strategies to maintain a favorable academic self-concept when evaluating their academic abilities. Consequently, the relation between achievement and academic self-concept might not be linear across the entire achievement continuum. Capitalizing on representative data from three large-scale assessments (i.e., TIMSS, PIRLS, PISA; N = 470,804), we conducted an integrative data analysis to address nonlinear trends in the relations between achievement and the corresponding self-concepts in mathematics and the verbal domain across 13 countries and 2 age groups (i.e., elementary and secondary school students). Polynomial and interrupted regression analyses showed nonlinear relations in secondary school students, demonstrating that the relations between achievement and the corresponding self-concepts were weaker for lower achieving students than for higher achieving students. Nonlinear effects were also present in younger students, but the pattern of results was rather heterogeneous. We discuss implications for theory as well as for the assessment and interpretation of self-concept. KW - academic achievement KW - academic self-concept KW - mathematics KW - reading KW - nonlinear relations Y1 - 2021 U6 - https://doi.org/10.1037/edu0000533 SN - 0022-0663 SN - 1939-2176 VL - 113 IS - 3 SP - 585 EP - 604 PB - American Psychological Association CY - Washington ER - TY - JOUR A1 - Bruckmaier, Georg A1 - Krauss, Stefan A1 - Binder, Karin A1 - Hilbert, Sven Lars A1 - Brunner, Martin T1 - Tversky and Kahneman’s cognitive illusions BT - who can solve them, and why? JF - Frontiers in psychology N2 - In the present paper we empirically investigate the psychometric properties of some of the most famous statistical and logical cognitive illusions from the "heuristics and biases" research program by Daniel Kahneman and Amos Tversky, who nearly 50 years ago introduced fascinating brain teasers such as the famous Linda problem, the Wason card selection task, and so-called Bayesian reasoning problems (e.g., the mammography task). In the meantime, a great number of articles has been published that empirically examine single cognitive illusions, theoretically explaining people's faulty thinking, or proposing and experimentally implementing measures to foster insight and to make these problems accessible to the human mind. Yet these problems have thus far usually been empirically analyzed on an individual-item level only (e.g., by experimentally comparing participants' performance on various versions of one of these problems). In this paper, by contrast, we examine these illusions as a group and look at the ability to solve them as a psychological construct. Based on an sample of N = 2,643 Luxembourgian school students of age 16-18 we investigate the internal psychometric structure of these illusions (i.e., Are they substantially correlated? Do they form a reflexive or a formative construct?), their connection to related constructs (e.g., Are they distinguishable from intelligence or mathematical competence in a confirmatory factor analysis?), and the question of which of a person's abilities can predict the correct solution of these brain teasers (by means of a regression analysis). KW - statistical reasoning KW - logical thinking KW - cognitive illusion KW - Monty Hall KW - problem KW - Wason task KW - Linda problem KW - hospital problem KW - Bayesian reasoning Y1 - 2021 U6 - https://doi.org/10.3389/fpsyg.2021.584689 SN - 1664-1078 VL - 12 PB - Frontiers Research Foundation CY - Lausanne ER - TY - JOUR A1 - Hasl, Andrea A1 - Voelkle, Manuel A1 - Kretschmann, Julia A1 - Richter, Dirk A1 - Brunner, Martin T1 - A dynamic structural equation approach to modeling wage dynamics and cumulative advantage across the lifespan JF - Multivariate Behavioral Research N2 - Wages and wage dynamics directly affect individuals' and families' daily lives. In this article, we show how major theoretical branches of research on wages and inequality-that is, cumulative advantage (CA), human capital theory, and the lifespan perspective-can be integrated into a coherent statistical framework and analyzed with multilevel dynamic structural equation modeling (DSEM). This opens up a new way to empirically investigate the mechanisms that drive growing inequality over time. We demonstrate the new approach by making use of longitudinal, representative U.S. data (NLSY-79). Analyses revealed fundamental between-person differences in both initial wages and autoregressive wage growth rates across the lifespan. Only 0.5% of the sample experienced a "strict" CA and unbounded wage growth, whereas most individuals revealed logarithmic wage growth over time. Adolescent intelligence and adult educational levels explained substantial heterogeneity in both parameters. We discuss how DSEM may help researchers study CA processes and related developmental dynamics, and we highlight the extensions and limitations of the DSEM framework. KW - Dynamic Structural Equation Modeling (DSEM) KW - wage dynamics KW - cumulative advantage (CA) KW - autoregressive wage growth KW - human capital theory Y1 - 2022 U6 - https://doi.org/10.1080/00273171.2022.2029339 SN - 0027-3171 SN - 1532-7906 VL - 58 IS - 3 SP - 504 EP - 525 PB - Routledge, Taylor & Francis Group CY - Abingdon ER - TY - JOUR A1 - Brunner, Martin A1 - Keller, Lena A1 - Stallasch, Sophie E. A1 - Kretschmann, Julia A1 - Hasl, Andrea A1 - Preckel, Franzis A1 - Luedtke, Oliver A1 - Hedges, Larry T1 - Meta-analyzing individual participant data from studies with complex survey designs BT - a tutorial on using the two-stage approach for data from educational large-scale assessments JF - Research synthesis methods N2 - Descriptive analyses of socially important or theoretically interesting phenomena and trends are a vital component of research in the behavioral, social, economic, and health sciences. Such analyses yield reliable results when using representative individual participant data (IPD) from studies with complex survey designs, including educational large-scale assessments (ELSAs) or social, health, and economic survey and panel studies. The meta-analytic integration of these results offers unique and novel research opportunities to provide strong empirical evidence of the consistency and generalizability of important phenomena and trends. Using ELSAs as an example, this tutorial offers methodological guidance on how to use the two-stage approach to IPD meta-analysis to account for the statistical challenges of complex survey designs (e.g., sampling weights, clustered and missing IPD), first, to conduct descriptive analyses (Stage 1), and second, to integrate results with three-level meta-analytic and meta-regression models to take into account dependencies among effect sizes (Stage 2). The two-stage approach is illustrated with IPD on reading achievement from the Programme for International Student Assessment (PISA). We demonstrate how to analyze and integrate standardized mean differences (e.g., gender differences), correlations (e.g., with students' socioeconomic status [SES]), and interactions between individual characteristics at the participant level (e.g., the interaction between gender and SES) across several PISA cycles. All the datafiles and R scripts we used are available online. Because complex social, health, or economic survey and panel studies share many methodological features with ELSAs, the guidance offered in this tutorial is also helpful for synthesizing research evidence from these studies. KW - complex survey designs KW - educational large-scale assessments KW - individual KW - participant data KW - meta-analysis KW - Programme for International Student KW - Assessment Y1 - 2022 U6 - https://doi.org/10.1002/jrsm.1584 SN - 1759-2879 SN - 1759-2887 VL - 14 IS - 1 SP - 5 EP - 35 PB - Wiley CY - Hoboken ER -