TY - JOUR A1 - Kontro, Inkeri A1 - Buschhüter, David T1 - Validity of Colorado Learning Attitudes about Science Survey for a high-achieving, Finnish population JF - Physical review. Physics education research N2 - The Colorado Learning Attitudes about Science Survey (CLASS) is an instrument which is widely used in physics education to characterize students' attitudes toward physics and learning physics and compare them with those of experts. While CLASS has been extensively validated for use in the context of higher education institutions in the United States, there has been less information about its use with European students. We have studied the structural, content, and substantive aspects of validity of CLASS by first doing a confirmatory factor analysis of N = 642 sets of student answers from the University of Helsinki, Finland. The students represented a culturally and demographically different subset of university physics students than in previous studies. The confirmatory factor analysis used a 3-factor, 15-item factor structure as a starting point and the resulting factor structure was similar to the original. Just minor modifications were needed for fit parameters to be in the acceptable range. We explored the differences by student interviews and consultation of experts. With the exception of one item, they supported the new 14-item, 3-factor structure. The results show that the interpretations made from CLASS results are mostly transferable, and CLASS remains a useful instrument for a wide variety of populations. Y1 - 2020 U6 - https://doi.org/10.1103/PhysRevPhysEducRes.16.020104 SN - 2469-9896 VL - 16 IS - 2 PB - American Physical Society CY - College Park, MD ER - TY - JOUR A1 - Vogelsang, Christoph A1 - Borowski, Andreas A1 - Buschhüter, David A1 - Enkrott, Patrick A1 - Kempin, Maren A1 - Kulgemeyer, Christoph A1 - Reinhold, Peter A1 - Riese, Josef A1 - Schecker, Horst A1 - Schröder, Jan T1 - Entwicklung von Professionswissen und Unterrichtsperformanz im Lehramtsstudium Physik T1 - Development of Professional Knowledge and Teaching Skills in Academic Pre-service Physics Teacher Education: Validity analyses concerning the interpretation of test scores BT - Analysen zu valider Testwertinterpretation JF - Zeitschrift für Pädagogik N2 - Angehende Physiklehrkräfte sollen im Rahmen ihres Studiums fachliches und fachdidaktisches Wissen erwerben, welches die Gestaltung lernförderlichen Unterrichts ermöglicht. Es ist allerdings empirisch nur wenig geklärt, wie sich dieses Wissen im Laufe des Studiums entwickelt und ob es zur Ausbildung von Handlungsfähigkeiten beiträgt. Um derartige Wirkungsaussagen treffen zu können, müssen Instrumente entwickelt werden, die eine valide Testwertinterpretation zulassen. In diesem Beitrag werden auf Basis von im Projekt Profile-P+ entwickelten Instrumenten Validitätsanalysen zur längsschnittlichen Entwicklung des Professionswissens von Physiklehramtsstudierenden im Verlauf des Bachelorstudiums und ihrer Fähigkeiten zur Planung und Reflexion von Physikunterricht sowie zum Erklären von physikalischen Sachverhalten vor und nach dem Praxissemester dargestellt. Neben Wissenstests kamen standardisierte Performanztests zum Einsatz. Die vorliegenden Ergebnisse sprechen dafür, dass die erhobenen Messwerte im Sinne von Wirkungsaussagen interpretiert werden können. N2 - In pre-service teacher education programs prospective physics teachers should acquire professional knowledge that enables them to carry out effective instruction. However, there is little empirical evidence with regard to the development of professional knowledge in the courses of their studies - it is even unclear what knowledge has an impact on teaching quality. In order to be able to analyse these questions, instruments must be developed that permit a valid interpretation of tests cores. Based on instruments developed in project Profile-P+, this article presents a validity argument for the interpretation of test scores for the development of pre-service physics teachers' professional knowledge during a bachelor degree program. The authors also develop validity arguments for the interpretation of tests cores for the development of their skills to plan and reflect on physics lessons and to explain physics during teaching practice. In addition to knowledge tests, standardized performance tests were used. The results of the analysis suggest that the measured values can be interpreted in the intended sense. KW - Physics KW - Teacher Education KW - Professional Knowledge KW - Performance Assessment KW - Longitudinal Analysis KW - Erhebungsinstrument KW - Längsschnittuntersuchung KW - Validität KW - Kompetenzerwerb KW - Test KW - Testauswertung KW - Lehrer KW - Lehramtsstudent KW - Schulpraktikum KW - Fachwissen KW - Unterrichtsgestaltung KW - Unterrichtsplanung KW - Fachdidaktik KW - Datenanalyse KW - Messung KW - Reflexion (Phil) KW - Physikunterricht KW - Interpretation KW - Konzeption KW - Deutschland Y1 - 2019 SN - 0044-3247 VL - 65 IS - 4 SP - 473 EP - 491 PB - Beltz CY - Weinheim ER - TY - JOUR A1 - Buschhüter, David A1 - Spoden, Christian A1 - Borowski, Andreas T1 - Physics knowledge of first semester physics students in Germany BT - a comparison of 1978 and 2013 cohorts JF - International journal of science education N2 - Over the last decades, the percentage of the age group choosing to pursue university studies has increased significantly across the world. At the same time, there are university teachers who believe that the standards have fallen. There is little research on whether students nowadays demonstrate knowledge or abilities similar to that of the preceding cohorts. However, in times of educational expansion, empirical evidence on student test performance is extremely helpful in evaluating how well educational systems cope with the increasing numbers of students. In this study, we compared a sample of 2322 physics freshmen from 2013 with another sample of 2718 physics freshmen from 1978 at universities in Germany with regard to their physics knowledge based on their results in the same entrance test. Previous results on mathematics knowledge and abilities in the same sample of students indicated that there was no severe decline in their average achievement. This paper compares the physics knowledge of the same two samples of students. Contrary to their mathematics results, their physics results showed a substantial decrease in physics knowledge as measured by the test. KW - University physics KW - entrance test KW - generational comparison Y1 - 2017 U6 - https://doi.org/10.1080/09500693.2017.1318457 SN - 0950-0693 SN - 1464-5289 VL - 39 IS - 9 SP - 1109 EP - 1132 PB - Routledge, Taylor & Francis Group CY - Abingdon ER - TY - THES A1 - Buschhüter, David T1 - Anforderungsrelevante mathematik- und physikbezogene Leistungsdispositionen von Physikanfängerinnen und - anfängern BT - Zeitlicher Wandel mathematischer und physikalischer Kentnisse und Fähigkeiten (1978/2013) und inkrementelle Validität von Physikbezogenen Leistungsdispositionen für Studienleistungen und Studienabbruch Y1 - 2017 ER - TY - JOUR A1 - Kulgemeyer, Christoph A1 - Borowski, Andreas A1 - Buschhüter, David A1 - Enkrott, Patrick A1 - Kempin, Maren A1 - Reinhold, Peter A1 - Riese, Josef A1 - Schecker, Horst A1 - Schröder, Jan A1 - Vogelsang, Christoph T1 - Professional knowledge affects action-related skills BT - the development of preservice physics teachers' explaining skills during a field experience JF - Journal of research in science teaching : the official journal of the National Association for Research in Science Teaching N2 - Professional knowledge is an important source of science teachers' actions in the classroom (e.g., personal professional content knowledge [pedagogical content knowledge, PCK] is the source of enacted PCK in the refined consensus model [RCM] for PCK). However, the evidence for this claim is ambiguous at best. This study applied a cross-lagged panel design to examine the relationship between professional knowledge and actions in one particular instructional situation: explaining physics. Pre- and post a field experience (one semester), 47 preservice physics teachers from four different universities were tested for their content knowledge (CK), PCK, pedagogical knowledge (PK), and action-related skills in explaining physics. The study showed that joint professional knowledge (the weighted sum of CK, PCK, and PK scores) at the beginning of the field experience impacted the development of explaining skills during the field experience (beta = .38**). We interpret this as a particular relationship between professional knowledge and science teachers' action-related skills (enacted PCK): professional knowledge is necessary for the development of explaining skills. That is evidence that personal PCK affects enacted PCK. In addition, field experiences are often supposed to bridge the theory-practice gap by transforming professional knowledge into instructional practice. Our results suggest that for field experiences to be effective, preservice teachers should start with profound professional knowledge. KW - enacted PCK KW - field experience KW - instructional explanation KW - instructional KW - quality KW - practicum KW - professional knowledge KW - school internship Y1 - 2020 U6 - https://doi.org/10.1002/tea.21632 SN - 0022-4308 SN - 1098-2736 VL - 57 IS - 10 SP - 1554 EP - 1582 PB - Wiley CY - Hoboken ER - TY - JOUR A1 - Wulff, Peter A1 - Buschhüter, David A1 - Westphal, Andrea A1 - Nowak, Anna A1 - Becker, Lisa A1 - Robalino, Hugo A1 - Stede, Manfred A1 - Borowski, Andreas T1 - Computer-based classification of preservice physics teachers’ written reflections JF - Journal of science education and technology N2 - Reflecting in written form on one's teaching enactments has been considered a facilitator for teachers' professional growth in university-based preservice teacher education. Writing a structured reflection can be facilitated through external feedback. However, researchers noted that feedback in preservice teacher education often relies on holistic, rather than more content-based, analytic feedback because educators oftentimes lack resources (e.g., time) to provide more analytic feedback. To overcome this impediment to feedback for written reflection, advances in computer technology can be of use. Hence, this study sought to utilize techniques of natural language processing and machine learning to train a computer-based classifier that classifies preservice physics teachers' written reflections on their teaching enactments in a German university teacher education program. To do so, a reflection model was adapted to physics education. It was then tested to what extent the computer-based classifier could accurately classify the elements of the reflection model in segments of preservice physics teachers' written reflections. Multinomial logistic regression using word count as a predictor was found to yield acceptable average human-computer agreement (F1-score on held-out test dataset of 0.56) so that it might fuel further development towards an automated feedback tool that supplements existing holistic feedback for written reflections with data-based, analytic feedback. KW - reflection KW - teacher professional development KW - hatural language KW - processing KW - machine learning Y1 - 2020 U6 - https://doi.org/10.1007/s10956-020-09865-1 SN - 1059-0145 SN - 1573-1839 VL - 30 IS - 1 SP - 1 EP - 15 PB - Springer CY - Dordrecht ER - TY - JOUR A1 - Wulff, Peter A1 - Buschhüter, David A1 - Westphal, Andrea A1 - Mientus, Lukas A1 - Nowak, Anna A1 - Borowski, Andreas T1 - Bridging the gap between qualitative and quantitative assessment in science education research with machine learning BT - a case for pretrained language models-based clustering JF - Journal of science education and technology N2 - Science education researchers typically face a trade-off between more quantitatively oriented confirmatory testing of hypotheses, or more qualitatively oriented exploration of novel hypotheses. More recently, open-ended, constructed response items were used to combine both approaches and advance assessment of complex science-related skills and competencies. For example, research in assessing science teachers' noticing and attention to classroom events benefitted from more open-ended response formats because teachers can present their own accounts. Then, open-ended responses are typically analyzed with some form of content analysis. However, language is noisy, ambiguous, and unsegmented and thus open-ended, constructed responses are complex to analyze. Uncovering patterns in these responses would benefit from more principled and systematic analysis tools. Consequently, computer-based methods with the help of machine learning and natural language processing were argued to be promising means to enhance assessment of noticing skills with constructed response formats. In particular, pretrained language models recently advanced the study of linguistic phenomena and thus could well advance assessment of complex constructs through constructed response items. This study examines potentials and challenges of a pretrained language model-based clustering approach to assess preservice physics teachers' attention to classroom events as elicited through open-ended written descriptions. It was examined to what extent the clustering approach could identify meaningful patterns in the constructed responses, and in what ways textual organization of the responses could be analyzed with the clusters. Preservice physics teachers (N = 75) were instructed to describe a standardized, video-recorded teaching situation in physics. The clustering approach was used to group related sentences. Results indicate that the pretrained language model-based clustering approach yields well-interpretable, specific, and robust clusters, which could be mapped to physics-specific and more general contents. Furthermore, the clusters facilitate advanced analysis of the textual organization of the constructed responses. Hence, we argue that machine learning and natural language processing provide science education researchers means to combine exploratory capabilities of qualitative research methods with the systematicity of quantitative methods. KW - Attention to classroom events KW - Noticing KW - NLP KW - ML Y1 - 2022 U6 - https://doi.org/10.1007/s10956-022-09969-w SN - 1059-0145 SN - 1573-1839 VL - 31 IS - 4 SP - 490 EP - 513 PB - Springer CY - Dordrecht ER -