Refine
Has Fulltext
- no (4) (remove)
Document Type
- Article (4)
Is part of the Bibliography
- yes (4)
Keywords
- NLP (2)
- Attention to classroom events (1)
- Deep learning (1)
- ML (1)
- Noticing (1)
- Reflective writing (1)
- Reflexion (1)
- Reflexionsmodell (1)
- Reflexionsqualität (1)
- Reflexionstiefe (1)
Institute
Science education researchers typically face a trade-off between more quantitatively oriented confirmatory testing of hypotheses, or more qualitatively oriented exploration of novel hypotheses. More recently, open-ended, constructed response items were used to combine both approaches and advance assessment of complex science-related skills and competencies. For example, research in assessing science teachers' noticing and attention to classroom events benefitted from more open-ended response formats because teachers can present their own accounts. Then, open-ended responses are typically analyzed with some form of content analysis. However, language is noisy, ambiguous, and unsegmented and thus open-ended, constructed responses are complex to analyze. Uncovering patterns in these responses would benefit from more principled and systematic analysis tools. Consequently, computer-based methods with the help of machine learning and natural language processing were argued to be promising means to enhance assessment of noticing skills with constructed response formats. In particular, pretrained language models recently advanced the study of linguistic phenomena and thus could well advance assessment of complex constructs through constructed response items. This study examines potentials and challenges of a pretrained language model-based clustering approach to assess preservice physics teachers' attention to classroom events as elicited through open-ended written descriptions. It was examined to what extent the clustering approach could identify meaningful patterns in the constructed responses, and in what ways textual organization of the responses could be analyzed with the clusters. Preservice physics teachers (N = 75) were instructed to describe a standardized, video-recorded teaching situation in physics. The clustering approach was used to group related sentences. Results indicate that the pretrained language model-based clustering approach yields well-interpretable, specific, and robust clusters, which could be mapped to physics-specific and more general contents. Furthermore, the clusters facilitate advanced analysis of the textual organization of the constructed responses. Hence, we argue that machine learning and natural language processing provide science education researchers means to combine exploratory capabilities of qualitative research methods with the systematicity of quantitative methods.
Computer-based analysis of preservice teachers' written reflections could enable educational scholars to design personalized and scalable intervention measures to support reflective writing. Algorithms and technologies in the domain of research related to artificial intelligence have been found to be useful in many tasks related to reflective writing analytics such as classification of text segments. However, mostly shallow learning algorithms have been employed so far. This study explores to what extent deep learning approaches can improve classification performance for segments of written reflections. To do so, a pretrained language model (BERT) was utilized to classify segments of preservice physics teachers' written reflections according to elements in a reflection-supporting model. Since BERT has been found to advance performance in many tasks, it was hypothesized to enhance classification performance for written reflections as well. We also compared the performance of BERT with other deep learning architectures and examined conditions for best performance. We found that BERT outperformed the other deep learning architectures and previously reported performances with shallow learning algorithms for classification of segments of reflective writing. BERT starts to outperform the other models when trained on about 20 to 30% of the training data. Furthermore, attribution analyses for inputs yielded insights into important features for BERT's classification decisions. Our study indicates that pretrained language models such as BERT can boost performance for language-related tasks in educational contexts such as classification.
Reflexion wird als notwendig für die professionelle Entwicklung von Lehrer:innen und die Verbesserung von Unterricht angesehen, wenngleich aus theoretischer Sicht große Uneinigkeit über den Begriff selbst, den Reflexionsprozess und die damit verbundenen Kompetenzen herrscht.
Ziel dieser Arbeit war die Entwicklung, Untersuchung und Weiterentwicklung eines Reflexionsmodells mit einem theoriebasierten, klaren Konzept des Reflexionsprozesses und einem passenden Anspruch an die Reflexionsleistung der Reflektierenden. Grundlage für die empirische Untersuchung waren N = 132 Selbstreflexionstexte von N = 22 Studierenden aus dem Praxissemester Physik. Zur Codierung der Texte wurden vier mittels qualitativer Inhaltsanalyse entwickelte Manuale angewandt. Mit quantitativen Methoden wurden Zusammenhänge zwischen strukturellen Elementen, Begründungen, Inhalten und dem Qualitätsmerkmal Reflexionstiefe überprüft.
Es zeigte sich ein "Überhang an Negativität": negative Bewertungen, negative Reflexionsauslöser und negative Inhalte hängen signifikant positiv mit größerer Reflexionstiefe zusammen. Auf Grundlage der empirischen Ergebnisse wurde das Reflexionsmodell mit externaler und internaler Zielorientierung (REIZ) entwickelt. Zudem wurde darauf aufbauend eine Definition für Reflexionstiefe in vier Argumentationsclustern formuliert. Für die Lehrkräftebildung wird der in REIZ dargestellte differenzierte Ansatz der Zielorientierung von Reflexion empfohlen.
Reflecting in written form on one's teaching enactments has been considered a facilitator for teachers' professional growth in university-based preservice teacher education. Writing a structured reflection can be facilitated through external feedback. However, researchers noted that feedback in preservice teacher education often relies on holistic, rather than more content-based, analytic feedback because educators oftentimes lack resources (e.g., time) to provide more analytic feedback. To overcome this impediment to feedback for written reflection, advances in computer technology can be of use. Hence, this study sought to utilize techniques of natural language processing and machine learning to train a computer-based classifier that classifies preservice physics teachers' written reflections on their teaching enactments in a German university teacher education program. To do so, a reflection model was adapted to physics education. It was then tested to what extent the computer-based classifier could accurately classify the elements of the reflection model in segments of preservice physics teachers' written reflections. Multinomial logistic regression using word count as a predictor was found to yield acceptable average human-computer agreement (F1-score on held-out test dataset of 0.56) so that it might fuel further development towards an automated feedback tool that supplements existing holistic feedback for written reflections with data-based, analytic feedback.