TY - GEN A1 - Knigge, Michel T1 - Use of evidence to promote inclusive education development commentary on Mel Ainscow. Promoting inclusion and equity in education BT - Lessons from international experiences T2 - Zweitveröffentlichungen der Universität Potsdam : Humanwissenschaftliche Reihe N2 - In his essay, Mel Ainscow looks at inclusion and equity from an international perspective and makes suggestions on how to develop inclusive education in a ‘whole-system approach’. After discussing different conceptions of inclusion and equity, he describes international policies which address them. From this international macro-level, Ainscow zooms in to the meso-level of the school and its immediate environment, defining dimensions to be considered for an inclusive school development. One of these dimensions is the ‘use of evidence’. In my comment, I want to focus on this dimension and discuss its scope and the potential to apply it in inclusive education development. As a first and important precondition, Ainscow explains that different circumstances lead to different linguistic uses of the term ‘inclusive education’. Thus, the term ‘inclusive education’ does not refer to an identical set of objectives across countries, and neither does the term ‘equity’. T3 - Zweitveröffentlichungen der Universität Potsdam : Humanwissenschaftliche Reihe - 872 KW - evidence KW - inclusion KW - education KW - evaluation KW - practice Y1 - 2020 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus4-519142 SN - 1866-8364 IS - 1 ER - TY - JOUR A1 - Knigge, Michel T1 - Use of evidence to promote inclusive education development commentary on Mel Ainscow. Promoting inclusion and equity in education BT - Lessons from international experiences JF - Nordic Journal of Studies in Educational Policy N2 - In his essay, Mel Ainscow looks at inclusion and equity from an international perspective and makes suggestions on how to develop inclusive education in a ‘whole-system approach’. After discussing different conceptions of inclusion and equity, he describes international policies which address them. From this international macro-level, Ainscow zooms in to the meso-level of the school and its immediate environment, defining dimensions to be considered for an inclusive school development. One of these dimensions is the ‘use of evidence’. In my comment, I want to focus on this dimension and discuss its scope and the potential to apply it in inclusive education development. As a first and important precondition, Ainscow explains that different circumstances lead to different linguistic uses of the term ‘inclusive education’. Thus, the term ‘inclusive education’ does not refer to an identical set of objectives across countries, and neither does the term ‘equity’. KW - evidence KW - inclusion KW - education KW - evaluation KW - practice Y1 - 2020 U6 - https://doi.org/10.1080/20020317.2020.1730093 SN - 2002-0317 VL - 6 IS - 1 SP - 21 EP - 24 PB - Taylor & Francis Group CY - London ER - TY - CHAP A1 - Rojahn, Marcel A1 - Gronau, Norbert ED - Bui, Tung X. T1 - Openness indicators for the evaluation of digital platforms between the launch and maturity phase T2 - Proceedings of the 57th Annual Hawaii International Conference on System Sciences N2 - In recent years, the evaluation of digital platforms has become an important focus in the field of information systems science. The identification of influential indicators that drive changes in digital platforms, specifically those related to openness, is still an unresolved issue. This paper addresses the challenge of identifying measurable indicators and characterizing the transition from launch to maturity in digital platforms. It proposes a systematic analytical approach to identify relevant openness indicators for evaluation purposes. The main contributions of this study are the following (1) the development of a comprehensive procedure for analyzing indicators, (2) the categorization of indicators as evaluation metrics within a multidimensional grid-box model, (3) the selection and evaluation of relevant indicators, (4) the identification and assessment of digital platform architectures during the launch-to-maturity transition, and (5) the evaluation of the applicability of the conceptualization and design process for digital platform evaluation. KW - federated industrial platform ecosystems KW - technologies KW - business models KW - data-driven artifacts KW - design-science research KW - digital platform openness KW - evaluation KW - morphological analysis Y1 - 2024 SN - 978-0-99813-317-1 SP - 4516 EP - 4525 PB - Department of IT Management Shidler College of Business University of Hawaii CY - Honolulu, HI ER - TY - CHAP A1 - Grum, Marcus A1 - Klippert, Monika A1 - Albers, Albert A1 - Gronau, Norbert A1 - Thim, Christof T1 - Examining the quality of knowledge transfers BT - the draft of an empirical research T2 - Proceedings of the Design Society N2 - Already successfully used products or designs, past projects or our own experiences can be the basis for the development of new products. As reference products or existing knowledge, it is reused in the development process and across generations of products. Since further, products are developed in cooperation, the development of new product generations is characterized by knowledge-intensive processes in which information and knowledge are exchanged between different kinds of knowledge carriers. The particular knowledge transfer here describes the identification of knowledge, its transmission from the knowledge carrier to the knowledge receiver, and its application by the knowledge receiver, which includes embodied knowledge of physical products. Initial empirical findings of the quantitative effects regarding the speed of knowledge transfers already have been examined. However, the factors influencing the quality of knowledge transfer to increase the efficiency and effectiveness of knowledge transfer in product development have not yet been examined empirically. Therefore, this paper prepares an experimental setting for the empirical investigation of the quality of knowledge transfers. KW - knowledge management KW - new product development KW - evaluation Y1 - 2021 U6 - https://doi.org/10.1017/pds.2021.404 SN - 2732-527X VL - 1 SP - 1431 EP - 1440 PB - Cambridge University Press CY - Cambridge ER - TY - JOUR A1 - Wilson, Charlie A1 - Guivarch, Céline A1 - Kriegler, Elmar A1 - van Ruijven, Bas A1 - van Vuuren, Detlef P. A1 - Krey, Volker A1 - Schwanitz, Valeria Jana A1 - Thompson, Erica L. T1 - Evaluating process-based integrated assessment models of climate change mitigation JF - Climatic change N2 - Process-based integrated assessment models (IAMs) project long-term transformation pathways in energy and land-use systems under what-if assumptions. IAM evaluation is necessary to improve the models’ usefulness as scientific tools applicable in the complex and contested domain of climate change mitigation. We contribute the first comprehensive synthesis of process-based IAM evaluation research, drawing on a wide range of examples across six different evaluation methods including historical simulations, stylised facts, and model diagnostics. For each evaluation method, we identify progress and milestones to date, and draw out lessons learnt as well as challenges remaining. We find that each evaluation method has distinctive strengths, as well as constraints on its application. We use these insights to propose a systematic evaluation framework combining multiple methods to establish the appropriateness, interpretability, credibility, and relevance of process-based IAMs as useful scientific tools for informing climate policy. We also set out a programme of evaluation research to be mainstreamed both within and outside the IAM community. KW - process-based integrated assessment model KW - IAM KW - evaluation KW - climate mitigation Y1 - 2021 U6 - https://doi.org/10.1007/s10584-021-03099-9 SN - 0165-0009 SN - 1573-1480 VL - 166 IS - 1-2 PB - Springer Science + Business Media B.V. CY - Dordrecht ER - TY - BOOK A1 - Hermanns, Jolanda A1 - Böhme, Katrin A1 - Meyering, Meike A1 - Fuchs, Isabelle A1 - Wagner, Simon A1 - Krauskopf, Karsten A1 - Knigge, Michel A1 - Rother, Stefanie A1 - Tosch, Frank A1 - Wendland, Mirko A1 - Wulff, Peter A1 - Mientus, Lukas A1 - Nowak, Anna A1 - Borowski, Andreas A1 - Baer, Ella A1 - Bosch, Jannis A1 - Wilbert, Jürgen A1 - Bräsel, Tim A1 - Fenn, Monika A1 - Kortenkamp, Ulrich A1 - Kuzle, Ana A1 - Reitz-Koncebovski, Karen A1 - Burg, Paula A1 - Lampart, Fabian A1 - Leubner, Martin A1 - Freitag-Hild, Britta A1 - Bitmann, Anna A1 - Reinhardt, Susanne A1 - Roos, Jana A1 - Hußner, Isabell A1 - Börner, Dustin A1 - Lazarides, Rebecca A1 - Glowinski, Ingrid A1 - Autenrieth, Marijke A1 - Radke, Thea A1 - Ehlert, Antje A1 - Menke, Anne A1 - Haupenthal, Anna A1 - Schramm, Satyam Antonio A1 - Kruse, Julia A1 - Körner, Dorothea A1 - Fischer, Jakob Thomas A1 - Kayser, Daniela Niesta ED - Hermanns, Jolanda T1 - PSI-Potsdam BT - Ergebnisbericht zu den Aktivitäten im Rahmen der Qualitätsoffensive Lehrerbildung (2019-2023) T3 - Potsdamer Beiträge für Lehrkräftebildung und Bildungsforschung N2 - An der Universität Potsdam wird seit 2015 im Rahmen der „Qualitätsoffensive Lehrerbildung“ das Projekt „Professionalisierung – Schulpraktische Studien – Inklusion“ (PSI-Potsdam) durchgeführt und am Zentrum für Lehrerbildung und Bildungsforschung (ZeLB) koordiniert. Zur ersten Projektförderphase (2015-2018) erschien der Band „PSI-Potsdam – Ergebnisbericht zu den Aktivitäten im Rahmen der Qualitätsoffensive Lehrerbildung (2015-2018)“ zum Auftakt der Reihe „Potsdamer Beiträge zur Lehrerbildung und Bildungsforschung“. Der vorliegende Band aus der gleichen Reihe gibt in den Kapiteln „Erhebungen“, „Lehrkonzepte“ und „Vernetzungen“ einen Überblick über alle Teilprojekte der zweiten Projektförderphase (2019-2023). Wissenschaftler:innen aus verschiedenen Fachdidaktiken, Fachwissenschaften sowie aus den Bildungswissenschaften und der Inklusionspädagogik haben im Rahmen des Projektes kooperiert. Sowohl praxisnahe Forschung als auch die Entwicklung neuer Lehrkonzepte sowie Strategien zur Vernetzung innerhalb der Lehrkräftebildung stehen im Fokus dieses Bandes. Die Praxisphasen, die im Rahmen des „Potsdamer Modells der Lehrerbildung“ eine zentrale Rolle spielen, wurden in einer großen Studie über alle Praxisphasen untersucht. Der Band gibt interessante Einblicke in die Ergebnisse der Teilprojekte und Anregungen sowohl für die eigene Forschung als auch für Entwicklungsarbeit wie zum Beispiel die Entwicklung neuer Lehrkonzepte. Herausgegeben wird dieser Band von PD Dr. Jolanda Hermanns (Gesamtkoordinatorin PSI-Potsdam und Chemiedidaktikerin). T3 - Potsdamer Beiträge zur Lehrkräftebildung und Bildungsforschung - 3 KW - Lehrerbildung KW - Evaluation KW - Testinstrumente KW - Konzepte KW - Vernetzung KW - teacher education KW - evaluation KW - test instruments KW - concepts KW - networking Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus4-601875 SN - 978-3-86956-568-2 SN - 2626-3556 SN - 2626-4722 IS - 3 PB - Universitätsverlag Potsdam CY - Potsdam ER - TY - JOUR A1 - Ghahremani, Sona A1 - Giese, Holger T1 - Evaluation of self-healing systems BT - An analysis of the state-of-the-art and required improvements JF - Computers N2 - Evaluating the performance of self-adaptive systems is challenging due to their interactions with often highly dynamic environments. In the specific case of self-healing systems, the performance evaluations of self-healing approaches and their parameter tuning rely on the considered characteristics of failure occurrences and the resulting interactions with the self-healing actions. In this paper, we first study the state-of-the-art for evaluating the performances of self-healing systems by means of a systematic literature review. We provide a classification of different input types for such systems and analyse the limitations of each input type. A main finding is that the employed inputs are often not sophisticated regarding the considered characteristics for failure occurrences. To further study the impact of the identified limitations, we present experiments demonstrating that wrong assumptions regarding the characteristics of the failure occurrences can result in large performance prediction errors, disadvantageous design-time decisions concerning the selection of alternative self-healing approaches, and disadvantageous deployment-time decisions concerning parameter tuning. Furthermore, the experiments indicate that employing multiple alternative input characteristics can help with reducing the risk of premature disadvantageous design-time decisions. KW - self-healing KW - failure model KW - performance KW - simulation KW - evaluation Y1 - 2020 U6 - https://doi.org/10.3390/computers9010016 SN - 2073-431X VL - 9 IS - 1 PB - MDPI CY - Basel ER - TY - JOUR A1 - Sallen, Jeffrey A1 - Hemming, Karen A1 - Richartz, Alfred T1 - Facilitating dual careers by improving resistance to chronic stress BT - effects of an intervention programme for elite student athletes JF - European journal of sport science : official journal of the European College of Sport Science N2 - The starting point of this contribution is the potential risk to health and performance from the combination of elite sporting careers with the pursuit of education. In European sport science and politics, structural measures to promote dual careers in elite sports have been discussed increasingly of late. In addition to organisational measures, there are calls for educational-psychological intervention programmes supporting the successful management of dual careers at the individual level. This paper presents an appropriate intervention programme and its evaluation: stress-resistance training for elite athletes (SRT-EA). It comprises 10 units, each lasting 90 minutes. It is intended for athletes and aims to improve their resistance to chronic stress. The evaluation was carried out in a quasi-experimental design, with three points of measurement (baseline, immediately after, and three months after) and two non-randomised groups: an intervention group (n=128) and an untreated control group (n=117). Participants were between 13 and 20 years of age (53.5% male) and represented various Olympic sports. Outcome variables were assessed with questionnaires. Significant short- and mid-term intervention effects were explored. The intervention increased stress-related knowledge, general self-efficacy, and stress sensitivity. Chronic stress level, stress symptoms, and stress reactivity were reduced. In line with the intention of the intervention, the results showed short- and mid-term, small to medium-sized effects. Accordingly, separate measurements at the end of the intervention and three months later showed mostly positive subjective experiences. Thus, the results reinforce the hope that educational-psychological stress-management interventions can support dual careers. KW - Chronic stress KW - stress-resistance KW - elite athletes KW - intervention KW - evaluation Y1 - 2017 U6 - https://doi.org/10.1080/17461391.2017.1407363 SN - 1746-1391 SN - 1536-7290 VL - 18 IS - 1 SP - 112 EP - 122 PB - Routledge, Taylor & Francis Group CY - Abingdon ER - TY - THES A1 - Buschmann, Jana T1 - Nutzungsfokussierte Evaluation in der Lehrkräftefortbildung Belcantare Brandenburg für musikunterrichtende Grundschullehrer*innen im ländlichen Raum T1 - Utilisation-focused evaluation of Belcantare Brandenburg, a CPD course for primary-level teachers of music in rural areas of the German federal state of Brandenburg N2 - Die vorliegende Publikation der Dissertationsschrift „Nutzungsfokussierte Evaluation in der Lehrkräftefortbildung Belcantare Brandenburg für musikunterrichtende Grundschul-lehrer*innen im ländlichen Raum“ ist eine akteursorientierte, explorativ angelegte Evaluation. Seit 2011 führt in den Regionen des Landes Brandenburg der Landesmusikrat Brandenburg e.V. in Kooperation mit mehreren Institutionen die zweijährige Fortbildung für fachnah sowie ausgebildete Musiklehrkräfte im Kompetenzfeld Singen und Lieddidaktik durch. Der zugrunde liegende Evaluationsansatz stellt die Interessen der kooperierenden Partner, welche praktische Konsequenzen aus den Ergebnissen der Evaluation zu ziehen beabsichtigen, in den Mittelpunkt der Forschungsarbeit. Es handelt sich somit um eine Auftragsforschung. Der Evaluation kommen die Funktionen zu, die inhaltliche Qualität der Lehrkräftefortbildung zu sichern und zu optimieren, den Erkenntnisgewinn zur Gestaltung eines fachdidaktischen Coachings zu erweitern, die Forschungsergebnisse zur Legitimation und Partizipation sichtbar zu machen sowie analytische Entscheidungshilfe zur Weiterführung Belcantare Brandenburgs nach 2022 bereitzustellen. Die von den Akteuren an die Autorin herangetragenen Forschungsanliegen wurden zu vier Fragestellungen zusammengefasst: 1. Wie zufrieden sind die Teilnehmenden mit der Veranstaltungsreihe? 2. Welche fachlichen, didaktischen und persönlichen Entwicklungen stellen sich während des Fortbildungszeitraumes aus der Wahrnehmungsperspektive der teilnehmenden Lehrkräfte ein? 3. Wie beurteilen die Coaching-Beteiligten die Chancen und Grenzen des musikdidaktischen Coachings als Fortbildungsform? 4. Welche Schlussfolgerungen lassen sich hinsichtlich professioneller Lehrkräftefortbildung aus der Gegenüberstellung der empirischen Erkenntnisse mit denen der Theorie ziehen? Diese Forschungsfragen wurden in zwei Forschungsphasen beantwortet: 1. Der empirische Datenkorpus wurde zwischen 2011-2015 gebildet. In dieser Zeit hatten zur projektbegleitenden Qualitätssicherung und -weiterführung der Pilot- und Folgestaffel Belcantare Brandenburgs die Forschungsfragen 1, 2 und 3 besondere Relevanz. Die Evaluationsstudie ist explorativ angelegt: Die Variablen zu den Forschungsfragen 1 und 2 sind durch Dokumentenanalysen sowie Interview-auswertungen mit der Projektleitung und teilnehmenden Lehrkräften sukzessive herausgearbeitet. Ebenso entsprechen die halb-geschlossenen Fragebögen als zentrale Erhebungsinstrumente der Forschungsfragen 1 und 2 dem explorativen Charakter und stellen auf diesem Weg sicher, dass den Teilnehmer*innen (N=40) die Möglichkeit zum Einbringen eigener Perspektiven eingeräumt wurde. Mit der Gesamtnote „sehr gut“ (1,39) seitens der befragten Lehrkräfte gilt die Gestaltung der Veranstaltungsreihe als ein Best-Practice-Beispiel: Für die Lehrkräfte sind das handlungsorientierte Erarbeiten von schülerpassenden und thematisch geeigneten, unmittelbar einsetzbaren oder wiederholt geübten Unterrichtsinhalten, Lerngegenständen und dazu passenden Materialien für den Unterricht die wesentlichen Kriterien zur Nutzung einer solchen Professionalisierungsmaßnahme. Die Lehrkräfteentwicklungen beider beforschter Staffeln zeigen, dass die fachnahen Kräfte bei sich größere Entwicklungszuwächse nach Beendigung des Projektes wahrnehmen als die Fachkräfte. Gleichzeitig liegt die selbsteingeschätzte Fachkompetenz der fachnahen Kräfte zu Fortbildungsende unter denen der Fachkräfte. Der Forschungsfrage 3 liegt ein ausschließlich qualitatives Design (N=16) zugrunde. Im Ergebnis konnten die Offene Form fachdidaktischen Coachings definiert werden, deren Parameter beschrieben und wesentliche Eigenschaften von Coach-Constellationen für ein binnendifferenziertes Coaching in der Lehrkräftefortbildung benannt werden. 2. Im Mai 2019 bildete sich aufgrund des sich verschärfenden Fachkräftemangels in Brandenburg das Bestreben der Kooperationspartner heraus, die Lehrkräftefortbildung nach 2022 als qualitätssichernde Maßnahme fortführen zu wollen. Diese Situation führte 2019 zur Aufnahme der Forschungsfrage 4, die eine umfassende und aktualisierte Analyse der theoretischen und bildungspolitischen Hintergründe der Intervention implizierte, mit dem Ziel, den Erkenntnisstand der Evaluation für eine erneute Empfehlung zu vertiefen. Das Thematisieren sowie das Gestalten von Selbstlernprozessen in der professionalisierenden Lehrkräftefortbildung stellte sich hierbei als ein zentrales Merkmal innovativer Lernkultur heraus. Die Publikation gliedert sich in vier Teile: Teil I stellt den Forschungsstand zur professionalisierenden Lehrkräfte¬fortbildung aus bildungswissenschaftlicher und musikpäda-gogischer Perspektive dar. Teil II der Arbeit stellt die komplexen Begründungs-zusammenhänge zum Evaluationsgegenstand her. Im III. Teil der Arbeit ist die Evaluationsstudie zu finden. Deren induktiv erschlossene Erkenntnisse werden in Teil IV der Arbeit dem Forschungsstand zur professionalisierenden Lehrkräftefortbildung gegenübergestellt. N2 - The doctoral thesis published in this book, Utilisation-focused evaluation of Belcantare Brandenburg, a CPD course for primary-level teachers of music in rural areas of the German federal state of Brandenburg, is an exploratory, actor-centred evaluation of the music teaching project Belcanatre Brandenburg which has been in process since 2011. Belcantare Brandenburg is a two-year CPD course for formally trained and untrained music teachers running in the state’s various regions and delivered by the Landesmusikrat (State Music Council) Brandenburg in collaboration with several institutions. Its objective is to develop primary-level educators’ skills around singing and working with songs in the classroom. The thesis was a commissioned piece of research, using an approach to evaluation that centres the interests of the cooperating organisations, which intend to obtain prompts to practical action from the results of the process. The purposes of the evaluation were quality assurance and optimisation for the course, the attainment of insights to inform the development of coaching for subject teachers, the publication of the evaluation process’ findings in order to demonstrate the added value generated by the course and facilitate stakeholder participation, and the provision of an analytical framework to guide decisions on the project’s continuation after 2022. After discussion with the cooperating parties to the project on their requirements of the evaluation, the author formulated four guiding questions as follows: 1. What levels of satisfaction with the course are observable among its participants? 2. How do the participants see themselves as having gained, in terms of subject-related and professional skills and personal development, during the course? 3. How do those taking part in the coaching (both coaches and coachees) assess the opportunities that present themselves with coaching in music teaching as a form of CPD, and where do they perceive limitations? 4. Which conclusions can we draw for teachers’ CPD from a comparative assessment of theoretical alongside empirical insights? The research that answered these four key questions took place in two stages as follows: 1. The author collected the corpus of empirical data used in the evaluation during the period 2011-2015. In this period, research questions 1-3 were of particular relevance in ongoing quality monitoring and development for the pilot course and its successor. The study has an exploratory design. The variables examined in relation to research questions 1 and 2 have emerged successively from document analysis and the analysis of interviews with partcipants and project managers. Similarly in line with the study’s exploratory character, the semi-closed design of the evaluation questionnaires that were the principal means of data collection for research questions 1 and 2 enabled their respondents (N=40) to contribute their individual points of view to the evaluation process. Using the grading system in place in German schools (1-5, with 1 being the best and 5 the weakest), participants awarded the course an average score of 1.39 wherefore the project can be regarded as an example of best practice in this area. In teachers’ view, the key criterion that made a CPD course of this type worthwhile was the opportunity to engage in a ‘hands-on’ process to develop teaching content and materials that met pupils’ needs and were appropriate to the topic at hand, directly usable in the classroom setting or suitable for repeated practice. The study’s findings on participants’ subjectively assessed professional development during both courses showed that teachers without specific formal qualifications in music teaching considered themselves to have developed further as teachers during the project’s course than those with such qualifications. This notwithstanding, after completion of the course, members of the former group assess themselves as less skilled in the subject of music than do members of the latter. Work on research question 3 was entirely qualitative in design (N=16). Its result was a definition of an ‘open form of coaching in subject teaching skills’ (Offene Form fachdidaktischen Coachings), encompassing descriptions of its parameters and outlines of the key aspects of coach/coachee pairings for coaching in teacher CPD that follows the principle of internal differentiation. 2. In May 2019, in the context of an increasing shortage of appropriately qualified teachers in the state of Brandenburg, the cooperating institutions reached a consensus on their intent to continue the course after 2022 in the interests of quality assurance for primary music teaching. Accordingly, in the same year, research question 4 joined the initial three. Its investigation called for a comprehensive, up-to-date analysis of the theoretical and education policy issues underlying the project as an intervention to support teaching quality, to the end of providing the evaluation with robust insights that would enable it to make a recommendation on the continuation or otherwise of the course. The analysis revealed the open discussion of and active engagement with personal learning processes to be one key characteristic of an innovative culture of learning in CPD for teachers with a focus on professional learning. The book has four parts as follows: Part I outlines the current state of research with regard to professional learning-focused CPD for teachers, taking the dual perspective of education research and music pedagogy. Part II explicates the complex links between this starting point and the topic of the evaluation at hand. The evaluation study itself comprises Part III, while Part IV delineates the insights inductively gained from it and considers them in the context of research on the subject as it stands at the present time. KW - Lehrkräftefortbildung KW - Musik KW - ländlicher Raum KW - fachdidaktisches Coaching KW - Selbstgesteuertes Lernen KW - evaluation KW - music KW - coaching KW - rural areas KW - self-directed learning KW - teacher training Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus4-525642 ER - TY - JOUR A1 - Waitelonis, Jörg A1 - Jürges, Henrik A1 - Sack, Harald T1 - Remixing entity linking evaluation datasets for focused benchmarking JF - Semantic Web N2 - In recent years, named entity linking (NEL) tools were primarily developed in terms of a general approach, whereas today numerous tools are focusing on specific domains such as e.g. the mapping of persons and organizations only, or the annotation of locations or events in microposts. However, the available benchmark datasets necessary for the evaluation of NEL tools do not reflect this focalizing trend. We have analyzed the evaluation process applied in the NEL benchmarking framework GERBIL [in: Proceedings of the 24th International Conference on World Wide Web (WWW’15), International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 2015, pp. 1133–1143, Semantic Web 9(5) (2018), 605–625] and all its benchmark datasets. Based on these insights we have extended the GERBIL framework to enable a more fine grained evaluation and in depth analysis of the available benchmark datasets with respect to different emphases. This paper presents the implementation of an adaptive filter for arbitrary entities and customized benchmark creation as well as the automated determination of typical NEL benchmark dataset properties, such as the extent of content-related ambiguity and diversity. These properties are integrated on different levels, which also enables to tailor customized new datasets out of the existing ones by remixing documents based on desired emphases. Besides a new system library to enrich provided NIF [in: International Semantic Web Conference (ISWC’13), Lecture Notes in Computer Science, Vol. 8219, Springer, Berlin, Heidelberg, 2013, pp. 98–113] datasets with statistical information, best practices for dataset remixing are presented, and an in depth analysis of the performance of entity linking systems on special focus datasets is presented. KW - Entity Linking KW - GERBIL KW - evaluation KW - benchmark Y1 - 2019 U6 - https://doi.org/10.3233/SW-180334 SN - 1570-0844 SN - 2210-4968 VL - 10 IS - 2 SP - 385 EP - 412 PB - IOS Press CY - Amsterdam ER -