POS-Tagging Historical Corpora: The Case of Early New High German

  • A key problem in automatic annotation of historical corpora is inconsistent spelling. Because the spelling of some word forms can differ between texts, a language model trained on already annotated treebanks may fail to recognize known word forms due to differences in spelling. In the present work, we explore the feasibility of an unsupervised method for spelling-adjustment for the purpose of improved part of speech (POS) tagging. To this end, we present a method for spelling normalization based on weighted edit distances, which exploits within-text spelling variation. We then evaluate the improvement in taging accuracy resulting from between-texts spelling normalization in two tagging experiments on several Early New High German (ENHG) texts.
Author:Ulrike DemskeORCiDGND, Pavel LogacevORCiDGND, Katrin Goldschmidt
Parent Title (English):Proceedings of the thirteenth workshop on treebanks and linguistic theories (TLT 13)
Publisher:TALAR - Tübingen Archive of Language Resources
Place of publication:Tübingen
Document Type:Conference Proceeding
Date of first Publication:2014/12/13
Year of Completion:2014
Publishing Institution:Universität Potsdam
Release Date:2020/02/04
First Page:103
Last Page:112
Organizational units:Philosophische Fakultät / Institut für Germanistik
Dewey Decimal Classification:4 Sprache / 41 Linguistik / 415 Grammatik