Скачать или смотреть Demonstrating an Automatic Gloss Checker for Annotated Corpora

Presentation given by Herbert Lange, Jocelyn Aznar, and Amy Isard at the Language Documentation and Archiving conference #LDA2022

Glossing is a well-established approach to annotating language data. High-quality glosses are essential for linguistic analysis and reuse of the corpus data. As a mostly manual annotation process, it can be error-prone. To mitigate this issue we present an automatic gloss checking procedure that highlights problematic annotations.

One well-established standard for interlinear glossing is Leipzig Glossing Rules [2]. For sign language other glossing standards are available [3]. In general, lemmas and morphological and other features can be combined to gloss language data. To automate the gloss checking we have to identify lemmas and features. Lemmas can be validated using dictionary lookup. Features have to be documented to keep them understandable for the future [1]. If such a documentation exists, it can be used to validate the consistency of the feature annotations. The feature checking procedure depends on how the features are documented. In practice some feature names are very common and have independent semantics while others only make sense in combination. This can lead to varying granularity of documented features. As a result morpheme glosses can either be directly looked up in the documentation or have to be analyzed by splitting the feature set into documented units which requires more effort. Finally, to improve the usefulness of the checking procedure we also use approximate matching in the case of dictionary lookup to identify typos as well other problems. If there is no information about glosses used or no suitable dictionary available, the list of lemmas and glosses encountered can be output.

By publishing our gloss checking tool we hope to help corpus creators to guarantee or improve the quality of their corpus annotations. It can either be used to validate a finalized corpus or help to create consistent annotations while annotating a corpus.