Use of Transformer-Based Models for Word-Level Transliteration of the Book of the Dean of Lismore

Gow-Smith, E., McConville, M. , Gillies, W., Scott, J. and Ó Maolalaigh, R. (2022) Use of Transformer-Based Models for Word-Level Transliteration of the Book of the Dean of Lismore. In: 4th Celtic Language Technology Workshop (CLTW 4) within LREC2022, Marseille, France, 20-25 June 2022, pp. 94-98. ISBN 9791095546733

Abstract

The Book of the Dean of Lismore (BDL) is a 16th-century Scottish Gaelic manuscript written in a non-standard orthography. In this work, we outline the problem of transliterating the text of the BDL into a standardised orthography, and perform exploratory experiments using Transformer-based models for this task. In particular, we focus on the task of word-level transliteration, and achieve a character-level BLEU score of 54.15 with our best model, a BART architecture pre-trained on the text of Scottish Gaelic Wikipedia and then fine-tuned on around 2,000 word-level parallel examples. Our initial experiments give promising results, but we highlight the shortcomings of our model, and discuss directions for future work.

• Item Type: Conference Proceedings
• Additional Information: This work was supported by Faclair na Gaidhlig ` and the Centre for Doctoral Training in Speech and Language Technologies (SLT) and their Applications funded by UK Research and Innovation [grant number EP/S023062/1].
• Keywords: Low-Resource Neural Machine Translation, Transformer-Based Models, Scottish Gaelic, Historical Manuscript
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: O Maolalaigh, Professor Roibeard and McConville, Dr Mark and Scott, Miss Jade
• Authors: Gow-Smith, E., McConville, M., Gillies, W., Scott, J., and Ó Maolalaigh, R.
• College/School: College of Arts > School of Critical Studies > English Language and Linguistics; College of Arts > School of Humanities > Celtic and Gaelic
• ISBN: 9791095546733
• Copyright Holders: Copyright © European Language Resources Association (ELRA) 2022
• First Published: First published in Proceedings of the 4th Celtic Language Technology Workshop within LREC2022: 94-98
• Publisher Policy: Reproduced under a Creative Commons License

Related URLs

• Publisher
• Organisation