Use of Transformer-Based Models for Word-Level Transliteration of the Book of the Dean of Lismore

Gow-Smith, E., McConville, M. , Gillies, W., Scott, J. and Ó Maolalaigh, R. (2022) Use of Transformer-Based Models for Word-Level Transliteration of the Book of the Dean of Lismore. In: 4th Celtic Language Technology Workshop (CLTW 4) within LREC2022, Marseille, France, 20-25 June 2022, pp. 94-98. ISBN 9791095546733

[img] Text
286116.pdf - Published Version
Available under License Creative Commons Attribution Non-commercial.


Publisher's URL:


The Book of the Dean of Lismore (BDL) is a 16th-century Scottish Gaelic manuscript written in a non-standard orthography. In this work, we outline the problem of transliterating the text of the BDL into a standardised orthography, and perform exploratory experiments using Transformer-based models for this task. In particular, we focus on the task of word-level transliteration, and achieve a character-level BLEU score of 54.15 with our best model, a BART architecture pre-trained on the text of Scottish Gaelic Wikipedia and then fine-tuned on around 2,000 word-level parallel examples. Our initial experiments give promising results, but we highlight the shortcomings of our model, and discuss directions for future work.

Item Type:Conference Proceedings
Additional Information:This work was supported by Faclair na Gaidhlig ` and the Centre for Doctoral Training in Speech and Language Technologies (SLT) and their Applications funded by UK Research and Innovation [grant number EP/S023062/1].
Keywords:Low-Resource Neural Machine Translation, Transformer-Based Models, Scottish Gaelic, Historical Manuscript
Glasgow Author(s) Enlighten ID:O Maolalaigh, Professor Roibeard and McConville, Dr Mark and Scott, Miss Jade
Authors: Gow-Smith, E., McConville, M., Gillies, W., Scott, J., and Ó Maolalaigh, R.
College/School:College of Arts > School of Critical Studies > English Language and Linguistics
College of Arts > School of Humanities > Celtic and Gaelic
Copyright Holders:Copyright © European Language Resources Association (ELRA) 2022
First Published:First published in Proceedings of the 4th Celtic Language Technology Workshop within LREC2022: 94-98
Publisher Policy:Reproduced under a Creative Commons License
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record