On the Reproducibility and Generalisation of the Linear Transformation of Word Embeddings

Yang, X., Ounis, I. , McCreadie, R., Macdonald, C. and Fang, A. (2018) On the Reproducibility and Generalisation of the Linear Transformation of Word Embeddings. In: 40th European Conference on Information Retrieval (ECIR 2018), Grenoble, France, 25-29 Mar 2018, pp. 263-275. ISBN 9783319769400 (doi:10.1007/978-3-319-76941-7_20)

Yang, X., Ounis, I. , McCreadie, R., Macdonald, C. and Fang, A. (2018) On the Reproducibility and Generalisation of the Linear Transformation of Word Embeddings. In: 40th European Conference on Information Retrieval (ECIR 2018), Grenoble, France, 25-29 Mar 2018, pp. 263-275. ISBN 9783319769400 (doi:10.1007/978-3-319-76941-7_20)

[img]
Preview
Text
154196.pdf - Accepted Version

563kB

Abstract

Linear transformation is a way to learn a linear relationship between two word embeddings, such that words in the two different embedding spaces can be semantically related. In this paper, we examine the reproducibility and generalisation of the linear transformation of word embeddings. Linear transformation is particularly useful when translating word embedding models in different languages, since it can capture the semantic relationships between two models. We first reproduce two linear transformation approaches, a recent one using orthogonal transformation and the original one using simple matrix transformation. Previous findings on a machine translation task are re-examined, validating that linear transformation is indeed an effective way to transform word embedding models in different languages. In particular, we show that the orthogonal transformation can better relate the different embedding models. Following the verification of previous findings, we then study the generalisation of linear transformation in a multi-language Twitter election classification task. We observe that the orthogonal transformation outperforms the matrix transformation. In particular, it significantly outperforms the random classifier by at least 10% under the F1 metric across English and Spanish datasets. In addition, we also provide best practices when using linear transformation for multi-language Twitter election classification.

Item Type:Conference Proceedings
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Macdonald, Dr Craig and Ounis, Professor Iadh and Fang, Mr Anjie and Mccreadie, Mr Richard and Yang, Dr Xiao
Authors: Yang, X., Ounis, I., McCreadie, R., Macdonald, C., and Fang, A.
College/School:College of Science and Engineering > School of Computing Science
ISSN:0302-9743
ISBN:9783319769400
Published Online:01 March 2018
Copyright Holders:Copyright © 2018 Springer International Publishing AG, part of Springer Nature
First Published:First published in Advances in Information Retrieval. ECIR 2018: 263-275
Publisher Policy:Reproduced in accordance with the publisher copyright policy

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
646621Explaining and Mitigating Electoral ViolenceSarah BirchEconomic and Social Research Council (ESRC)ES/L016435/1SPS - POLITICS