Metric Learning for Categorical and Ambiguous Features: An Adversarial Approach

Yang, X. , Dong, M., Guo, Y. and Xue, J.-H. (2021) Metric Learning for Categorical and Ambiguous Features: An Adversarial Approach. In: European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD 2020), 14-18 Sep 2020, pp. 223-238. ISBN 9783030676605 (doi: 10.1007/978-3-030-67661-2_14)

Abstract

Metric learning learns a distance metric from data and has significantly improved the classification accuracy of distance-based classifiers such as k-nearest neighbors. However, metric learning has rarely been applied to categorical data, which are prevalent in health and social sciences, but inherently difficult to classify due to high feature ambiguity and small sample size. More specifically, ambiguity arises as the boundaries between ordinal or nominal levels are not always sharply defined. In this paper, we mitigate the impact of feature ambiguity by considering the worst-case perturbation of each instance and propose to learn the Mahalanobis distance through adversarial training. The geometric interpretation shows that our method dynamically divides the instance space into three regions and exploits the information on the “adversarially vulnerable” region. This information, which has not been considered in previous methods, makes our method more suitable than them for small-sized data. Moreover, we establish the generalization bound for a general form of adversarial training. It suggests that the sample complexity rate remains at the same order as that of standard training only if the Mahalanobis distance is regularized with the elementwise 1-norm. Experiments on ordinal and mixed ordinal-and-nominal datasets demonstrate the effectiveness of the proposed method when encountering the problems of high feature ambiguity and small sample size.

• Item Type: Conference Proceedings
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Yang, Dr Xiaochen
• Authors: Yang, X., Dong, M., Guo, Y., and Xue, J.-H.
• College/School: College of Science and Engineering > School of Mathematics and Statistics > Statistics
• ISSN: 0302-9743
• ISBN: 9783030676605
• Published Online: 25 February 2021
• Copyright Holders: Copyright © 2021 Springer Nature Switzerland AG
• First Published: First published in Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2020. Lecture Notes in Computer Science, vol 12458: 223-238
• Publisher Policy: Reproduced in accordance with the copyright policy of the publisher

Related URLs

• Organisation