WassRank: Listwise Document Ranking Using Optimal Transport Theory

Yu, H.-T., Jatowt, A., Joho, H., Jose, J. M. , Yang, X. and Chen, L. (2019) WassRank: Listwise Document Ranking Using Optimal Transport Theory. In: Twelfth ACM International Conference on Web Search and Data Mining, Melbourne, Australia, 11-15 Feb 2019, pp. 24-32. ISBN 9781450359405 (doi:10.1145/3289600.3291006)

Full text not currently available from Enlighten.

Abstract

Learning to rank has been intensively studied and has shown great value in many fields, such as web search, question answering and recommender systems. This paper focuses on listwise document ranking, where all documents associated with the same query in the training data are used as the input. We propose a novel ranking method, referred to as WassRank, under which the problem of listwise document ranking boils down to the task of learning the optimal ranking function that achieves the minimum Wasserstein distance. Specifically, given the query level predictions and the ground truth labels, we first map them into two probability vectors. Analogous to the optimal transport problem, we view each probability vector as a pile of relevance mass with peaks indicating higher relevance. The listwise ranking loss is formulated as the minimum cost (the Wasserstein distance) of transporting (or reshaping) the pile of predicted relevance mass so that it matches the pile of ground-truth relevance mass. The smaller the Wasserstein distance is, the closer the prediction gets to the ground-truth. To better capture the inherent relevance-based order information among documents with different relevance labels and lower the variance of predictions for documents with the same relevance label, ranking-specific cost matrix is imposed. To validate the effectiveness of WassRank, we conduct a series of experiments on two benchmark collections. The experimental results demonstrate that: compared with four non-trivial listwise ranking methods (i.e., LambdaRank, ListNet, ListMLE and ApxNDCG), WassRank can achieve substantially improved performance in terms of nDCG and ERR across different rank positions. Specifically, the maximum improvements of WassRank over LambdaRank, ListNet, ListMLE and ApxNDCG in terms of nDCG@1 are 15%, 5%, 7%, 5%, respectively.

Item Type:Conference Proceedings
Additional Information:This research has been supported by JSPS KAKENHI Grant Number JP17K12784.
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Jose, Professor Joemon and Chen, Dr Long and Joho, Mr Hideo and Yang, Dr Xiao
Authors: Yu, H.-T., Jatowt, A., Joho, H., Jose, J. M., Yang, X., and Chen, L.
College/School:College of Science and Engineering > School of Computing Science
College of Social Sciences > School of Social and Political Sciences
Journal Name:Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining - WSDM '19
Publisher:ACM Press
ISBN:9781450359405

University Staff: Request a correction | Enlighten Editors: Update this record