University of Glasgow at WebCLEF 2005: experiments in per-field normalisation and language specific stemming

Macdonald, C., Plachouras, V., He, B., Lioma, C. and Ounis, I. (2006) University of Glasgow at WebCLEF 2005: experiments in per-field normalisation and language specific stemming. Lecture Notes in Computer Science, 4022, pp. 898-907. (doi: 10.1007/11878773_100)

Abstract

We participated in the WebCLEF 2005 monolingual task. In this task, a search system aims to retrieve relevant documents from a multilingual corpus of Web documents from Web sites of European governments. Both the documents and the queries are written in a wide range of European languages. A challenge in this setting is to detect the language of documents and topics, and to process them appropriately. We develop a language specific technique for applying the correct stemming approach, as well as for removing the correct stopwords from the queries. We represent documents using three fields, namely content, title, and anchor text of incoming hyperlinks. We use a technique called per-field normalisation, which extends the Divergence From Randomness (DFR) framework, to normalise the term frequencies, and to combine them across the three fields. We also employ the length of the URL path of Web documents. The ranking is based on combinations of both the language specific stemming, if applied, and the per-field normalisation. We use our Terrier platform for all our experiments. The overall performance of our techniques is outstanding, achieving the overall top four performing runs, as well as the top performing run without metadata in the monolingual task. The best run only uses per-field normalisation, without applying stemming.

• Item Type: Articles
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: He, Mr Ben and Macdonald, Professor Craig and Ounis, Professor Iadh
• Authors: Macdonald, C., Plachouras, V., He, B., Lioma, C., and Ounis, I.
• Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science; Q Science > QA Mathematics > QA76 Computer software
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: Lecture Notes in Computer Science
• Publisher: Springer
• ISSN: 1611-3349
• Copyright Holders: Copyright © 2006 Springer
• First Published: First published in Lecture Notes in Computer Science 4022:898-907
• Publisher Policy: Reproduced in accordance with the copyright policy of the publisher.