Streamlining Evaluation with ir-measures

MacAvaney, S. , Macdonald, C. and Ounis, I. (2022) Streamlining Evaluation with ir-measures. In: 44th European Conference on Information Retrieval (ECIR 2022), Stavanger, Norway, 10-14 Apr 2022, pp. 305-310. ISBN 9783030997359 (doi: 10.1007/978-3-030-99739-7_38)

Abstract

We present ir-measures, a new tool that makes it convenient to calculate a diverse set of evaluation measures used in information retrieval. Rather than implementing its own measure calculations, ir-measures provides a common interface to a handful of evaluation tools. The necessary tools are automatically invoked (potentially multiple times) to calculate all the desired metrics, simplifying the evaluation process for the user. The tool also makes it easier for researchers to use recently-proposed measures (such as those from the C/W/L framework) alongside traditional measures, potentially encouraging their adoption.

• Item Type: Conference Proceedings
• Additional Information: We acknowledge EPSRC grant EP/R018634/1: Closed-Loop Data Science for Complex, Computationally- & Data-Intensive Analytics.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: MacAvaney, Dr Sean and Macdonald, Professor Craig and Ounis, Professor Iadh
• Authors: MacAvaney, S., Macdonald, C., and Ounis, I.
• College/School: College of Science and Engineering > School of Computing Science
• ISSN: 0302-9743
• ISBN: 9783030997359
• Published Online: 05 April 2022
• Copyright Holders: Copyright © 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
• First Published: First published in Advances in Information Retrieval. ECIR 2022. Lecture Notes in Computer Science, vol 13186
• Publisher Policy: Reproduced in accordance with the publisher copyright policy

Related URLs

• Organisation

Project Code	Award No	Project Name	Principal Investigator	Funder's Name	Funder Ref	Lead Dept
300982		Exploiting Closed-Loop Aspects in Computationally and Data Intensive Analytics	Roderick Murray-Smith	Engineering and Physical Sciences Research Council (EPSRC)	EP/R018634/1	Computing Science