Ueda, A., Santos, R., Macdonald, C. and Ounis, I. (2021) Structured Fine-Tuning of Contextual Embeddings for Effective Biomedical Retrieval. In: SIGIR 2021: 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 11-15 July 2021, pp. 2031-2035. (doi: 10.1145/3404835.3463075)
Text
239428.pdf - Accepted Version 647kB |
Abstract
Biomedical literature retrieval has greatly benefited from recent advances in neural language modeling. In particular, fine-tuning pretrained contextual language models has shown impressive results in recent biomedical retrieval evaluation campaigns. Nevertheless, current approaches neglect the inherent structure available from biomedical abstracts, which are (often explicitly) organised into semantically coherent sections such as background, methods, results, and conclusions. In this paper, we investigate the suitability of leveraging biomedical abstract sections for fine-tuning pretrained contextual language models at a finer granularity. Our results on two TREC biomedical test collections demonstrate the effectiveness of the proposed structured fine-tuning regime in contrast to a standard fine-tuning that does not leverage structure. Through an ablation study, we show that models fine-tuned on individual sections are able to capture potentially useful word contexts that may be otherwise ignored by structure-agnostic models.
Item Type: | Conference Proceedings |
---|---|
Additional Information: | This work was partially funded by the authors’ individual grants from CNPq, CAPES, and FAPEMIG. |
Status: | Published |
Refereed: | Yes |
Glasgow Author(s) Enlighten ID: | Macdonald, Professor Craig and Ounis, Professor Iadh and Ueda, Alberto |
Authors: | Ueda, A., Santos, R., Macdonald, C., and Ounis, I. |
College/School: | College of Science and Engineering > School of Computing Science |
Published Online: | 11 July 2021 |
Copyright Holders: | Copyright © 2021 Association for Computing Machinery |
Publisher Policy: | Reproduced in accordance with the publisher copyright policy |
University Staff: Request a correction | Enlighten Editors: Update this record