Speggiorin, A., Dalton, J. and Leuski, A. (2022) TaskMAD: a Platform for Multimodal Task-Centric Knowledge-Grounded Conversational Experimentation. In: SIGIR 2022: 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, 11-15 Jul 2022, pp. 3240-3244. ISBN 9781450387323 (doi: 10.1145/3477495.3531679)
![[img]](https://eprints.gla.ac.uk/style/images/fileicons/text.png) |
Text
270394.pdf - Accepted Version 3MB |
Abstract
The role of conversational assistants continues to evolve, beyond simple voice commands to ones that support rich and complex tasks in the home, car, and even virtual reality. Going beyond simple voice command and control requires agents and datasets blending structured dialogue, information seeking, grounded reasoning, and contextual question-answering in a multimodal environment with rich image and video content. In this demo, we introduce Task-oriented Multimodal Agent Dialogue (TaskMAD), a new platform that supports the creation of interactive multimodal and task-centric datasets in a Wizard-of-Oz experimental setup. TaskMAD includes support for text and voice, federated retrieval from text and knowledge bases, and structured logging of interactions for offline labeling. Its architecture supports a spectrum of tasks that span open-domain exploratory search to traditional frame-based dialogue tasks. It's open-source and offers rich capability as a platform used to collect data for the Amazon Alexa Prize Taskbot challenge, TREC Conversational Assistance track, undergraduate student research, and others. TaskMAD is distributed under the MIT license.
| Item Type: | Conference Proceedings |
|---|---|
| Status: | Published |
| Refereed: | Yes |
| Glasgow Author(s) Enlighten ID: | Speggiorin, Mr Alessandro and Dalton, Dr Jeff |
| Authors: | Speggiorin, A., Dalton, J., and Leuski, A. |
| College/School: | College of Science and Engineering > School of Computing Science |
| ISBN: | 9781450387323 |
| Copyright Holders: | Copyright © 2022 The Authors |
| First Published: | First published in SIGIR 2022: 45th International ACM SIGIR Conference on Research and Development in Information Retrieval: 3240-3244 |
| Publisher Policy: | Reproduced in accordance with the publisher copyright policy |
| Related URLs: |
University Staff: Request a correction | Enlighten Editors: Update this record
Funder and Project Information
Funder and Project Information