Gemmell, C., Mackie, I., Owoicho, P. O., Rossetto, F., Fischer, S. and Dalton, J. (2022) GRILLBot: A Multi-modal Conversational Agent for Complex Real-world Tasks. In: 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2022), Edinburgh, UK, 7-9 Sept 2022, pp. 654-658.
![[img]](https://eprints.gla.ac.uk/style/images/fileicons/text.png) |
Text
279043.pdf - Published Version Available under License Creative Commons Attribution. 7MB |
Publisher's URL: https://aclanthology.org/2022.sigdial-1.63
Abstract
We present GRILLBot, an open-source multi-modal task-oriented voice assistant to help users perform complex tasks, focusing on the domains of cooking and home improvement. GRILLBot curates and leverages web information extraction to build coverage over a broad range of tasks for which a user can receive guidance. To represent each task, we propose TaskGraphs as a dynamic graph unifying steps, requirements, and curated domain knowledge enabling contextual question answering, and detailed explanations. Multi-modal elements play a key role in GRILLBot both helping the user navigate through the task and enriching the experience with helpful videos and images that are automatically linked throughout the task. We leverage a contextual neural semantic parser to enable flexible navigation when interacting with the system by jointly encoding stateful information with the conversation history. GRILLBot enables dynamic and adaptable task planning and assistance for complex tasks by combining elements of task representations that incorporate text and structure, combined with neural models for search, question answering, and dialogue state management. GRILLBot competed in the Alexa prize TaskBot Challenge as one of the finalists.
| Item Type: | Conference Proceedings |
|---|---|
| Status: | Published |
| Refereed: | Yes |
| Glasgow Author(s) Enlighten ID: | Rossetto, Federico and Fischer, Ms Sophie and Gemmell, Carlos and Mackie, Iain and Owoicho, Paul Ogbonoko and Dalton, Dr Jeff |
| Authors: | Gemmell, C., Mackie, I., Owoicho, P. O., Rossetto, F., Fischer, S., and Dalton, J. |
| College/School: | College of Science and Engineering College of Science and Engineering > School of Computing Science |
| Copyright Holders: | Copyright © 2022 Association for Computational Linguistics |
| Publisher Policy: | Reproduced under a Creative Commons licence |
| Related URLs: |
University Staff: Request a correction | Enlighten Editors: Update this record
Funder and Project Information
Funder and Project Information