Model-based, multimodal interaction in document browsing

Eslambolchilar, P. and Murray-Smith, R. (2006) Model-based, multimodal interaction in document browsing. Lecture Notes in Computer Science, 4299, pp. 1-12. (doi: 10.1007/11965152_1)

Full text not currently available from Enlighten.


In this paper we introduce a dynamic system approach to the design of multimodal interactive systems. We use an example where we support human behavior in browsing a document, by adapting the dynamics of navigation and the visual feedback (using a focus-in-context (F+C) method) to support the current inferred task. We also demonstrate non-speech audio feedback, based on a language model. We argue that to design interaction we need models of key aspects of the process, here for example, we need models for the dynamic system, language model and sonification. We show how the user’s intention is coupled to the visualization technique via the dynamic model, and how the focus-in-context method couples details in context to audio samples via the language identification system. We present probabilistic audio feedback as an example of a multimodal approach to sensing different languages in a multilingual text. This general approach is well suited to mobile and wearable applications, and shared displays.

Item Type:Articles
Glasgow Author(s) Enlighten ID:Murray-Smith, Professor Roderick
Authors: Eslambolchilar, P., and Murray-Smith, R.
Subjects:Q Science > QA Mathematics > QA75 Electronic computers. Computer science
College/School:College of Science and Engineering > School of Computing Science
Journal Name:Lecture Notes in Computer Science
ISSN (Online):1611-3349
Published Online:23 January 2007

University Staff: Request a correction | Enlighten Editors: Update this record