A Conversational Model of Multimodal Interaction in Information Systems

Adelheit Stein, Ulrich Thiel

We propose a comprehensive framework for modeling and specifying multimodal interactions. To this end, we employ an extended notion of ' dialogue acts’ which can be realized by linguistic and non-linguistic means. First, a set of constraints is presented that describes the temporal structure and all patterns of exchange during a cooperative information-seeking dialogue. Second, we introduce a strategic level of description which allows the specification of the topical structure according to an information-seeking strategy. The model was used to design and implement the MERIT system, and led to a reduction in the complexity of the user interface while preserving most of the useful, but sometimes confusing, dialogue options of advanced direct manipulation interfaces.

This page is copyrighted by AAAI. All rights reserved. Your use of this site constitutes acceptance of all of AAAI's terms and conditions and privacy policy.