Cognitive Status and Form of Reference in Multimodal Human-Computer Interaction

Andrew Kehler, University of California, San Diego

We analyze a corpus of referring expressions collected from user interactions with a multimodal travel guide application. The analysis suggests that, in dramatic contrast to human-human interaction, the interpretation of referring expressions can be computed with very high accuracy using a model which pairs a highly impoverished notion of discourse state with a simple set of rules that are insensitive to the type of referring expression used. We attribute this unexpected result to the implicit manner in which the interface conveys the system’s beliefs about the operative discourse state, to which users appear to tailor their choice of referring expressions. This result offers new insight into the way that computer interfaces can shape a user’s language behavior, insights which can be exploited to bring otherwise difficult interpretation problems into the realm of tractability.

This page is copyrighted by AAAI. All rights reserved. Your use of this site constitutes acceptance of all of AAAI's terms and conditions and privacy policy.