Coordinating Text and Graphics in Explanation Generation

Steven K. Feiner, Kathleen R. McKeown

To generate multimedia explanations, a system must be able to coordinate the use of different media in a single explanation. In this paper, we present the architecture that we have developed for COMET (Coordinated Multimedia Explanation Testbed), a system that generates directions for equipment maintenance and repair, and we show how it addresses the coordination problem. COMET includes a single content planner that produces a common content description used by multiple media-specific generators, and a media coordinator that performs a fine-grained division of information among media. Bidirectional interaction between media-specific generators allows influence across media. We describe COMET’s current capabilities and provide an overview of our plans for extending the system.

This page is copyrighted by AAAI. All rights reserved. Your use of this site constitutes acceptance of all of AAAI's terms and conditions and privacy policy.