Optimizing Fixed-Size Stochastic Controllers for POMDPs

Christopher Amato, Daniel S. Bernstein, Shlomo Zilberstein

In this paper, we discuss a new approach that represents POMDP policies as finite-state controllers and formulates the optimal policy of a desired size as a nonlinear program (NLP). This new representation allows a wide range of powerful nonlinear programming algorithms to be used to solve POMDPs. Although solving the NLP optimally is often intractable, the results we obtain using an off-the-shelf optimization method are competitive with state-of-the-art POMDP algorithms. Our approach is simple to implement and it opens up promising research directions for solving POMDPs using nonlinear programming methods.

Subjects: 15.5 Decision Theory; 9.3 Mathematical Foundations

Submitted: May 2, 2008

This page is copyrighted by AAAI. All rights reserved. Your use of this site constitutes acceptance of all of AAAI's terms and conditions and privacy policy.