A game-based abstraction-refinement framework for Markov decision processes

Kattenbelt, M., Kwiatkowska, M., Norman, G. and Parker, D. (2010) A game-based abstraction-refinement framework for Markov decision processes. Formal Methods in System Design, 36(3), pp. 246-280. (doi:10.1007/s10703-010-0097-6)

Full text not currently available from Enlighten.

Abstract

In the field of model checking, abstraction refinement has proved to be an extremely successful methodology for combating the state-space explosion problem. However, little practical progress has been made in the setting of probabilistic verification. In this paper we present a novel abstraction-refinement framework for Markov decision processes (MDPs), which are widely used for modelling and verifying systems that exhibit both probabilistic and nondeterministic behaviour. Our framework comprises an abstraction approach based on stochastic two-player games, two refinement methods and an efficient algorithm for an abstraction-refinement loop. The key idea behind the abstraction approach is to maintain a separation between nondeterminism present in the original MDP and nondeterminism introduced during the abstraction process, each type being represented by a different player in the game. Crucially, this allows lower and upper bounds to be computed for the values of reachability properties of the MDP. These give a quantitative measure of the quality of the abstraction and form the basis of the corresponding refinement methods. We describe a prototype implementation of our framework and present experimental results demonstrating automatic generation of compact, yet precise, abstractions for a large selection of real-world case studies.

Item Type:Articles (Other)
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Norman, Dr Gethin
Authors: Kattenbelt, M., Kwiatkowska, M., Norman, G., and Parker, D.
Subjects:Q Science > QA Mathematics > QA75 Electronic computers. Computer science
College/School:College of Science and Engineering > School of Computing Science
Journal Name:Formal Methods in System Design
Publisher:Springer
ISSN:0925-9856
ISSN (Online):1572-8102
Published Online:03 August 2010

University Staff: Request a correction | Enlighten Editors: Update this record