A game-based abstraction-refinement framework for Markov decision processes

Kattenbelt, M., Kwiatkowska, M., Norman, G. and Parker, D. (2010) A game-based abstraction-refinement framework for Markov decision processes. Formal Methods in System Design, 36(3), pp. 246-280. (doi: 10.1007/s10703-010-0097-6)

Abstract

In the field of model checking, abstraction refinement has proved to be an extremely successful methodology for combating the state-space explosion problem. However, little practical progress has been made in the setting of probabilistic verification. In this paper we present a novel abstraction-refinement framework for Markov decision processes (MDPs), which are widely used for modelling and verifying systems that exhibit both probabilistic and nondeterministic behaviour. Our framework comprises an abstraction approach based on stochastic two-player games, two refinement methods and an efficient algorithm for an abstraction-refinement loop. The key idea behind the abstraction approach is to maintain a separation between nondeterminism present in the original MDP and nondeterminism introduced during the abstraction process, each type being represented by a different player in the game. Crucially, this allows lower and upper bounds to be computed for the values of reachability properties of the MDP. These give a quantitative measure of the quality of the abstraction and form the basis of the corresponding refinement methods. We describe a prototype implementation of our framework and present experimental results demonstrating automatic generation of compact, yet precise, abstractions for a large selection of real-world case studies.

• Item Type: Articles (Other)
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Norman, Dr Gethin
• Authors: Kattenbelt, M., Kwiatkowska, M., Norman, G., and Parker, D.
• Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: Formal Methods in System Design
• Publisher: Springer
• ISSN: 0925-9856
• ISSN (Online): 1572-8102
• Published Online: 03 August 2010