Symbolic Magnifying Lens Abstraction in Markov Decision Processes

Roy, P., Parker, D., Norman, G. and de Alfaro, L. (2008) Symbolic Magnifying Lens Abstraction in Markov Decision Processes. In: International Conference on Quantitative Evaluation of Systems (QEST'08), St. Malo, France, 14-17 Sept. 2008, pp. 103-112. (doi: 10.1109/QEST.2008.41)

Abstract

In this paper, we combine abstraction-refinement and symbolic techniques to fight the state-space explosion problem when model checking Markov decision processes (MDPs). The abstract-refinement technique, called "magnifying-lens abstraction" (MLA), partitions the state-space into regions and computes upper and lower bounds for reachability and safety properties on the regions, rather than the states. To compute such bounds, MLA iterates over the regions, analyzing the concrete states of each region in turn - as if one was sliding a magnifying lens across the system to view the states. The algorithm adaptively refines the regions, using smaller regions where more detail is required, until the difference between the bounds is below a specified accuracy. The symbolic technique is based on multi-terminal binary decision diagrams (MTBDDs) which have been used extensively to provide compact encodings of probabilistic models. We introduce a symbolic version of the MLA algorithm, called "symbolic MLA", which combines the power of both practical techniques when verifying MDPs. An implementation of symbolic MLA in the probabilistic model checker PRISM and experimental results to illustrate the advantages of our approach are presented.

• Item Type: Conference Proceedings
• Additional Information: Isbn: 9780769533605
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Norman, Dr Gethin
• Authors: Roy, P., Parker, D., Norman, G., and de Alfaro, L.
• Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
• College/School: College of Science and Engineering > School of Computing Science
• First Published: First published in Proceedings. Quantitative Evaluation of Systems (QEST'08) 2008:103-112
• Publisher Policy: Reproduced in accordance with the copyright policy of the publisher