Verification and control of partially observable probabilistic systems

Norman, G. , Parker, D. and Zou, X. (2017) Verification and control of partially observable probabilistic systems. Real-Time Systems, 53(3), pp. 354-402. (doi: 10.1007/s11241-017-9269-4)

Abstract

We present automated techniques for the verification and control of partially observable, probabilistic systems for both discrete and dense models of time. For the discrete-time case, we formally model these systems using partially observable Markov decision processes; for dense time, we propose an extension of probabilistic timed automata in which local states are partially visible to an observer or controller. We give probabilistic temporal logics that can express a range of quantitative properties of these models, relating to the probability of an event’s occurrence or the expected value of a reward measure. We then propose techniques to either verify that such a property holds or synthesise a controller for the model which makes it true. Our approach is based on a grid-based abstraction of the uncountable belief space induced by partial observability and, for dense-time models, an integer discretisation of real-time behaviour. The former is necessarily approximate since the underlying problem is undecidable, however we show how both lower and upper bounds on numerical results can be generated. We illustrate the effectiveness of the approach by implementing it in the PRISM model checker and applying it to several case studies from the domains of task and network scheduling, computer security and planning.

• Item Type: Articles
• Additional Information: This work was partly supported by the EPSRC grant “Automated Game-Theoretic Verification of Security Systems” (EP/K038575/1) and the PRINCESS project, funded by the DARPA BRASS programme.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Norman, Dr Gethin
• Authors: Norman, G., Parker, D., and Zou, X.
• College/School: College of Science and Engineering > School of Computing Science
• Journal Name: Real-Time Systems
• Publisher: Springer
• ISSN: 0922-6443
• ISSN (Online): 1573-1383
• Published Online: 08 March 2017
• Copyright Holders: Copyright © 2017 The Authors
• First Published: First published in Real-Time Systems 53(3):354-402
• Publisher Policy: Reproduced under a Creative Commons License