Behave Nicely! Automatic Generation of Code for Behaviour Driven Development Test Suites

Bob, R. and Storer, T. (2019) Behave Nicely! Automatic Generation of Code for Behaviour Driven Development Test Suites. In: 19th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM 2019), Cleveland, OH, USA, 30 Sep - 01 Oct 2019, pp. 228-237. ISBN 9781728149370 (doi:10.1109/SCAM.2019.00033)

[img]
Preview
Text
192036.pdf - Accepted Version

260kB

Abstract

Behaviour driven development (BDD) has gained widespread use in the software industry. System specifications can be expressed as test scenarios, describing the circumstances, actions and expected outcomes. These scenarios are written in a structured natural language (Gherkin), with each step in the scenario associated with a corresponding step implementation function in the underlying programming language. A challenge recognised by industry is ensuring that the natural language scenarios, step implementation functions and underlying system implementation remain consistent with one another, requiring on-going maintenance effort as changes are made to a system. To address this, we have developed behave_nicely, a tool, for automatically generating step implementation functions from structured natural language steps, with the intention of eliminating the need for maintaining step implementation functions. We evaluated our approach on a sample of 20 white box and 50 black box projects using behaviour driven development, drawn from GitHub. Our results show that behave_nicely can generate step implementation functions for 80% of the white box and 17% of black box projects. We conclude that (a) there is significant potential for automating the process of code generation for BDD tests and (b) that the development of guidelines for writing tests in Gherkin would significantly improve the results.

Item Type:Conference Proceedings
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Bob, Ms Ruxandra and Storer, Dr Tim
Authors: Bob, R., and Storer, T.
Subjects:Q Science > QA Mathematics > QA76 Computer software
College/School:College of Science and Engineering > School of Computing Science
ISSN:2470-6892
ISBN:9781728149370
Copyright Holders:Copyright © 2019 IEEE
Publisher Policy:Reproduced in accordance with the copyright policy of the publisher
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record