Alhamed, M. and Storer, T. (2021) Playing Planning Poker in Crowds: Human Computation of Software Effort Estimates. In: 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), Madrid, Spain, 22-30 May 2021, pp. 1-12. ISBN 9781665402965 (doi: 10.1109/ICSE43902.2021.00014)
![[img]](https://eprints.gla.ac.uk/style/images/fileicons/text.png) |
Text
234713.pdf - Accepted Version 523kB |
Abstract
Reliable cost effective effort estimation remains a considerable challenge for software projects. Recent work has demonstrated that the popular Planning Poker practice can produce reliable estimates when undertaken within a software team of knowledgeable domain experts. However, the process depends on the availability of experts and can be time-consuming to perform, making it impractical for large scale or open source projects that may curate many thousands of outstanding tasks. This paper reports on a full study to investigate the feasibility of using crowd workers supplied with limited information about a task to provide comparably accurate estimates using Planning Poker. We describe the design of a Crowd Planning Poker (CPP) process implemented on Amazon Mechanical Turk and the results of a substantial set of trials, involving more than 5000 crowd workers and 39 diverse software tasks. Our results show that a carefully organised and selected crowd of workers can produce effort estimates that are of similar accuracy to those of a single expert.
| Item Type: | Conference Proceedings |
|---|---|
| Status: | Published |
| Refereed: | Yes |
| Glasgow Author(s) Enlighten ID: | Storer, Dr Tim and Alhamed, Mr Mohammed |
| Authors: | Alhamed, M., and Storer, T. |
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science Q Science > QA Mathematics > QA76 Computer software |
| College/School: | College of Science and Engineering > School of Computing Science |
| ISSN: | 1558-1225 |
| ISBN: | 9781665402965 |
| Copyright Holders: | Copyright © 2021 The Authors |
| Publisher Policy: | Reproduced in accordance with the publisher copyright policy |
| Related URLs: |
University Staff: Request a correction | Enlighten Editors: Update this record
Altmetric
Altmetric