An efficient time optimized scheme for progressive analytics in big data

Anagnostopoulos, C. , Kolomvatsos, K. and Hadjiefthymiades, S. (2015) An efficient time optimized scheme for progressive analytics in big data. Big Data Research, 2(4), pp. 155-165. (doi: 10.1016/j.bdr.2015.02.001)

108908.pdf - Accepted Version



Big data analytics is the key research subject for future data driven decision making applications. Due to the large amount of data, progressive analytics could provide an efficient way for querying big data clusters. Each cluster contains only a piece of the examined data. Continuous queries over these data sources require intelligent mechanisms to result the final outcome (query response) in the minimum time with the maximum performance. A Query Controller (QC) is responsible to manage continuous/sequential queries and return the final outcome to users or applications. In this paper, we propose a mechanism that can be adopted by the QC. The proposed mechanism is capable of managing partial results retrieved by a number of processors each one responsible for each cluster. Each processor executes a query over a specific cluster of data. Our mechanism adopts two sequential decision making models for handling the incoming partial results. The first model is based on a finite horizon time-optimized model and the second one is based on an infinite horizon optimally scheduled model. We provide mathematical formulations for solving the discussed problem and present simulation results. Through a large number of experiments, we reveal the advantages of the proposed models and give numerical results comparing them with a deterministic model. These results indicate that the proposed models can efficiently reduce the required time for returning the final outcome to the user/application while keeping the quality of the aggregated result at high levels.

Item Type:Articles
Glasgow Author(s) Enlighten ID:Anagnostopoulos, Dr Christos and Kolomvatsos, Dr Kostas
Authors: Anagnostopoulos, C., Kolomvatsos, K., and Hadjiefthymiades, S.
Subjects:Q Science > QA Mathematics
Q Science > QA Mathematics > QA76 Computer software
College/School:College of Science and Engineering > School of Computing Science
Journal Name:Big Data Research
Copyright Holders:Copyright © 2015 Elsevier
First Published:First published in Big Data Research 2(4):155-165
Publisher Policy:Reproduced in accordance with the copyright policy of the publisher.

University Staff: Request a correction | Enlighten Editors: Update this record