Tactile Communication System for the Interaction between Deafblind and Robots

Ozioko, O., Navaraj, W. T., Yogeswaran, N., Hersh, M. and Dahiya, R. (2018) Tactile Communication System for the Interaction between Deafblind and Robots. In: 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Nanjing, China, 27-31 Aug 2018, pp. 416-421. ISBN 9781538679814 (doi: 10.1109/ROMAN.2018.8525725)

[img]
Preview
Text
201386.pdf - Accepted Version

1MB

Abstract

This paper presents a touch-based system for communication by deafblind people with humans and robots. With presented system the deafblind people can send control signal to robots by writing block letters on an array of 8 × 8 diamond-shaped capacitive touch sensors. For better conformability with curved surfaces, the touch sensors arrays have been fabricated on flexible polyvinyl chloride (PVC) substrate. The characters written by deafblind on the tactile sensing array are recognized by Convolution Neural Network (CNN), which is trained with data from Extended Modified National Institute of Standards and Technology (EMNIST) database. A mobile app was also developed to efficiently recognize and display the written letters. Finally, a Baxter robot was simulated using Robot Operating System (ROS) and was remotely controlled by deafblind through English block letters written on the tactile sensing array. The result shows that the robot could successfully execute the `pick or drop' task. The developed system demonstrates the efficacy of tactile based communication by deafblind people to independently interact with a robot and their environment.

Item Type:Conference Proceedings
Additional Information:This work was supported in part by Engineering and Physical Sciences Research Council (EPSRC) Fellowship for Growth - Printable Tactile Skin (EP/M002527/1), and Tertiary Education and Trust Fund (TETFund), Nigeria. We are also thankful to Electronic Systems Design Centre (ESDC) for the support in terms of device design and characterization.
Status:Published
Refereed:Yes
Glasgow Author(s) Enlighten ID:Yogeswaran, Mr Nivasan and Dahiya, Professor Ravinder and Hersh, Dr Marion and Ozioko, Mr Oliver and Navaraj, Mr William
Authors: Ozioko, O., Navaraj, W. T., Yogeswaran, N., Hersh, M., and Dahiya, R.
College/School:College of Science and Engineering > School of Engineering > Biomedical Engineering
College of Science and Engineering > School of Engineering > Electronics and Nanoscale Engineering
ISSN:1944-9437
ISBN:9781538679814
Copyright Holders:Copyright © 2018 IEEE
First Published:First published in 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN): 416-421
Publisher Policy:Reproduced in accordance with the publisher copyright policy
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record

Project CodeAward NoProject NamePrincipal InvestigatorFunder's NameFunder RefLead Dept
170185Engineering Fellowships for Growth: Printed Tactile SKINRavinder DahiyaEngineering and Physical Sciences Research Council (EPSRC)EP/M002527/1ENG - Electronics & Nanoscale Engineering