Interactive Conversational AI with IoT Devices for Enhanced Human-Robot Interaction

Manolescu, Vasile Denis and AlZu'bi, Hamzah and Secco, Emanuele Lindo (2025) Interactive Conversational AI with IoT Devices for Enhanced Human-Robot Interaction. Journal of Intelligent Communication. (Accepted for Publication)

Text D-jic-review-assignment-317-Article+Text-4588.pdf - Published Version Available under License Creative Commons Attribution. Download (931kB)

Abstract

Significance - The rapid advancements in conversational AI and IoT technologies have opened up new possibilities for human-machine interaction. Despite the progress, a gap exists in integrating these two fields to create more centralized, intuitive, and engaging user experiences. Current integrations typically consist of specialized hardware-software pairs that do not fully leverage the capabilities of advanced conversational models, thereby limiting their applicability. This research proposes a general solution to bridge the capabilities of various IoT devices with the oversight and control abilities of AI language models, enhancing the potential for more versatile and natural IoT-AI-human interactions.
Aim and Approach - This research presents the design and development of an IoT system operated by an AI language model and conversationally managed by humans to operate robots. Based on this setup, the initial goal is to create a framework for interactively controlling a robotic arm. The approach involves using a Raspberry Pi as a central control system and ChatGPT API to manage conversations and execute given commands.
Results - The developed IoT-AI system demonstrated efficient and reliable human-robot interaction where the user can entertain a conversational interaction with the robotic arm. It effectively captures user voice inputs, processes them through advanced AI models, and generates appropriate commands for the robotic arm, achieving an average voice-to-motion latency of 5.5 s. An example of commands are “engage arm”, “move right 20” (i.e. move the robotic arm to the right of 20 cm) combined with more conversational commands such as “can you hear me?”, “what’s your name?”. While some latency and voice recognition challenges exist, the overall performance confirms the viability of using conversational AI for natural and intuitive robotic control.
Conclusions - This research successfully integrates conversational AI with IoT devices, resulting in a more user-centric and efficient human-robot interaction. The system highlights the significant potential of precisely translating natural language commands into robotic actions, enhancing user experience and operational efficiency.

Item Type:	Article
Additional Information and Comments:	Copyright © 2024 by the author(s). Published by UK Scientific Publishing Limited. This is an open access article under the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Faculty / Department:	Faculty of Human and Digital Sciences > School of Computer Science and the Environment
Depositing User:	Emanuele Secco
Date Deposited:	20 Jan 2025 14:35
Last Modified:	20 Jan 2025 14:35
URI:	https://hira.hope.ac.uk/id/eprint/4557

Actions (login required)

View Item