Towards Unified Interface: Bridging Internet of Things and Augmented Reality

Abstract: Augmented Reality (AR) and Mixed Reality (MR) provide users with immersive experiences where virtual elements blend seamlessly into the physical environment. However, to fully harness the potential of AR/MR devices, it is crucial to extend their capabilities by enabling interaction with physical objects connected through the Internet of Things (IoT). Achieving this integration is often hindered by the heterogeneity of underlying technologies, and typically demands the expertise of specialized developers to build the necessary communication mechanisms. To overcome these challenges, this article presents a framework that simplifies the integration between AR/MR and IoT systems, enabling real-time and dynamic interaction. The proposed AR/MR-IoT framework relies on widely used open-source protocols and tools such as MQTT, HTTPS, and Node-RED. Following a detailed breakdown of the framework's architecture, its practical application is demonstrated through a use case involving the monitoring and control of a smart power socket using Microsoft HoloLens AR/MR glasses. Performance evaluation of this scenario shows that the framework consistently handles interaction and data update requests with response times under 100 milliseconds in typical operating conditions.

Keywords: IoT; Internet of Things; augmented reality; mixed reality; open-source framework; sensors; actuators