Dynamic AI-IoT: enabling updatable AI models in ultra-low-power 5G IoT devices

This article addresses the challenge of integrating dynamic AI capabilities into ultralow-power (ULP) IoT devices, a critical necessity in the rapidly evolving landscape of 5G and potential 6G technologies. We introduce the Dynamic AI-IoT architecture, a novel framework designed to eliminate the need for cumbersome firmware updates. This architecture leverages Narrowband IoT (NB-IoT) to facilitate smooth cloud interactions and incorporates tailored firmware extensions for enabling dynamic interactions with Tiny Machine Learning (TinyML) models. A sophisticated memory management mechanism, grounded in memory alignment and dynamic AI operations resolution, is introduced to efficiently handle AI tasks. Empirical experiments demonstrate the feasibility of implementing a Dynamic AI-IoT system using ULP IoT devices on a 5G testbed. The results show model updates taking less than one second and an average inference time of approximately 46 ms.