Enhancing arrhythmia diagnosis through ECG deep learning classification deploying and augmented reality 3D heart visualisation and interaction

Cardiovascular diseases (CVDs) continue to be a leading cause of mortality globally, highlighting the urgent need for timely and accurate diagnosis. Electrocardiography (ECG) is a vital diagnostic tool for detecting and monitoring various heart conditions by analysing the heart’s electrical activity; however, manually identifying ECG features and classifying heartbeats is a complex and time-consuming process that demands significant expertise. To address this challenge, we have developed ArythmiAR, a novel system that integrates Convolutional Neural Networks (CNN) with Augmented Reality (AR) to enable interactive diagnosis with 3D visualisation and real-time engagement. ArythmiAR offers several key innovations: deep learning-based ECG classification for precise arrhythmia detection, 3D heart modelling and assembly for detailed visualisation, an AR interface for deploying CNN models, 3D localisation of heart sub-regions responsible for arrhythmia anomalies, and enhanced 3D visualisation and interaction capabilities. Our study explores various ECG classification techniques, employing data rebalancing strategies to enhance model performance, with a particular focus on Multilayer Perceptron (MLP) and CNN models, which demonstrated highly competitive results on the PhysioNet MIT-BIH Arrhythmia dataset, achieving an accuracy of 99.07% with the MLP model. Remarkably, this work also involves deploying the ECG classification deep learning model within an AR environment, presenting a prototype for augmented rendering that allows users to localise, visualise, and interact with specific heart regions responsible for arrhythmias. This platform empowers medical professionals to make more accurate diagnoses and develop effective treatment strategies, thereby improving overall patient care.