Towards emotion recognition in immersive virtual environments: a method for facial emotion recognition

Kahina Amara*, Naeem Ramzan, Nadia Zenati, Oualid Djekoune, Cherif Larbes, Mohamed Amine Guerroudji, Djamel Aouam

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Downloads (Pure)

Abstract

Virtual Reality (VR) is, thus, proposed as a powerful tool to simulate complex, real situations and environments, offering researchers unprecedented opportunities to investigate human behaviour in closely controlled designs in controlled laboratory conditions. Facial emotion recognition has attracted a great deal of interest for interaction in virtual reality, healthcare system: Therapeutic applications, surveillance video application etc. In this paper, we propose a method for facial emotion recognition for immersive virtual environment based on 2D and 3D geometrical features. We used our collected dataset of 17 subjects' performance of six basic facial emotions (anger, fear, happiness, surprise, sadness, and neutral) using three devices: Kinect (v1), Kinect (v2), and RGB HD camera. In addition, we present the performance results of the RGB data for facial emotion recognition using Bagged Trees algorithm. To assess the performance of the proposed system, we used leave-one-out-subject cross-validation. We compared the 2D and 3D data performance for facial expression recognition. The obtained results show the superior performance of the RGB-D features provided by Kinect (v2). Our findings highlight that the 2D images are not robust enough for facial emotion recognition. The built facial emotion models will animate virtual characters that can express emotions via facial expressions. This could be deployed for Chatting, Learning and Therapeutic Intervention.

Original languageEnglish
Title of host publicationICCSA 2021 Conference on Computer Science's Complex Systems and their Applications 2021
Subtitle of host publicationProceedings of the 2nd International Conference on Computer Science's Complex Systems and their Applications Oum El Bouaghi, Algeria, May 25 and 26, 2021.
EditorsToufik Marir, Abdelhabib Bourouis, Rohallah Benaboud, Varun Gupta, Chetna Gupta
PublisherCEUR Workshop Proceedings
Pages253-263
Number of pages11
Volume2904
Publication statusPublished - 25 May 2021
Event2nd International Conference on Computer Science's Complex Systems and their Applications, ICCSA 2021 - Oum El Bouaghi, Algeria
Duration: 25 May 202126 May 2021

Publication series

NameCEUR Workshop Proceedings
PublisherCEUR Workshop Proceedings
Volume2904
ISSN (Print)1613-0073

Conference

Conference2nd International Conference on Computer Science's Complex Systems and their Applications, ICCSA 2021
Country/TerritoryAlgeria
CityOum El Bouaghi
Period25/05/2126/05/21

Keywords

  • avatar animation
  • facial emotion recognition
  • geometrical features
  • immersive environment
  • interaction
  • machine learning
  • Rgb
  • Rgb-d
  • virtual reality

Fingerprint

Dive into the research topics of 'Towards emotion recognition in immersive virtual environments: a method for facial emotion recognition'. Together they form a unique fingerprint.

Cite this