A novel facial emotion recognition method for stress inference of facial nerve paralysis patients

Basit Öğe Kaydı
dc.authorid0000-0003-1840-9958en_US
dc.contributor.authorXu, Cuiting
dc.contributor.authorYan, Chunchuan
dc.contributor.authorJiang, Mingzhe
dc.contributor.authorAlenezi, Fayadh
dc.contributor.authorAlhudhaif, Adi
dc.contributor.authorPolat, Kemal
dc.date.accessioned2023-09-29T06:13:15Z
dc.date.available2023-09-29T06:13:15Z
dc.date.issued2022en_US
dc.departmentBAİBÜ, Mühendislik Fakültesi, Elektrik Elektronik Mühendisliği Bölümüen_US
dc.descriptionAcknowledgments This work was supported in part by the National Natural ScienceFoundation of China under grants (61873349) ; the General Logistics Department of PLA (BLB19J005) ; The Guangzhou Science and Tech-nology Planning Project (202003000040) .en_US
dc.description.abstractFacial nerve paralysis results in muscle weakness or complete paralysis on one side of the face. Patients suffer from difficulties in speech, mastication and emotional expression, impacting their quality of life by causing anxiety and depression. The emotional well-being of a facial nerve paralysis patient is usually followed up during and after treatment as part of quality-of-life measures through questionnaires. The commonly used questionnaire may help recognize whether a patient has been through a depressive state but is unable to understand their basic emotions dynamically. Automatic emotion recognition from facial expression images could be a solution to help understand facial nerve paralysis patients, recognize their stress in advance, and assist their treatment. However, their facial expressions are different from healthy people due to facial muscle inability, which makes existing emotion recognition data and models from healthy people invalid. Recent studies on facial images mainly focus on the automatic diagnosis of facial nerve paralysis level and thus lack full basic emotions. Different nerve paralysis levels also increase inconsistency in expressing the same emotion among patients. To enable emotion recognition and stress inference from facial images for facial nerve paralysis patients, we established an emotional facial expressions dataset from 45 patients with six basic emotions. The problem of limited data size in building a deep learning model VGGNet was solved by leveraging facial images from healthy people in transfer learning. Our proposed model reached an accuracy of 66.58% recognizing basic emotions from patients, which was 19.63% higher than the model trained only from the facial nerve paralysis data and was 42.69% higher than testing directly on the model trained from healthy data. Logically, the results show that patients with less severe facial nerve paralysis reached a higher emotion recognition accuracy. Additionally, although disgust, anger, and fear were especially challenging to specify from each other, the accuracy was 85.97% recognizing any stress-related negative emotions, making stress inference feasible.en_US
dc.description.sponsorshipNational Natural ScienceFoundation of China [61873349]; General Logistics Department of PLA [BLB19J005]; Guangzhou Science and Tech-nology Planning Project [202003000040]en_US
dc.identifier.citationXu, C., Yan, C., Jiang, M., Alenezi, F., Alhudhaif, A., Alnaim, N., ... & Wu, W. (2022). A novel facial emotion recognition method for stress inference of facial nerve paralysis patients. Expert Systems with Applications, 197, 116705.en_US
dc.identifier.doi10.1016/j.eswa.2022.116705
dc.identifier.endpage9en_US
dc.identifier.issn0957-4174
dc.identifier.issn1873-6793
dc.identifier.scopus2-s2.0-85125856059en_US
dc.identifier.scopusqualityQ1en_US
dc.identifier.startpage1en_US
dc.identifier.urihttp://dx.doi.org/10.1016/j.eswa.2022.116705
dc.identifier.urihttps://hdl.handle.net/20.500.12491/11766
dc.identifier.volume197en_US
dc.identifier.wosWOS:000792297500004en_US
dc.identifier.wosqualityQ1en_US
dc.indekslendigikaynakWeb of Scienceen_US
dc.indekslendigikaynakScopusen_US
dc.institutionauthorPolat, Kemal
dc.language.isoenen_US
dc.publisherPergamon-Elsevier Science Ltden_US
dc.relation.ispartofExpert Systems with Applicationsen_US
dc.relation.publicationcategoryMakale - Uluslararası Hakemli Dergi - Kurum Öğretim Elemanıen_US
dc.rightsinfo:eu-repo/semantics/closedAccessen_US
dc.subjectFacial Nerve Paralysisen_US
dc.subjectFacial Expression Recognitionen_US
dc.subjectEmotion Recognitionen_US
dc.subjectExpression Recognitionen_US
dc.subjectDisordersen_US
dc.subjectAttentionen_US
dc.titleA novel facial emotion recognition method for stress inference of facial nerve paralysis patientsen_US
dc.typeArticleen_US

Dosyalar

Orijinal paket
Listeleniyor 1 - 1 / 1
Küçük Resim Yok
Ä°sim:
cuiting-xu.pdf
Boyut:
5.32 MB
Biçim:
Adobe Portable Document Format
Açıklama:
Tam Metin/Full Text
Ä°ndir
Lisans paketi
Listeleniyor 1 - 1 / 1
Küçük Resim Yok
Ä°sim:
license.txt
Boyut:
1.44 KB
Biçim:
Item-specific license agreed upon to submission
Açıklama:
Ä°ndir

Koleksiyon

WoS İndeksli Yayınlar Koleksiyonu
Elektrik Elektronik Mühendisliği Bölümü Koleksiyonu
Scopus İndeksli Yayınlar Koleksiyonu