Ethologicovideographic Correlates of Mental Disorders in Nonverbal Behavior (Part I: Frequency and Duration Characteristics of Facial-Pantomimic Reactions)

Background: the lack of obvious signs of mental disorders available for objective registration is known problems of psychiatry. The study of nonverbal behavior based on the ethological paradigm using automatic detection technologies may be one of the approaches to solving this problem. The aim of study was to compare the facial and pantomimic activity of patients with neurotic disorders and pathology of the schizophrenia spectrum in a controlled ethologovideographic study to search for behavioral biomarkers of these disorders. Patients, Control Group and Methods: 19 patients with schizophrenia spectrum disorders (SSD), 23 with neurotic disorders (ND), and 22 healthy subjects of control group (CG) were examined. The severity of SSD symptoms was determined using the PANSS scale; for ND on used the Hamilton Anxiety (HAMA-14) and Depression (HAMD-17) scales. Analysis of non-verbal behavior was carried out using the biometric video analytics complex “MIX VR-19” based on action units (AU) of the Facial Action Coding System (FACS). Results: according to the frequency and duration of facial and pantomimic reactions, facial mimics in patients with Sch was characterized by greater activity in the areas of the mouth and eyes, with a decrease of mimic activity in the area of eyebrows and head movements. Frontal orientation towards the interviewer was less frequent, while a longer duration of sadness expression and a shorter duration of surprise reactions were revealed in ND patients. Factor analysis showed that AU formed the facial complexes “thoughtfulness”, “remembering”, “lack of negativism”, “lack of distrust” in control; “search for a solution”, “surprise”, “helplessness”, “embarrassment/shyness”, “distrust” characterized NR group while “incredulous surprise”, “experience of failure”, “skepticism/distrust”, “satisfaction/superiority” were found in SSD group. Conclusions: the mimic-pantomimic response of mentally healthy persons reflected cognitive processes and was characterized by emotional neutrality in the structure of behavioral complexes of attention and contact as well as of exploring behavior. A combination of cognitive and affective facial reactions was observed within the framework of avoidant-defensive agonistic behavior and neophobia in ND. Facial reactions of emotionally charged content predominated within the complex of preventive-aggressive agonistic behavior were the most frequent patterns of nonverbal behavior in SSD.

1. Bobrov AE. Methodological issues of mental disorders diagnosis and modern programs for training psychiatrists. Social and clinical psychiatry. 2014;24(2):50–54.

2. Samokhvalov VP. Evolutionary psychiatry. IMIS, 1993:286. (In Russ.).

3. Kornetov AN, Samokhvalov AA, Korobov AA, Kornetov NA. Ethology in psychiatry. Kyiv, 1990:217. (In Russ.).

4. Ganzin IV. Kinesic markers of mental disorders. Taurida journal of psychiatry Taurida journal of psychiatry = Acta psychiatrica, psychologica, psychotherapeutica et ethologica Tavrica. 2003;3:40–43. (In Russ.).

5. Scherer S, Stratou G, Mahmoud M. Boberg J, Gratch J, Rizzo A (Skip), Morency L-P. Automatic Behavior Descriptors for Psychological Disorder Analysis. Image and Vision Computing. doi: 10.1145/2522848.2522886

6. Pegoraro LF, Setz EZ, Dalgalarrondo P. Ethological approach to autism spectrum disorders Evol. Psychol. 2014;12(1):223–244. doi: 10.1177/147470491401200 116. PMID: 25299761; PMCID: PMC10481087.

7. Fiquer JT, Moreno RA, Brunoni AR, Barros VB, Fernandes F, Gorenstein C. What is the nonverbal communication of depression? Assessing expressive differences between depressive patients and healthy volunteers during clinical interviews. J Affect Disord. 2018 Oct 1;238:636–644. doi: 10.1016/j. jad.2018.05.071. Epub 2018 Jun 9. PMID: 29957481.

8. Gupta T. The Experience and Expression of Emotion in Psychosis-Risk. Evanston, Illinois, 2022:166. doi: 10.21985/n2-7k2a-4521

9. Shamrey VK, Marchenko AA, Lobachev AV, Tarumov DA. Modern methods of objectification of mental disorders in military personnel. Social and clinical psychiatry. 2021;31(2):51–57. (In Russ.).

10. Ekman P, Friesen WV, Hager JC. Facial Action Coding System. The Manual. Salt Lake City UT; 2002:514.

11. Fei Z, Yang E, Day-Uei Li D, Butler S, Ijomah W, Li X, Zhou H. Deep convolution network based emotion analysis towards mental health care. Neurocomputing. 2020;388:212–227. doi: 10.1016/j.neucom. 2020.01.034

12. Yao L, Wan Y, Ni H, Xu B. Action unit classification for facial expression recognition using active learning and SVM. Multimed Tools 2021; Appl 80, 24287–24301. doi: 10.1007/s11042-021-10836-w.

13. Allwood J, Cerrato L, Jokinen K, Navaretta K. The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena. Language Resources and Evaluation. 2007;41(3– 4):273–287. doi: 10.1007/s10579-007-9061-5

14. Marchenko AA. Neurotic disorders in military personnel: clinical presentation, diagnostics, pathomorphosis. SPb., 2009:46. (In Russ.).

15. Leucht S. Measurements of response, remission, and recovery in schizophrenia and examples for their clinical application. J Clin Psychiatry. 2014;75 Suppl 1:8–14. doi: 10.4088/JCP.13049su1c.02. PMID: 24581453.

16. Viola P, Jones MJ. Robust Real-time Object Detection. International Journal of Computer Vision. 2004;57(2):137–154.

17. Samokhvalov VP, Samokhvalova OE. Toward a Neuroethology of Schizophrenia: Findings from the Crimean Project. Handbook of Schizophrenia Spectrum Disorders, Volume II: Phenotypic and Endophenotypic Presentations. 2011;121-164.

18. Brüne M, Sonntag C, Abdel-Hamid M, Lehmkämper C, Juckel G, Troisi A. Nonverbal behavior during standardized interviews in patients with schizophrenia spectrum disorders. J Nerv Ment Dis. 2008 Apr;196(4):282–288. doi: 10.1097/NMD.0b013e31816a4922. PMID: 18414122.

19. Sidyakin VG. Issues of ethological physiology of humans and animals. Simferopol, 2000:464. (In Russ.).