Ethologovideographic Correlates of Mental Disorders in Nonverbal Behavior (Message 2: Discriminant, Thermodynamic and Network Characteristics of Facial and Pantomimic Reactions)

Background: the study of nonverbal behavior based on the ethological paradigm using computer vision technologies is currently considered as one of the approaches to objectification of mental disorders. At the same time, their analysis using multidimensional data processing methods, primarily from the standpoint of thermodynamic and network analysis, is of particular interest. Purpose of the work: to summarize the results of discriminant, thermodynamic and network analysis of facial and pantomimic activity of patients with neurotic disorders and schizophrenia spectrum pathology. Patients, Comparison Group and Methods: 19 patients with schizophrenia spectrum disorders (Sch), 23 with neurotic disorders (ND). 22 healthy subjects made up control group (CG). Analysis of non-verbal behavior was carried out using the biometric video analytics complex “MIX VR-19” based on action units (AU) of the FACS. Results: the logarithm of the thermodynamic probability of the system of facial reactions was maximal in individuals with ND (50.2), minimal in people with Sch (33.1), and it occupied an intermediate position in the CG (44.2). The system organization was also noticeably lower in Sch (0.08) than in ND (0.11) and in CG (0.14). Analysis of the graphs showed that the nodes page ranks in healthy subjects were characterized by the highest weight of AU61, AU1, AU63 and AU64. For ND, the nodes with maximum page ranks were AU14 and AU64. In Sch, the highest rank was observed for nodes AU62 and AU2. Conclusions: the greatest contribution to the differentiation between the groups of people with ND, Sch and CG was made by such AU as ironic smile, squinting and opening the mouth. The dynamics of entropy and organization of the facial-pantomimic reactions system in the CG reflected the tension of adaptation mechanisms at stages with a predominance of cognitive load; in Sch, there was an insufficiency of such tension during load, and in ND, tension was observed at stages with affective and personal themes. The graph of facial-pantomimic reactions in healthy individuals was determined by integrativeness with a large number of connections between nodes; In patients with HP, the graph was sequentially connected, while in individuals with SR it was represented by isolated clusters of AU.

1. Marchenko AA, Lobachev AV, Vinogradova OS, Moiseev DV, Dmitriev PI, Shchelkanova ES, Nazarova MR, Volodarskaya AA, Rudakova KV, Dang VCh. Ethologovideographic correlates of mental disorders in military personnel (Part I: Frequency and Duration Characteristics of Facial-Pantomimic Reactions). Psychiatry (Moscow) (Psikhiatriya). 2024;22(6):43–52. (In Russ.). doi: 10.30629/2618-6667-2024-22-6-54-62

2. Bishay M, Palasek P, Priebe S, Patras I. SchiNet: Automatic Estimation of Symptoms of Schizophrenia from Facial Behaviour Analysis. IEEE Transactions on Affective Computing. 2021;12(4):949–961. doi: 10.1109/taffc.2019.2907628

3. Joshi J, Goecke R, Alghowinem S, Dhall A. Multimodal assistive technologies for depression diagnosis and monitoring. J. Multimodal User Interfaces. 2013;7:217–228. doi: 10.1007/s12193-013-0123-2

4. de Melo WC, Granger E, Hadid A. Combining Global and Local Convolutional 3D Networks for Detecting Depression from Facial Expressions. In: 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019). Lille, France, 2019:1–8.

5. Scherer S, Stratou G, Mahmoud M., Boberg J,. Gratch J,. Rizzo A, Morency L-P. Automatic Behavior Descriptors for Psychological Disorder Analysis. Image and Vision Computing. 2013;32. doi: 10.1109/FG.2013.6553789

6. Liu Y, Zhang X, Lin Y, Wang H. Facial expression recognition via deep action units graph network based on psychological mechanism. IEEE Transactions on Cognitive and Developmental Systems. 2019;12(2):311–322. doi: 10.1002/aisy.202400524

7. Jeganathan J, Campbell M, Hyett M. et al. Quantifying dynamic facial expressions under naturalistic conditions. Elife. 2022;11:e79581.

8. Jack RE, Garrod OGB, Schyns PG. Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Curr Biol. 2014 Jan 20;24(2):187– 192. doi: 10.1016/j.cub.2013.11.064 Epub 2014 Jan 2. PMID: 24388852.

9. Khan RA, Meyer A, Konik H, Bouakaz S. Facial expression recognition using entropy and brightness features. In: Proc. 11th Int. Conf. Intell. Syst. Design Appl. (ISDA). 2011:737–742.

10. Myznikov IL, Nabokov NL, Rogovanov DYu, Khankevich YR. Description and presentation of the results of electroencephalogram processing using information models. Aerospace and environmental medicine. 2016;50(1):66–72. (In Russ.).

11. Samokhvalov VP, Samokhvalova OE. Toward a Neuroethology of Schizophrenia: Findings from the Crimean Project. In: M.S. Ritsner (ed.) Handbook of Schizophrenia Spectrum Disorders, Volume II. Phenotypic and Endophenotypic Presentations. — Springer Dordrecht Heidelberg London New York, 2011:121–164.

12. Marconi M, Do Carmo Blanco N, Zimmer C, Guyon A. Eye movements in response to different cognitive activities measured by eyetracking: a prospective study on some of the neurolinguistics programming theories. J Eye Mov Res. 2023;16(2):2. doi: 10.16910/jemr.16.2.2

13. Rumke H. Het Kernsymptoom der Schizophrenie en het Praecoxgevoel. Studies en Voordrachten over Psychiatrie. Amsterdam, 1948:53–58.

14. Chaika YuV. History, structure and prospects of development of the psychopathological method (message 2). Ukrainian Bulletin of Psychoneurology. 2004;12(4):12–16.

15. Sass L, Pienkos E, Skodlar B, Stanghellini G, Fuchs T, Parnas J, Jones N. EAWE: Examination of Anomalous World Experience. Psychopathology. 2017;50(1):10– 54. doi: 10.1159/000454928 Epub 2017 Mar 8. PMID: 28268224.

16. Anderzhanova E, Kirmeier T, Wotjak CT. Animal models in psychiatric research: The RDoC system as a new framework for endophenotype-oriented translational neuroscience. Neurobiol Stress. 2017;25(7):47–56. doi: 10.1016/j.ynstr.2017.03.003

17. Belzer KD, Schneier FR. Tools for Assessing Generalized Anxiety Disorder. Psychiatric Time. 2006;25(3):4–6.

18. Brüne M, Belsky J, Fabrega H, Feierman HR, Gilbert P, Glantz K, Polimeni J, Price JS, Sanjuan J, Sullivan R, Troisi A, Wilson DR. The crisis of psychiatry — insights and prospects from evolutionary theory. World Psychiatry. 2012 Feb;11(1):55–7. doi: 10.1016/j.wpsyc.2012.01.009 PMID: 22295011; PMCID: PMC3266750.

19. Kornetov AN, Samokhvalov AA, Korobov AA, Kornetov NA. Ethology in psychiatry. Kyiv, 1990:217. (In Russ.).

20. Samokhvalov VP. Evolutionary psychiatry. IMIS, 1993:286. (In Russ.).