A Web-based Game for Teaching Facial Expressions to Schizophrenic Patients
16 October 2016
accepted: 27 April 2017
20 December 2017 (online)
Background: Recognizing facial expressions is an important social skill. In some psychological disorders such as schizophrenia, loss of this skill may complicate the patient’s daily life. Prior research has shown that information technology may help to develop facial expression recognition skills through educational software and games.
Objectives: To examine if a computer game designed for teaching facial expressions would improve facial expression recognition skills of patients with schizophrenia.
Methods: We developed a website composed of eight serious games. Thirty-two patients were given a pre-test composed of 21 facial expression photographs. Eighteen patients were in the study group while 14 were in the control group. Patients in the study group were asked to play the games on the website. After a period of one month, we performed a post-test for all patients.
Results: The median score of the correct answers was 17.5 in the control group whereas it was 16.5 in the study group (of 21) in pretest. The median post-test score was 18 in the control group (p=0.052) whereas it was 20 in the study group (p<0.001).
Conclusions: Computer games may be used for the purpose of educating people who have difficulty in recognizing facial expressions.
Citation: Gülkesen KH, Işleyen F, Cinemre B, Şamur MK, Sen Kaya S, Zayim N. A web-based game for teaching facial expressions to schizophrenic patients. Appl Clin Inform 2017; 8: 719–730 https://doi.org/10.4338/ACI-2016-10-RA-0172
Clinical Relevance Statement
The use of computer games, or in other words serious games, should be investigated to improve individual health in those with various medical conditions.
Human Subjects Protections
The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and the study protocol was approved by Akdeniz University Ethical Committee of Clinical Research.
- 1 Kohler CG, Walker JB, Martin EA, Healey KM, Moberg PJ. Facial emotion perception in schizophrenia: a meta-analytic review. Schizophr Bull 2010; 36: 1009-1019.
- 2 Laroi F, Fonteneau B, Mourad H. A Raballo. Basic emotion recognition and psychopathology in schizophrenia. J Nerv Ment Dis 2010; 198: 79-81.
- 3 Corbett BA, Carmean V, Ravizza S, Wendelken C, Henry ML, Carter C. SM Rivera.. A functional and structural study of emotion and face processing in children with autism. Psychiatry Res 2009; 173: 196-205.
- 4 Wolwer W, Frommann N, Halfmann S, Piaszek A, Streit M, Gaebel W. Remediation of impairments in facial affect recognition in schizophrenia: efficacy and specificity of a new training program. Schizophr Res 2005; 80: 295-303.
- 5 Frommann N, Streit M, Wolwer W. Remediation of facial affect recognition impairments in patients with schizophrenia: a new training program. Psychiatry Res 2003; 117: 281-284.
- 6 Silver H, Goodman C, Knoll G, Isakov V. Brief emotion training improves recognition of facial emotions in chronic schizophrenia. A pilot study. Psychiatry Res 2004; 128: 147-154.
- 7 Russell TA, Chu E, Phillips ML. A pilot study to investigate the effectiveness of emotion recognition remediation in schizophrenia using the micro-expression training tool. Br J Clin Psychol 2006; 45: 579-583.
- 8 Bergeron BP. Developing Serious Games. Hingham, USA: Charles River Media; 2006
- 9 Ekman P, Friesen WV, O’Sullivan M, Chan A, Diacoyanni-Tarlatzis I, Heider K, Krause R, LeCompte WA, Pitcairn T, Ricci-Bitti PE, Scherer K, Tomita M, Tzavaras A. Universals and cultural differences in the judgments of facial expressions of emotion. J Pers Soc Psychol 1987; 53: 712-717.
- 10 Ekman P, Friesen WV. Constants across cultures in the face and emotion. J Pers Soc Psychol 1971; 17: 124-129.
- 11 Gaebel W, Wolwer W. Facial expression and emotional face recognition in schizophrenia and depression. Eur Arch Psychiatry Clin Neurosci 1992; 242: 46-52.
- 12 Bach DR, Buxtorf K, Grandjean D, Strik WK. The influence of emotion clarity on emotional prosody identification in paranoid schizophrenia. Psychol Med 2009; 39: 927-938.
- 13 Addington J, Addington D. Facial affect recognition and information processing in schizophrenia and bipolar disorder. Schizophr Res 1998; 32: 171-181.
- 14 Huang Y, Tang S, Helmeste D, Shioiri T, Someya T. Differential judgement of static facial expressions of emotions in three cultures. Psychiatry Clin Neurosci 2001; 55: 479-483.
- 15 Ekman P, Friesen WV. Constants across cultures in the face and emotion. J Pers Soc Psychol 1971; 17: 124-129.
- 16 Gulkesen KH, Cinemre B, Isleyen F, Zayim N, Kemal M. What are the characteristics of people who are successful on recognizing human facial expressions on computer monitor?. Proceedings of the Biomedical Engineering and Informatics [BMEI], 4th International Conference. ; 15–17 October 2011; Shanghai, China: IEEE Computer Society; 2011; 4 (09) pp. 1901-1903.
- 17 Nielsen J, Mack RL. Usability inspection methods. New York, USA: Wiley; 1994
- 18 Hudson W. How many users does it take to change a Web site?. ACM SIGCHI Bulletin –a supplement to interactions. 2001 ; 2001 [May-June]: 6.
- 19 Isleyen F, Gulkesen KH, Cinemre B, Samur MK, Zayim N, Sen Kaya S. Evaluation of the usability of a serious game aiming to teach facial expressions to schizophrenic patients. Stud Health Technol Inform 2014; 205: 662-666.
- 20 Association AP. Schizophrenia; Diagnostic and statistical manual of mental disorders: DSM-IV. Washington, DC: American Psychiatric Publishing; 2000
- 21 Andreasen NC. The Scale for the Assessment of Negative Symptoms [SANS]: conceptual and theoretical foundations. Br J Psychiatry Suppl 1989: 49-58.
- 22 Erkoç, Arkonaç O. Ataklı C, Özmen E. Negatif semptomlan de[gbreve]erlendirme ölçe[gbreve]inin güvenilirli[gbreve]i ve geçerlili[gbreve]i. Dü ünen Adam 1991; 4 (02) 14-15.
- 23 Andreasen NC. Scale for the Assessment of Positive Symptoms [SANS]. Iowa City, USA: University of Iowa; 1984
- 24 Erkoç, Arkonaç O, Ataklı C, Özmen E. Pozitif Semptomları De[gbreve]erlendirme Ölçe[gbreve]inin Güvenilirli[gbreve]i ve Geçerlili[gbreve]i. Dü ünen Adam 1991; 4: 20-24.
- 25 Overall JE, Gorham DR. The Brief Psychiatric Rating Scale. Psychol Rep 1962; 10: 799-812.
- 26 Soykan Ç. Institutional differences and case typically related to diagnosis, symptom severity, prognosis and treatment. MSc, Middle East Technical University, Ankara. Turkey: 1990
- 27 Zangwill OL. Clinical Tests of Memory Impairment. Proc R Soc Med 1943; 36 (11) 576-580.
- 28 Karaka S. B[Idotaccent]LNOT Bataryası El Kitabı. Ankara, Turkey: Eryılmaz Offset Matbaacılık Gazetecilik Ltd. ti; 2006
- 29 Çitak S, Çakıcı ET, Çakıcı M. Neuropsychological assessment in schizophrenic patients treated with haloperidol or clozapine. BCP 2009; 19: 5-14.
- 30 Berg EA. A simple objective technique for measuring flexibility in thinking. J Gen Psychol. 1948; 1948: 15-22.
- 31 Heaton RK. The Wisconsin Card Sorting Test manual: Odessa, USA: Psychological Assessment Resources Inc.;. 1981
- 32 Karaka S, Kafadar H. izofrenideki Bili sel Süreçlerin De[gbreve]erlendirilmesinde Nöropsikolojik Testler: Bellek ve Dikkatin Ölçülmesi. izofreni Dizisi 1999; 4: 132-152.
- 33 Porteus SD. The Porteus maze test and intelligence. Palo Alto, USA: Pacific Books; 1950
- 34 Merry SN, Stasiak K, Shepherd M, Frampton C, Fleming T, Lucassen MF. The effectiveness of SPARX, a computerised self help intervention for adolescents seeking help for depression: randomised controlled non-inferiority trial. BMJ 2012; 344: e2598.
- 35 Walshe DG, Lewis EJ, Kim SI, O’Sullivan K, Wiederhold BK. Exploring the use of computer games and virtual reality in exposure therapy for fear of driving following a motor vehicle accident. Cyberpsychol Behav 2003; 6: 329-334.
- 36 Santamaría JJ, Soto A, Fernandez-Aranda F, Krug I, Forcano L, Gunnard K, Kalapanidas E, Lam T, Raguin T, Davarakis C, Menchon JM, Jimenez-Murcia S. Serious games as additional psychological support: A review of the literature. J Cyber Ther Rehabil 2011; 4: 469-476.
- 37 Silver M, Oakes P. Evaluation of a new computer intervention to teach people with autism or Asperger syndrome to recognize and predict emotions in others. Autism 2001; 5: 299-316.
- 38 Du S, Martinez AM. The resolution of facial expressions of emotion. J Vis 2011; 11: 24.
- 39 Biehl M, Matsumoto D, Ekman P, Hearn V, Heider K, Kudoh T, Ton V. Matsumoto and Ekman‘s Japanese and Caucasian Facial Expressions of Emotion [JACFEE]: Reliability Data and Cross-National Differences. J Nonverbal Behav 1997; 21: 3-21.
- 40 Palermo R, Coltheart M. Photographs of facial expression: accuracy, response times, and ratings of intensity. Behav Res Methods Instrum Comput 2004; 36: 634-638.
- 41 Kohler CG, Turner TH, Bilker WB, Brensinger CM, Siegel SJ, Kanes SJ, Gur RE, Gur RC. Facial emotion recognition in schizophrenia: intensity effects and error pattern. Am J Psychiatry 2003; 160: 1768-1774.
- 42 Elfenbein HA, Ambady N. Universals and cultural differences in recognizing emotions. Psychol Sci 2003; 12: 159-164.
- 43 Peterson LR, Peterson MJ. Short-term retention of individual verbal items. J Exp Psychol 1959; 58: 193-198.
- 44 Karpicke JD, III HLR. Repeated retrieval during learning is the key to long-term retention. J Mem Lang 2007; 57: 151-162.
- 45 Roediger HL, Karpicke JD. Test-enhanced learning: taking memory tests improves long-term retention. Psychol Sci 2006; 17: 249-255.
- 46 Sachs G, Steger-Wuchse D, Kryspin-Exner I, Gur RC, Katschnig H. Facial recognition deficits and cognition in schizophrenia. Schizophr Res 2004; 68: 27-35.
- 47 Bozikas VP, Kosmidis MH, Anezoulaki D, Giannakou M, Karavatos A. Relationship of affect recognition with psychopathology and cognitive performance in schizophrenia. J Int Neuropsychol Soc 2004; 10: 549-558.
- 48 Bryson G, Bell M, Lysaker P. Affect recognition in schizophrenia: a function of global impairment or a specific cognitive deficit. Psychiatry Res 1997; 71: 105-113.