The Role of Auditory Status and Emotion Intensity in Facial Emotion Recognition by Adolescents

Abstract

Poor emotion recognition has been linked to social and emotional problems in everyday life. This may particularly affect teenagers navigating social scenes in adolescence, when peers become more important than family, and those with disabilities like hearing loss that may affect emotion recognition. This study investigates the effect of auditory status (deaf or hard of hearing using cochlear implant, CI, versus typical hearing, TH) on visual emotion recognition in adolescents, and its association with social relationships. Participants included 24 adolescents with TH (M = 13.7 years) and 34 adolescent long-term CI users (M = 13.3 years). Adolescents completed an emotion recognition task viewing video clips of faces morphing from a neutral expression to six facial expressions at 60, 80, or 100% emotion intensities via a gating task, and questionnaires about social relationships. Adolescents with TH significantly outperformed the CI group in the 80% emotion intensity condition (p = 0.02), but not the other conditions (p > 0.05). Better visual emotion recognition in CI users coincided with older age, better speech in noise ability, and more positive friendship quality. Significant performance differences with 80% emotion intensity suggest the CI group processes facial expressions qualitatively differently from TH peers, which has implications for therapeutic intervention for social skills in CI users.

Keywords

adolescent - deaf or hard of hearing - typical hearing - visual emotion recognition - videos

Publication History

Article published online:
19 February 2026

• References

• Armstrong, B.A., Neville, H.J., Hillyard, S.A., Mitchell, T.V. (2002). Auditory deprivation affects processing of motion, but not color. Brain Res Cogn Brain Res, 14(3):422–434.
• Asher, S.R., Hymel, S., Renshaw, P.D. (1984). Loneliness in children. Child Dev, 55(4):1456–1464. https://doi.org/10.2307/1130015
• Bal, E., Harden, E., Lamb, D., Van Hecke, A.V., Denver, J.W., Porges, S.W. (2010). Emotion recognition in children with autism spectrum disorders: relations to eye gaze and autonomic state. J Autism Dev Disord, 40(3):358–370.
• Barrett, H.C., Bryant, G. (2008). Vocal emotion recognition across disparate cultures. J Cogn Cult, 8(1–2):135–148. https://doi.org/10.1163/156770908X289242
• Bettger, J., Emmorey, K., McCullough, S., Bellugi, U. (1997). Enhanced facial discrimination: effects of experience with American sign language. J Deaf Stud Deaf Educ, 2(4):223–233.
• Blair, R.J. (2003). Facial expressions, their communicatory functions and neuro-cognitive substrates. Philos Trans R Soc Lond B Biol Sci, 358(1431):561–572.
• Brown, T.T., Erhart, M., Avesar, D., Dale, A.M., Halgren, E., Evans, J.L. (2014). Atypical right hemisphere specialization for object representations in an adolescent with specific language impairment. Front Hum Neurosci, 8:82.
• Chatterjee, M., Gajre, S., Kulkarni, A.M., Barrett, K.C., Limb, C.J. (2024). Predictors of emotional prosody identification by school-age children with cochlear implants and their peers with normal hearing. Ear Hear, 45(2):411–424.
• Ching, T.Y., Day, J., Van Buynder, P., et al. (2014). Language and speech perception of young children with bimodal fitting or bilateral cochlear implants. Cochlear Implants Int, 15(0 1, Suppl 1):S43–S46.
• Chronaki, G., Hadwin, J.A., Garner, M., Maurage, P., Sonuga-Barke, E.J. (2015). The development of emotion recognition from facial expressions and non-linguistic vocalizations during childhood. Br J Dev Psychol, 33(2):218–236.
• Cupples, L., Ching, T.Y., Button, L., et al. (2018). Spoken language and everyday functioning in 5-year-old children using hearing aids or cochlear implants. Int J Audiol, 57(sup2):S55–S69.
• Della Longa, L., Nosarti, C., Farroni, T. (2022). Emotion recognition in preterm and full-term school-age children. Int J Environ Res Public Health, 19(11):6507–6526.
• Dettman, S.J., Dowell, R.C., Choo, D., et al. (2016). Long-term communication outcomes for children receiving cochlear implants younger than 12 months: a multicenter study. Otol Neurotol, 37(2):e82–e95. https://doi.org/10.1097/MAO.0000000000000915
• Dyck, M.J., Farrugia, C., Shochet, I.M., Holmes-Brown, M. (2004). Emotion recognition/understanding ability in hearing or vision-impaired children: do sounds, sights, or words make the difference? J Child Psychol Psychiatry, 45(4):789–800.
• Ekman, P., Friesen, W.V. (1976). Pictures of Facial Affect. Consulting Psychologists Press
• Gagnon, E.B., Eskridge, H., Brown, K.D., Park, L.R. (2021). The impact of cumulative cochlear implant wear time on spoken language outcomes at age 3 years. J Speech Lang Hear Res, 64(4):1369–1375.
• Gray, C., Hosie, J., Russell, P., Ormel, E. (2002). Emotional development in deaf children: facial expressions, display rules, and theory of mind. In M. D. Clark, M. Marschark, & M. Karchmer (Eds.), Context, Cognition, and Deafness (pp. 135–157). Gallaudet University Press
• Greenberg, M.T., Kusche, C.A. (1993). Promoting Social and Emotional Development in Deaf Children: The PATHS Project. University of Washington Press
• Herba, C., Phillips, M. (2004). Annotation: Development of facial expression recognition from childhood to adolescence: behavioural and neurological perspectives. J Child Psychol Psychiatry, 45(7):1185–1198.
• Herba, C.M., Landau, S., Russell, T., Ecker, C., Phillips, M.L. (2006). The development of emotion-processing in children: effects of age, emotion, and intensity. J Child Psychol Psychiatry, 47(11):1098–1106.
• Hess, U., Blairy, S., Kleck, R.E. (1997). The intensity of emotional facial expressions and decoding accuracy. J Nonverbal Behav, 21(4):241–257. 10.1023/A:1024952730333
• Hoffman, M.F., Cejas, I., Quittner, A.L.; CDaCI Investigative Team. (2016). Comparisons of longitudinal trajectories of social competence: parent ratings of children with cochlear implants versus hearing peers. Otol Neurotol, 37(2):152–159.
• Hoffman, M.F., Quittner, A.L., Cejas, I. (2015). Comparisons of social competence in young children with and without hearing loss: a dynamic systems framework. J Deaf Stud Deaf Educ, 20(2):115–124.
• Hopyan-Misakyan, T.M., Gordon, K.A., Dennis, M., Papsin, B.C. (2009). Recognition of affective speech prosody and facial affect in deaf children with unilateral right cochlear implants. Child Neuropsychol, 15(2):136–146.
• Huber, M., Burger, T., Illg, A., et al. (2015). Mental health problems in adolescents with cochlear implants: peer problems persist after controlling for additional handicaps. Front Psychol, 6:953.
• Izard, C., Fine, S., Schultz, D., Mostow, A., Ackerman, B, Youngstrom, E. (2001). Emotion knowledge as a predictor of social behavior and academic competence in children at risk. Psychol Sci, 12(1):18–23.
• Jack, R.E., Garrod, O.G.B., Schyns, P.G. (2014). Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Curr Biol, 24(2):187–192.
• Jones, A.C., Gutierrez, R., Ludlow, A.K. (2018). The role of motion and intensity in deaf children's recognition of real human facial expressions of emotion. Cogn Emot, 32(1):102–115.
• Karltorp, E., Eklöf, M., Östlund, E., Asp, F., Tideholm, B., Löfkvist, U. (2020). Cochlear implants before 9 months of age led to more natural spoken language development without increased surgical risks. Acta Paediatr, 109(2):332–341.
• Kessels, R.P., Gerritsen, L., Montagne, B., Ackl, N., Diehl, J., Danek, A. (2007). Recognition of facial expressions of different emotional intensities in patients with frontotemporal lobar degeneration. Behav Neurol, 18(1):31–36.
• Kessler, H., Doyen-Waldecker, C., Hofer, C., Hoffmann, H., Traue, H.C., Abler, B. (2011). Neural correlates of the perception of dynamic versus static facial expressions of emotion. Psychosoc Med, 8:Doc03.
• Kilts, C.D., Egan, G., Gideon, D.A., Ely, T.D., Hoffman, J.M. (2003). Dissociable neural pathways are involved in the recognition of emotion in static and dynamic facial expressions. Neuroimage, 18(1):156–168.
• Kouwenberg, M., Rieffe, C., Theunissen, S.C., de Rooij, M. (2012). Peer victimization experienced by children and adolescents who are deaf or hard of hearing. PLoS One, 7(12):e52174.
• Ludemann, P. M., & Nelson, C. A. (1988). Categorical representation of facial expressions by 7-month-old infants. Developmental Psychology, 24(4), 492–501. https://doi.org/10.1037/0012-1649.24.4.492
• McDonald, S., Saunders, J.C. (2005). Differential impairment in recognition of emotion across different media in people with severe traumatic brain injury. J Int Neuropsychol Soc, 11(4):392–399.
• Moeller, M.P., Schick, B. (2006). Relations between maternal input and theory of mind understanding in deaf children. Child Dev, 77(3):751–766.
• Montagne, B., Kessels, R.P., De Haan, E.H., Perrett, D.I. (2007). The Emotion Recognition Task: a paradigm to measure the perception of facial emotional expressions at different intensities. Percept Mot Skills, 104(2):589–598.
• Morgan, G., Meristo, M., Mann, W., Hjelmquist, E., Surian, L., Siegal, M. (2014). Mental state language and quality of conversational experience in deaf and hearing children. Cogn Dev, 29:41–49. https://doi.org/10.1016/j.cogdev.2013.10.002
• Most, T., Aviner, C. (2009). Auditory, visual, and auditory-visual perception of emotions by individuals with cochlear implants, hearing aids, and normal hearing. J Deaf Stud Deaf Educ, 14(4):449–464.
• Most, T., Michaelis, H. (2012). Auditory, visual, and auditory-visual perceptions of emotions by young children with hearing loss versus children with normal hearing. J Speech Lang Hear Res, 55(4):1148–1162.
• Nelson, N.L., Mondloch, C.J. (2018). Children's visual attention to emotional expressions varies with stimulus movement. J Exp Child Psychol, 172:13–24.
• Nicholas, J.G., Geers, A.E. (2003). Personal, social, and family adjustment in school-aged children with a cochlear implant. Ear Hear, 24(1, Suppl):69S–81S.
• Niparko, J.K., Tobey, E.A., Thal, D.J., et al; CDaCI Investigative Team. (2010). Spoken language development in children following cochlear implantation. JAMA, 303(15):1498–1506.
• Nittrouer, S., Lowenstein, J.H. (2026). Language and literacy skills in children with cochlear implants: past and present findings. In N. M. Young & K. I. Kirkk (Eds.), Pediatric Cochlear Implantation: Learning and the Brain (2nd ed., pp. 293–316). Springer
• Park, L.R., Gagnon, E.B., Thompson, E., Brown, K.D. (2019). Age at full-time use predicts language outcomes better than age of surgery in children who use cochlear implants. Am J Audiol, 28(4):986–992.
• Parker, J.G., Asher, S.R. (1993). Friendship and friendship quality in middle childhood: links with peer group acceptance and feelings of loneliness and social dissatisfaction. Dev Psychol, 29(4):611–621. https://doi.org/10.1037/0012-1649.29.4.611
• Peterson, C.C., Siegal, M. (1995). Deafness, conversation and theory of mind. J Child Psychol Psychiatry, 36(3):459–474.
• Philippot, P., Feldman, R.S. (1990). Age and social competence in preschoolers' decoding of facial expression. Br J Soc Psychol, 29(Pt 1):43–54.
• Punch, R., Hyde, M. (2011). Social participation of children and adolescents with cochlear implants: a qualitative analysis of parent, teacher, and child interviews. J Deaf Stud Deaf Educ, 16(4):474–493.
• Ren, L., Zhang, Y., Zhang, J., et al. (2022). Voice emotion recognition by Mandarin-speaking children with cochlear implants. Ear Hear, 43(1):165–180.
• Rodger, H., Lao, J., Stoll, C., et al. (2021). The recognition of facial expressions of emotion in deaf and hearing individuals. Heliyon, 7(5):e07018.
• Sato, W., Yoshikawa, S. (2004). BRIEF REPORT: the dynamic aspect of emotional facial expressions. Cogn Emot, 18(5):701–710. https://doi.org/10.1080/02699930341000176
• Smith, M.L., Cottrell, G.W., Gosselin, F., Schyns, P.G. (2005). Transmitting and decoding facial expressions. Psychol Sci, 16(3):184–189.
• Tomblin, J.B., Harrison, M., Ambrose, S.E., Walker, E.A., Oleson, J.J., Moeller, M.P. (2015). Language outcomes in young children with mild to severe hearing loss. Ear Hear, 36(0 1, Suppl 1):76S–91S.
• Tonks, J., Williams, W.H., Frampton, I., Yates, P., Slater, A. (2007). Reading emotions after child brain injury: a comparison between children with brain injury and non-injured controls. Brain Inj, 21(7):731–739.
• Uhler, K.M., Gilley, P.M., Hunnicut, C., Madrid, A., Warner-Czyz, A.D. (2026). Is there a relationship between infant and toddler speech discrimination abilities and mean length of utterance at 33 months? J Speech Lang Hear Res, 1-19. https://doi.org/10.1044/2025_JSLHR-24-00719
• Walle, E.A., Campos, J.J. (2012). Interpersonal responding to discrete emotions: a functionalist approach to the development of affect specificity. Emot Rev, 4(4):413–422.
• Wang, Y., Su, Y., Fang, P., Zhou, Q. (2011). Facial expression recognition: can preschoolers with cochlear implants and hearing aids catch it? Res Dev Disabil, 32(6):2583–2588.
• Wang, Y., Su, Y., Yan, S. (2016). Facial expression recognition in children with cochlear implants and hearing aids. Front Psychol, 7:1989.
• Warner-Czyz, A.D., Evans, D., Turkstra, L., Scheppele, M., Song, C., Evans, J.L. (2019). Effect of auditory status on visual emotion recognition in adolescents. Cochlear Implants Int, 20(3):127–137.
• Warner-Czyz, A.D., Loy, B., Pourchot, H., White, T., Cokely, E. (2018). Effect of hearing loss on peer victimization in school-age children. Except Child, 84(3):280–297. https://doi.org/10.1177/0014402918754880
• Wiefferink, C.H., Rieffe, C., Ketelaar, L., De Raeve, L., Frijns, J.H. (2013). Emotion understanding in deaf children with a cochlear implant. J Deaf Stud Deaf Educ, 18(2):175–186.
• Wiefferink, C.H., Rieffe, C., Ketelaar, L., Frijns, J.H. (2012). Predicting social functioning in children with a cochlear implant and in normal-hearing children: the role of emotion regulation. Int J Pediatr Otorhinolaryngol, 76(6):883–889.
• Young-Browne, G., Rosenfeld, H.M., Horowitz, F.D. (1977). Infant discrimination of facial expressions. Child Dev, 48(2):555–562. https://doi.org/10.2307/1128653
• Ziv, M., Most, T., Cohen, S. (2013). Understanding of emotions and false beliefs among hearing children versus deaf children. J Deaf Stud Deaf Educ, 18(2):161–174.