J Am Acad Audiol 2018; 29(10): 875-884
DOI: 10.3766/jaaa.17030
Articles
Thieme Medical Publishers 333 Seventh Avenue, New York, NY 10001, USA.

The Effect of Presentation Mode and Production Type on Word Memory for Hearing Impaired Signers

Riki Taitelbaum Swead
*   Department of Communication Disorders, Ariel University, Ariel, Israel
†   Meuhedet Health Services, Tel Aviv, Israel
,
Yaniv Mama
‡   Department of Behavioral Sciences and Psychology, Ariel University, Ariel, Israel
,
Michal Icht
*   Department of Communication Disorders, Ariel University, Ariel, Israel
› Author Affiliations
Further Information

Corresponding author

Riki Taitelbaum Swead
Department of Communication Disorders
Ariel University, Ariel
Israel   

Publication History

Publication Date:
29 May 2020 (online)

 

Abstract

Background:

Production effect (PE) is a memory phenomenon referring to better memory for produced (vocalized) than for non-produced (silently read) items. Reading aloud was found to improve verbal memory for normal-hearing individuals, as well as for cochlear implant users, studying visually and aurally presented material.

Purpose:

The present study tested the effect of presentation mode (written or signed) and production type (vocalization or signing) on word memory in a group of hearing impaired young adults, sign-language users.

Research Design:

A PE paradigm was used, in which participants learned lexical items by two presentation modes, written or signed. We evaluated the efficacy of two types of productions: vocalization and signing, using a free recall test.

Study Sample:

Twenty hearing-impaired young adults, Israeli sign language (ISL) users, participated in the study, ten individuals who mainly use manual communication (MC) (ISL as a first language), and ten who mainly use total communication (TC).

Data Collection and Analysis:

For each condition, we calculated the proportion of study words recalled. A mixed-design analysis of variance was conducted, with learning condition (written-vocalize, written-signed, and manual-signed) and production type (production and no-production) as within-subject variables, and group (MC and TC) as a between-subject variable.

Results:

Production benefit was documented across all learning conditions, with better memory for produced over non-produced words. Recall rates were higher when learning written words relative to signed words. Production by signing yielded better memory relative to vocalizing.

Conclusions:

The results are explained in light of the encoding distinctiveness account, namely, the larger the number of unique encoding processes involved at study, the better the memory benefit.


#

Reading words aloud improves their memory relative to reading them silently, a phenomenon known as production effect (PE; [Macleod et al, 2010]). As its name implies, the memory advantage does not necessarily relate to vocalizing (reading aloud), but to “producing.” In fact, many types of productions were found to improve memory relative to silent reading, such as mouthing, writing, typing, and singing ([Forrin et al, 2012]; [Quinlan and Taylor, 2013]). Common to these productions is the active execution of a distinct, item-specific response.

The benefit of production was documented for a variety of study materials, e.g., nonwords ([Macleod et al, 2010], Experiment 6), text ([Ozubko et al, 2012]), and pictures ([Icht and Mama, 2015]). It was also found for auditory-presented words ([Mama and Icht, 2016a]; [Taitelbaum Swead et al, 2017]). Many populations show a PE, from preschool children ([Icht and Mama, 2015]) to younger and older adults ([Lin and MacLeod, 2012]). Therefore, it was offered as a simple yet efficient mnemonic, improving long-term verbal memory. The present study tested the PE in a group of hearing impaired young adults, sign-language users (proficient Hebrew readers). Half of them used manual communication (MC) (Israeli sign language, ISL) as their preferred modality, and half used total communication (TC) (sign and oral language). These participants learned written or manually signed study items. We compared the benefit they gained from two types of productions: vocalization and signing.

The underlying mechanism of the PE is encoding distinctiveness ([Macleod et al, 2010]). According to this account, production at study creates a unique processing dimension that makes the produced items more distinct relative to other non-produced items. This distinctiveness can be used heuristically at test. Because the produced items are qualitatively more distinct and salient at study, they are better recognized or retrieved than the non-produced items at test ([Forrin et al, 2014]). Possibly, a distinctive element forms part of the encoding record of each word read aloud, an element not shared by the silent words that were studied in the same list ([Ozubko and Macleod, 2010]). Indeed, production benefit is prominent mainly when studying mixed lists, in which the produced and non-produced items are blended in a single list.

Encoding distinctiveness has been also interpreted from a quantitative perspective, by the number of distinct encoding processes that take place at study. The greater their number, the higher the memory benefit ([Forrin et al, 2012]; [Mama and Icht, 2016a]). In other words, the production that includes more distinct, nonoverlapping processes provides a larger boost to memory. As noted by [Forrin et al (2012)], “the more processes involved in the encoding, the more that can later be replayed to aid recognition—and distinctive processes should be particularly memorable because they are deeply encoded at study” (p. 1047).

Consider a typical PE experiment, with words learned by silent or aloud reading. Silent reading involves a single visual encoding process. However, reading aloud includes three encoding processes: visual (seeing the word), sensory motor (articulating it), and auditory (hearing one’s own voice). Thus, items that are read aloud at study are better remembered at test, whether recognition ([MacLeod, 2011]; [Forrin et al, 2012]; [Ozubko et al, 2012]) or free recall ([Lin and MacLeod, 2012]; [Icht et al, 2014]; [Mama and Icht, 2016b]).

Importantly, the number of distinct encoding processes involved in learning varies depending on the presentation mode (e.g., visual or auditory) and the production type. For example, while studying aurally presented words, saying aloud involves two distinct processes: auditory (hearing the words, once during their presentation and again at production) and sensory motor (articulation). Writing aurally presented words includes three unique encoding procedures: auditory (hearing the words), sensory motor (writing), and visual (seeing the written word). Indeed, [Mama and Icht (2016a)] found that production by writing was superior to vocalization when the study items were aurally presented (heard).

Recently, the benefit of vocal production (relative to no-production) was documented for prelingually deafened cochlear implant (CI) users, across the visual and auditory modalities ([Taitelbaum Swead et al, 2017]). Reviewing the literature, this population typically shows memory deficits (e.g., decreased working-memory spans), especially when learning auditory material ([Pisoni et al, 2011]; [Geers et al, 2013]). However, [Taitelbaum Swead et al (2017)] demonstrated that vocalization enables the CI users’ comparable memory performance to that of hearing individuals and concluded that saying words aloud may serve as a mnemonic for this population. Following these results, the present study focused on another unique group of hearing impaired (HI) individuals – severely to profoundly deaf sign language users (either MC or TC). Let us briefly describe this population and its special linguistic characteristics.

LANGUAGE MODALITIES OF HI NATIVE SIGNERS

During the past decades, the profile of the hearing-impaired population has been rapidly changing. Rates of cochlear implantation of severely and profoundly deaf children (at a very early age) are high and constantly rising ([Johnston, 2004]). The use of CIs (and other new hearing technologies) makes spoken language an option for most children (oral communication, which excludes the use of manual signs; [Fitzpatrick et al, 2013]).

Yet, CIs do not restore completely normal sensory information. In some cases, mainly children who were implanted at an older age, the acoustic cues provided by the implant are not sufficient for the development of an oral language. Typically, these children show low levels of speech perception, and their linguistic system is based on visual input rather than auditory input from the implant. For these individuals, TC (the simultaneous use of oral and manual language) may be used ([Meyer et al, 1998]). Usually in classrooms, spoken language is used (acquainting with sound patterns and lip-reading) and simultaneously manually coded language is signed ([Karchmer and Mitchell, 2003]).

Although this approach is very common in educational settings, it is relatively difficult to apply, and teachers of the deaf report that it is hard to synchronously sign and speak in an accurate manner ([Marmor and Petitto, 1979]). Users of TC tend to rely more heavily on the manual modality. As children, they commonly learn new vocabulary items and language rules using manually encoded language ([Svirsky et al, 2000]). In adulthood, some TC individuals tend to switch between languages, rather than produce simultaneous sign–speech expressions ([Emmorey et al, 2008]).

Not all profoundly deaf children are implanted, for example, children of signing deaf parents, who choose to refrain from the surgical procedure ([Lane and Grodin, 1997]). These deaf native signers are a small subsample of the deaf population, who rely on visual routes to learning and language access ([Bavelier et al, 2006]). Many of them use sign language as their principal means of communication. Typically, these individuals achieve their (sign) language development milestones at the same rate and time as normally hearing (NH) children ([Newport and Meier, 1985]; [Emmorey, 2002]; [Johnston and Schembri, 2007]). Some children with an early and profound hearing loss are likely to be lifelong users of sign language, using nonverbal (manual) communication (MC, [Johnston, 2004]).

In the present study, we investigated these subgroups of severely to profoundly deaf young adults, sign language users (note that all were high school students or graduates, learning in integrated classes, thus proficient Hebrew readers). Half of our participants were rehabilitated by hearing aids or CIs, TC users. The other half were deaf native signers (most of them did not use any hearing device), MC users (although they can use verbal communication, their preferred modality was the manual one). To improve learning outcomes in these populations, who depend mainly on the visual modality, it is important to identify useful memory techniques. Specifically, we tested which visual presentation mode (written or manual), and which production type (spoken or signed) are most effective in learning, leading to better memory performance. Answering these questions may have important clinical implications, e.g., suggesting a beneficial memory tool for this population. It may also shed light on some theoretical aspects related to HI signers’ memory, as will be discussed next.


#

MEMORY OF HI NATIVE SIGNERS

The literature on long-term memory of the deaf (the topic of the present study) is relatively scarce. The research on memory abilities of HI individuals focuses mainly on working memory ([Wilson and Emmorey, 1997]; [1998]). Working memory and long-term memory were initially treated as quite separate, but subsequent research has shown that these memory mechanisms are closely related ([Baddeley, 2000]), thus discussing working memory is relevant to the present study.

Back in the 70s of the last century, Baddeley and his associates proposed their model of working memory ([Baddeley, 1986]). One of its components is the “phonological loop,” which function as a short-term storage for verbal content (e.g., phonological information). Presumably, it consists of two elements: (a) the phonological store, which contains auditory memory traces, and (b) the articulatory loop, serves for articulatory rehearsal. Based on this model, auditory verbal information enters automatically into the phonological store. However, visually presented language has no direct access into the phonological store, but first it is converted into phonological codes (by silent articulation). The phonological loop plays a key role in vocabulary acquisition in early childhood ([Baddeley et al, 1998]), as well as later on, in second language learning.

Interestingly, a similar mechanism, parallel in structure to the phonological loop for speech, seems to operate in the working memory of deaf native users of sign language ([Wilson and Emmorey, 1997]). This mechanism is used for the temporary storage of sign language materials. Sign presentation has a direct access to this storage buffer, whereas picture presentation enters indirectly, by a manual articulatory rehearsal process. Based on this model, the structure of the phonological loop is independent of the modality that receives the verbal input – the auditory modality (speech signals) for hearing individuals or the manual modality (sign language) for native users of American Sign Language (ASL). [Wilson and Emmorey (1998)] concluded, “a phonological loop of approximately equivalent structure can arise in either of two very distinct sensorimotor modalities” (p. 584). Indeed, many working-memory effects (e.g., similarity, length, and suppression effects) are parallel for sign and speech ([Wilson, 2001]). Neurophysiological data (imaging studies) also support this notion. It was found that “inner signing” of sentences engages similar functional networks in the brain to those associated with the activation of the phonological loop (“inner speech”; [McGuire et al, 1997]).

The phonological loop is an important feature of long-term phonological learning. For example, it is associated with the development of vocabulary in children and with the speed of acquisition of L2 vocabulary items in adults ([Baddeley, 2000]). Because working memory is a gateway to long-term storage, working-memory abilities may affect long-term memory performance. The present study compared the efficacy of the sign-based and the speech-based working-memory mechanisms of profoundly deaf signers ([Wilson and Emmorey, 1997]). We evaluated a pair of methods to learn visually presented vocabulary items—reading them (using the written mode) or seeing them signed (using the manual mode). We also examined what type of active production yields better memory—vocalizing (saying the words aloud) or signing.


#

THE PRESENT STUDY

Twenty young adults, severely to profoundly deaf, sign language users (MC and TC) participated in this study. They learned three different lists of familiar words (randomly ordered) in a PE paradigm, visually presented in a written mode or in a manual (signed) mode. Half of the study words were not produced (silently read or seen) and the remaining half was actively produced (read aloud or signed).

This pair of variables, presentation mode (written or manual) and production type (vocalization or signing), formed three experimental conditions: (a) written-vocalize (WV): written words were visually presented, half of them learned by silent reading (no-production) and the remaining half by reading aloud (vocal production); (b) written-signed (WS): written words were presented, learned by silent reading or by signing (production); and (c) manual-signed (MS): manually presented words (in ISL) were presented (digital video displayed on a personal computer monitor) and learned by looking (no-production) or by signing (production). These experimental conditions are summarized in [Table 1].

Table 1

A Summary of the Experimental Conditions Used in the Present Study

Experimental Condition

Presentation Mode

Production Type

No-Production Type

Language/s

WV

Written

Vocalization

Silent reading

Hebrew

WS

Written

Signing

Silent reading

Hebrew, ISL

MS

Manually signed

Signing

Looking at a sign

ISL

After each study list, the participants performed a written free recall test. This procedure enabled us to determine whether recall performance differ based on the presentation mode (the manner in which the study content is communicated—written or manual) and to the production type (vocalization or signing).


#

METHOD

Participants

The study included 20 hearing impaired young adults (ten MCs and ten TCs). At school, all participants learned in integrated classes for deaf children (thus, all were skilled Hebrew readers) and presently reported using ISL as their main communication method. All participants confirmed normal or corrected visual ability and reported (by filling a written questionnaire) no history of developmental, cognitive, or neurological problems. Based on their preferred communication mode, the participants were divided into two subgroups, the signing deaf group (MC, n = 10, mean age = 25.4 yr) and the TC users (n = 10, mean age = 21.4 yr). See [Table 2] for detailed participants’ data.

Table 2

Individual Background Data of the Participants

Group

No.

Gender

Etiology

Degree of Hearing Loss

Type of Sensory Device

Signers (MC)

1

F

Genetic

Profound

None

2

F

Genetic

Profound

None

3

F

Genetic

Profound

None

4

F

Genetic

Profound

1 HA

5

F

Genetic

Profound

1 HA

6

F

Meningitis

Profound

1 HA

7

M

Meningitis

Profound

None

8

M

Genetic

Profound

1 HA

9

M

Genetic

Profound

None

10

M

CMV

Profound

None

TC users

11

F

Genetic

Severe

2 HA

12

F

Genetic

Severe

2 HA

13

F

Unknown

Profound

1 CI

14

F

Unknown

Severe

2 HA

15

F

Unknown

Profound

2 CI

16

F

Unknown

Severe

2 HA

17

F

CMV

Profound

1 CI

18

M

Genetic

Severe

1 HA

19

M

Unknown

Profound

2 CI

20

M

Meningitis

Severe

2 HA

Note: CMV = cytomegalovirus, F = female, HA = hearing aid, M = male.


This study received prior approval from the local ethics committee. Informed consent was obtained from all participants.


#

Apparatus and Stimuli

The pool of study words consisted of 120 Hebrew words, bisyllabic familiar nouns, three to five letters long (e.g., ball, closet, or boat; taken from: [Icht et al, 2014]; [Mama and Icht, 2016a]) for the experimental conditions that involved written words (WV and WS). For the written presentation, two random samples (of 30 words each) were selected for each participant. At study, each word was visually presented at the center of a 19-inch color monitor of a personal computer using PowerPoint software. The words were presented in black (28-point Arial) against a white background. On each trial, a small icon (sized 2 cm2) of an eye, a microphone, or a hand appeared about 5 cm above the study word. The icon indicated the appropriate production for that word: silent reading (no-production) was indicated by the eye, vocal production was cued by the microphone, and signing by the hand icon.

For the manual presentation, 30 additional words (taken from the same sources) were translated to ISL by a trained research assistant and were video recorded by a SONY Handycam. Each signed word was edited using the Winamp Media Player 5.7 software into a brief video clip (about 3 sec long). During the study phase, each of the manually signed study words was presented via personal computer under the control of PowerPoint program. On each trial, an icon of an eye or of a hand appeared at the right upper corner of the video window, indicating the appropriate mode of production (looking and signing, respectively).


#

Design

Study

Each participant performed the three learning conditions, thus studied three different word lists in a random order: (a) WV – studying written words, by silent reading or by reading aloud, (b) WS – studying written words, by silent reading or by signing, and (c) MS – studying manually presented words, by looking or by signing.

In each of these learning conditions, 30 study words were randomly selected for presentation (written or manual). Half (15 words) was not produced (silently read or seen), and the remaining half was produced (vocalized or signed).

Each experimental block begun with the presentation of a study word (written or manually signed) accompanied with an icon (eye, microphone, or hand) for 3 sec. A blank screen for 1 sec followed (thus, the interval between words was about 4 sec). This interstimulus interval enabled the participants to produce the study words, either by vocalization or signing, with no time pressure (all productions took no more than 2 sec). A ten-minute break was given between each block.


#

Filler Task

Four arithmetic problems (multiplication of four-digit numbers) printed on an A4 paper were prepared for filler tasks. After each study phase, participants were given 4 min to complete these problems.


#

Free Recall Test

A written free recall test followed each study phase and filler task. Participants were required to write down from memory as many study words as she or he could recall. The experimenter provided them with an empty sheet of paper and a pencil. There was no time limit for this task.


#

Procedure

The participants were tested individually in a quiet room. On arrival, each participant read and signed the informed consent form and was seated at a distance of about 60 cm from the center of a computer screen. The participant was asked to learn each word based on the icon and was informed that memory test would follow the presentation of the words.

For the written presentation, the study words were presented on the computer screen. For the manual presentation, the signs (brief video clips) were presented. A research assistant was present in the experiment room throughout the session, ascertaining the accurate production (or no-production) of the study words.

After the first study phase (30 words), the participants performed the short filler task, followed by the free recall test, performed by writing down as many study words from memory as possible. Later, a ten-minute break was given and the second learning condition begun (study, filler task, and test). Then, the third experimental condition followed in a similar fashion. All participants successfully performed the three experimental tasks, and no errors (in production type or other mistakes) occurred. The whole experimental session lasted no more than 50 min.


#
#
#

RESULTS

As a first step, we conducted a mixed-design analysis of variance (ANOVA), with learning condition (WV, WS, or MS) and production type (production or no-production) as within-subject variables and group (MC or TC) as a between-subject variable. This analysis revealed a significant main effect for production type, F (1,18) = 59.759, p < 0.001, ηp 2 = 0.769, and for learning condition, F (2,36) = 4.267, p = 0.022, ηp 2 = 0.192. A significant interaction between these variables (condition and production) was found, F (2,36) = 7.693, p = 0.002, ηp 2 = 0.299, suggesting different PE sizes in the different conditions. Main effect for group, as well as all interactions with group, were found to be insignificant. Because no differences were found between MCs and TCs, the following analyses will refer to all participants as a single group.

Shown in [Figure 1] are the proportions of words correctly recalled for the three learning conditions (WV, WS, and MS), for all participants. Visual inspection of [Figure 1] depicts the superiority of produced over non-produced words, across all conditions (WV - 0.28 versus 0.17; WS - 0.38 versus 0.14; MS - 0.30 versus 0.24, respectively). In other words, the participants remembered more produced than non-produced words in all learning conditions, a PE.

Zoom Image
Figure 1 Proportion of correctly recalled words for the two presentation modes (written and manual) and production types (vocalization and signing), averaged for all participants. The error bars are standard errors of their respective means.

Next, a two-way repeated-measures ANOVA with learning condition (×3) and production type (×2) as within-subject variables (for all participants) yielded main effects for condition, F (2,38) = 3.967, p = 0.027, ηp 2 = 0.173, and for production, F (1,19) = 62.951, p < 0.001, ηp 2 = 0.768, as well as an interaction, F (2,38) = 7.759, p = 0.001, ηp 2 = 0.290. This ANOVA was followed by planned comparisons to directly compare the recall rates in each learning condition. These analyses revealed no difference between the WS and the MS conditions, F (1,19) = 1.152, p = 0.297, ηp 2 = 0.057, and it also showed the superiority of these conditions (WS and MS) relative to the WV condition, F (1,19) = 5.237, p = 0.034, ηp 2 = 0.216 (three paired-sample t-tests revealed that all PEs were statistically significant, p < 0.05).

Another one-way repeated-measures ANOVA for the PE size (i.e., the difference in memory performance between the produced and non-produced words) in each learning condition revealed a significant effect, F (2,38) = 7.759, p = 0.001, ηp 2 = 0.290. Planned comparisons to compare the magnitude of the PE in each learning condition yielded no difference between the WV and the MS conditions, F (1,19) = 1.233, p = 0.281, ηp 2 = 0.061, and a larger PE in the WS condition relative to both WV and MS conditions, F (1,19) = 14.924, p = 0.001, ηp 2 = 0.440.

To compare the efficiency of the different types of active production conditions (reading aloud, signing a written word, and signing a manual word), a one-way repeated-measures ANOVA was performed. This analysis revealed a significant difference, F (2,38) = 4.944, p = 0.0125, ηp 2 = 0.206. Planned comparisons revealed comparable memory performance in the WV (reading aloud) and MS (signing a manual word) conditions, F (1,19) = 0.238, p = 0.631, ηp 2 = 0.012, and significantly better memory in the WS condition (signing a written word) relative to the two other conditions, F (1,19) = 11.495, p = 0.003, ηp 2 = 0.377.

Finally, another one-way repeated-measures ANOVA between the no-production conditions (reading silently in the WV and WS conditions and looking at a manually signed word in the MS condition) also revealed a significant effect, F (2,38), p = 0.001, ηp 2 = 0.329. Planned comparisons revealed no difference between WV and WS (reading silently), F (1,19) = 2.209, p = 0.154, ηp 2 = 0.104. The MS no-production condition (looking at a signed word) was found to be superior to the other conditions, F (1,19) = 14.529, p = 0.001, ηp 2 = 0.433.


#

DISCUSSION

Understanding memory functioning among HI individuals has theoretical importance, identifying the relations between (spoken and signed) language and cognition. From the clinical perspective, illuminating memory abilities is essential in improving academic outcomes for HI learners. The present study focused on a special group of HI individuals, deaf native signers. Cognitive functions of native-signing deaf individuals who grew up in deaf families may be more visually oriented than of other deaf individuals ([Emmorey, 2002]; [Corina et al, 2013]). We evaluated their verbal long-term memory performance when studying written versus manually signed words (both visually presented), produced by vocalizing versus signing.

The results showed a production benefit across all learning conditions (WV, WS, and MS), with enhanced recall of produced over non-produced words. The size of the PE (the difference between produced and non-produced words) was largest in the WS condition. In other words, memory performance was superior when learning written words by signing. In addition, overall memory (for produced and non-produced word) was weakest in the WV condition. We will now discuss these findings and their implications.

The PE in HI Sign Language Users

The first finding of the present study was a PE (production benefit) along all learning conditions. Across the modes of presentation (written and manual) and the types of production (vocalization and signed), the act of producing study items improved their memory relative to studying by no-production. This was an expected result because the PE had been previously documented for several populations (e.g., preschool children, [Icht and Mama, 2015]; younger and older adults, [Lin and MacLeod, 2012]; dysarthric adults, [Icht et al, 2016]), including HI CI users ([Taitelbaum Swead et al, 2017]).

These results stress the role of production in improving HI signer’s verbal memory. Passive learning (silently reading or looking at signs) was found to be less effective than actively producing the to-be-remembered material (either by reading aloud or by signing). (For a discussion on passive versus active language learning, see [Shafer and Garrido-Nag, 2007].) The number of distinct encoding processes that involves in learning can explain these results (encoding distinctiveness account, [Macleod et al, 2010]). Note that this account emphasizes “unique” processes at encoding. Repeating an encoding process (at presentation and again at production) is less beneficial than adding a unique encoding process. Take, for example, a hearing individual studying visually presented word by writing. The first encoding process is visual—reading the word. The additional processes are motor (the act of writing) and reading again (her own handwriting). This rereading is of limited benefit to memory because it is not unique. However, when the same individual (NH) learns a visually presented word by vocal production, all three encoding processes are distinctive—visual (reading), motor (articulation), and auditory (hearing her own voice). Hence, this latter condition is more effective in memorizing ([Forrin et al, 2012]).

The present results fit well with the encoding distinctiveness rationale. Importantly, the findings support that for the group of HI sign language users, reading a written word and seeing a manual sign are different and distinct encoding processes (although both are visual). Indeed, several studies confirmed the presence of a sign-based loop in the working-memory mechanism for this population. This loop is parallel in function to the speech-based phonological loop of NH individuals ([Wilson and Emmorey, 1997]; [1998]). Additional support for the difference between reading and seeing a sign is provided by [Emmorey et al (2009)] who suggested that monitoring for sign language may be kinesthetic (although sign perception is visual).

Counting the number of unique encoding processes in each of our learning conditions reveals an interesting (yet not surprising) pattern. [Table 3] summarizes the types and number of distinct processes in each condition (note that hearing one’s own voice at vocalizing was not counted because the participants are HI who mostly rely on visual language routes). Inspecting [Table 3] clearly reveals that the condition that involves the larger number of processes results in improved memory performance. These results suggest that it is important to encourage HI signers to actively produce study material to improve memory, regardless the presentation mode (written or manual) and the production type (vocalize or sign).

Table 3

The Number of Distinct Encoding Processes Involved in Each of the Learning Conditions of the Present Study

Experimental Condition

Learning Condition

Type of Distinct Encoding Processes

Number of Distinct Encoding Processes

Reading

Articulating

Manual

Seeing a Sign

WV

Silent reading

+

1

Vocalization

+

+

2

WS

Silent reading

+

1

Signing a written word

+

+

+

3

MS

Looking at a sign

+

1

Signing a signed word

+

+

2


#

Signing Written Words as a Recommended Learning Condition for HI Signers

The present results demonstrated that the most efficient learning condition was using the written presentation mode and allowing production by signing. This condition yielded the highest recall rates (about 38%), for both groups of HI signers (MC and TC). This finding is in-line with the encoding distinctiveness account. Indeed, with the WS learning condition, signing the written words involved the highest number of distinct encoding processes (three) relative to all other types of learning conditions (see [Table 3]). Related pattern of results, an advantage for bimodal learning (speech and sign) over either speech or sign alone, had been previously documented for deaf signers ([Hamilton and Holzman, 1989]).

As was suggested by [Wilson and Emmorey (1997]; [1998]), for deaf users of sign language, a special working-memory system exists. It consists of a phonological storage buffer and an articulatory rehearsal mechanism, which are equivalent in structure for the phonological loop for speech (i.e., a sign-based storage buffer and an inner signing rehearsal mechanism). Possibly, using the written presentation mode and requiring production by signing activates two different sensorimotor paths in working memory, thus increasing performance. Indeed, [Morford et al (2011)] who evaluated deaf bilinguals for whom ASL was the first language and English was the second language claimed that parallel activation of both representations can simultaneously occur.

An alternative explanation for the advantage of signing written words may be found in the context of the “translation effect” ([Conway and Gathercole, 1990]). According to this theory, when processing activities in different modalities (“translations”) occur at the study, they create unique memory traces and, as a result, memory improves. In other words, multimodal encoding, which involves processes in different domains (e.g., studying aurally presented words by writing) is more deep than a single modal encoding, which involves processing activities only in one modality (e.g., studying written words by writing). In [Conway and Gathercole’s (1990)] words, “Translations between specialized processing domains that occur at encoding, lead to formation of distinctive memories, and, hence, to better retention” (p. 513).

The translation effect stresses switching between modalities at encoding. In the present study, two conditions involved such modalities’ translation, the WV condition (vocalizing or written words) and the WS condition (signing written words). Interestingly, the latter also involved switching between languages because the stimuli were written Hebrew words and the productions were ISL manual signs. Possibly, the lexical processing in two versus one language (i.e., an actual translation versus reproduction of words) may explain the superiority of the WS condition (for a comparable pattern of findings in French–English bilinguals, see [Paivio and Lambert, 1981]).

Related findings were reported by [Morford et al (2011)] testing deaf bilinguals (ASL and English). These deaf readers (comparable with the present sample) were found to activate the ASL translations of written words in English even when the task did not explicitly require the use of ASL (e.g., under conditions in which the translation was neither present perceptually nor required to perform the task). Such activation spread between words in two languages differing in modality may hint that deaf bilinguals are able to activate English and ASL phonological representations simultaneously with less competition, possibly gaining comprehension benefits and cognitive advantages in executive function. Further studies would address the possibility (suggested by an anonymous reviewer) that the phonological activation depends on the language of production, either speech or sign, and the higher processing levels (e.g., semantics, syntax) are indeed activated simultaneously.


#

Caveats and Future Directions

The present study clearly shows the superiority of signing written words as a means to improve long-term memory for HI signers. Yet, it has some limitations. First, memory was evaluated using a written free recall test (written tests were used in most of the PE experiments). In other words, the mode of presentation (written and signed) and the production type (vocalization and signing) varied across the study conditions, but the response mode was fixed—writing. In a recent study, [Mama and Icht (2016b)] found that the retrieval mode (written or aloud) affects the PE. Specifically, the authors showed that with visual study (written words), vocalizing was superior to written production, in both retrieval modes—written and aloud. However, the difference between the productions was significantly smaller in the aloud retrieval mode relative to the written retrieval mode (a retrieval-cost). Because the present study also used visual presentation (written or signed words), we chose the written retrieval mode, to document a significant difference between the production types (a PE), as was found. Future studies may test the effect of other retrieval modes (vocal or signed) on the pattern of results. Specifically, based on the retrieval-cost model of the PE ([Mama and Icht, 2016b]) in the WS condition (a written presentation mode and production by signing versus no-production by reading) recall by signing would improve performance (for the non-produced words) relative to recall by writing.

Another important point may be that the MC subgroup, who typically prefer the MC mode may find vocalizing words to be too difficult and too demanding. The vocal production may be more resource consuming, resulting in a reduced efficiency of learning and lowered memory performance (usually, this is a modality the participants do not use). This possibility seems unlikely because we did not find differences in memory performance between our two participants’ groups (MC and TC). Moreover, in many cases, difficult encoding procedures were found to enhance learning outcomes. According to the “desirable difficulties account” ([Bjork, 1994]; [1999]), difficult study methods yield more durable and flexible learning and improve long-term memory. For example, varying the conditions of learning (rather than keeping them constant and predictable) or taking an initial test on the study material (rather than relearning it). In the context of the PE, [Icht et al (2016)] found a better memory for vocalized words (relative to silently read words) for a group of dysarthric adults, characterized by speech difficulties (following a neurological damage). Therefore, in case vocalization is relatively difficult, it may further enhance memory. This possibility should be directly tested in future studies.

Abbreviations

ANOVA: analysis of variance
CI: cochlear implant
ISL: Israeli sign language
MC: manual communication
MS: manual-signed
PE: production effect
TC: total communication
WS: written-signed
WV: written-vocalize


#
#
#

No conflict of interest has been declared by the author(s).

Acknowledgments

The authors would like to thank Tzlil Shimoni and Vivian Shachar for collecting data for this study.

  • REFERENCES

  • Baddeley A. 1986. Working Memory. Oxford, United Kingdom: Oxford University Press;
  • Baddeley A. 2000; The episodic buffer: a new component of working memory?. Trends Cogn Sci 4 (11) 417-423
  • Baddeley A, Gathercole S, Papagno C. 1998; The phonological loop as a language learning device. Psychol Rev 105 (01) 158-173
  • Bavelier D, Dye MW, Hauser PC. 2006; Do deaf individuals see better?. Trends Cogn Sci 10 (11) 512-518
  • Bjork RA. 1994. Memory and metamemory considerations in the training of human beings. In: Metcalfe J, Shimamura A. Metacognition: Knowing about Knowing. Cambridge, MA: MIT Press; 185-205
  • Bjork RA. 1999. Assessing our own competence: heuristics and illusions. In: Gopher D, Koriat A. Attention and Performance XVII: Cognitive Regulation of Performance: Interaction of Theory and Application. Cambridge, MA: MIT Press; 435-459
  • Conway MA, Gathercole SE. 1990; Writing and long-term memory: evidence for a “translation” hypothesis. Q J Exp Psychol A 42: 513-527
  • Corina DP, Lawyer LA, Hauser P, Hirshorn E. 2013; Lexical processing in deaf readers: an FMRI investigation of reading proficiency. PLoS One 8 (01) e54696
  • Emmorey K. 2002. Language, Cognition, and the Brain: Insights from Sign Language Research. Mahwah, NJ: Lawrence Erlbaum Associates;
  • Emmorey K, Borinstein HB, Thompson R, Gollan TH. 2008; Bimodal bilingualism. Biling (Camb Engl) 11 (01) 43-61
  • Emmorey K, Bosworth R, Kraljic T. 2009; Visual feedback and self-monitoring of sign language. J Mem Lang 61 (03) 398-411
  • Fitzpatrick EM, Stevens A, Garritty C, Moher D. 2013; The effects of sign language on spoken language acquisition in children with hearing loss: a systematic review protocol. Syst Rev 2: 108
  • Forrin ND, Jonker TR, MacLeod CM. 2014; Production improves memory equivalently following elaborative vs non-elaborative processing. Memory 22 (05) 470-480
  • Forrin ND, Macleod CM, Ozubko JD. 2012; Widening the boundaries of the production effect. Mem Cognit 40 (07) 1046-1055
  • Geers AE, Pisoni DB, Brenner C. 2013; Complex working memory span in cochlear implanted and normal hearing teenagers. Otol Neurotol 34 (03) 396-401
  • Hamilton H, Holzman TG. 1989; Linguistic encoding in short-term memory as a function of stimulus type. Mem Cognit 17 (05) 541-550
  • Icht M, Bergerzon-Biton O, Mama Y. 2016; The production effect in adults with dysarthria: improving long-term verbal memory by vocal production. Neuropsychol Rehabil 1-13
  • Icht M, Mama Y. 2015; The production effect in memory: a prominent mnemonic in children. J Child Lang 42 (05) 1102-1124
  • Icht M, Mama Y, Algom D. 2014; The production effect in memory: multiple species of distinctiveness. Front Psychol 5: 886
  • Johnston T. 2004; W(h)ither the deaf community? Population, genetics, and the future of Australian sign language. Am Ann Deaf 148 (05) 358-375
  • Johnston T, Schembri A. 2007. Australian Sign Language (Auslan): An Introduction to Sign Language Linguistics. Cambridge, United Kingdom: Cambridge University Press;
  • Karchmer MA, Mitchell RE. 2003. Demographic and achievement characteristics of deaf and hard of hearing students. In: Marschark M, Spencer PE. Oxford Handbook of Deaf Studies, Language, and Education. New York, NY: Oxford University Press; 21-37
  • Lane H, Grodin M. 1997; Ethical issues in cochlear implant surgery: an exploration into disease, disability, and the best interests of the child. Kennedy Inst Ethics J 7 (03) 231-251
  • Lin OY, MacLeod CM. 2012; Aging and the production effect: a test of the distinctiveness account. Can J Exp Psychol 66 (03) 212-216
  • MacLeod CM. 2011; I said, you said: the production effect gets personal. Psychon Bull Rev 18 (06) 1197-1202
  • MacLeod CM, Gopie N, Hourihan KL, Neary KR, Ozubko JD. 2010; The production effect: delineation of a phenomenon. J Exp Psychol Learn Mem Cogn 36 (03) 671-685
  • Mama Y, Icht M. 2016; a Auditioning the distinctiveness account: expanding the production effect to the auditory modality reveals the superiority of writing over vocalising. Memory 24 (01) 98-113
  • Mama Y, Icht M. 2016; b Influence of retrieval mode on effects of production: evidence for costs in free recall. Can J Exp Psychol 70 (02) 177-185
  • Marmor GS, Petitto L. 1979; Simultaneous communication in the classroom: how well is English grammar represented?. Sign Lang Stud 23 (01) 99-136
  • McGuire PK, Robertson D, Thacker A, David AS, Kitson N, Frackowiak RS, Frith CD. 1997; Neural correlates of thinking in sign language. Neuroreport 8 (03) 695-698
  • Meyer TA, Svirsky MA, Kirk KI, Miyamoto RT. 1998; Improvements in speech perception by children with profound prelingual hearing loss: effects of device, communication mode, and chronological age. J Speech Lang Hear Res 41 (04) 846-858
  • Morford JP, Wilkinson E, Villwock A, Piñar P, Kroll JF. 2011; When deaf signers read English: do written words activate their sign translations?. Cognition 118 (02) 286-292
  • Newport E, Meier R. 1985. The acquisition of American sign language. In: Slobin D. The Crosslinguistic Study of Language Acquisition. Hillsdale, NJ: Erlbaum; 881-938
  • Ozubko JD, Hourihan KL, MacLeod CM. 2012; Production benefits learning: the production effect endures and improves memory for text. Memory 20 (07) 717-727
  • Ozubko JD, Macleod CM. 2010; The production effect in memory: evidence that distinctiveness underlies the benefit. J Exp Psychol Learn Mem Cogn 36 (06) 1543-1547
  • Paivio A, Lambert W. 1981; Dual coding and bilingual memory. J Verbal Learn Verbal Behav 20 (05) 532-539
  • Pisoni DB, Kronenberger WG, Roman AS, Geers AE. 2011; Measures of digit span and verbal rehearsal speed in deaf children after more than 10 years of cochlear implantation. Ear Hear 32 (1, Suppl) 60S-74S
  • Quinlan CK, Taylor TL. 2013; Enhancing the production effect in memory. Memory 21 (08) 904-915
  • Shafer VL, Garrido-Nag K. 2007. The neurodevelopmental bases of language, chapter 2. In: Hoff E, Shatz M. Blackwell Handbook of Language Development. Malden, MA: Blackwell Publishing; 21-45 ‏
  • Svirsky MA, Robbins AM, Kirk KI, Pisoni DB, Miyamoto RT. 2000; Language development in profoundly deaf children with cochlear implants. Psychol Sci 11 (02) 153-158
  • Taitelbaum-Swead R, Icht M, Mama Y. 2017; The effect of learning modality and auditory feedback on word memory: cochlear implanted vs normal hearing adults. J Am Acad Audiol 28 (03) 222-231
  • Wilson M. 2001; The case for sensorimotor coding in working memory. Psychon Bull Rev 8 (01) 44-57
  • Wilson M, Emmorey K. 1997; A visuospatial “phonological loop” in working memory: evidence from American sign language. Mem Cognit 25 (03) 313-320
  • Wilson M, Emmorey K. 1998; A “word length effect” for sign language: further evidence for the role of language in structuring working memory. Mem Cognit 26 (03) 584-590

Corresponding author

Riki Taitelbaum Swead
Department of Communication Disorders
Ariel University, Ariel
Israel   

  • REFERENCES

  • Baddeley A. 1986. Working Memory. Oxford, United Kingdom: Oxford University Press;
  • Baddeley A. 2000; The episodic buffer: a new component of working memory?. Trends Cogn Sci 4 (11) 417-423
  • Baddeley A, Gathercole S, Papagno C. 1998; The phonological loop as a language learning device. Psychol Rev 105 (01) 158-173
  • Bavelier D, Dye MW, Hauser PC. 2006; Do deaf individuals see better?. Trends Cogn Sci 10 (11) 512-518
  • Bjork RA. 1994. Memory and metamemory considerations in the training of human beings. In: Metcalfe J, Shimamura A. Metacognition: Knowing about Knowing. Cambridge, MA: MIT Press; 185-205
  • Bjork RA. 1999. Assessing our own competence: heuristics and illusions. In: Gopher D, Koriat A. Attention and Performance XVII: Cognitive Regulation of Performance: Interaction of Theory and Application. Cambridge, MA: MIT Press; 435-459
  • Conway MA, Gathercole SE. 1990; Writing and long-term memory: evidence for a “translation” hypothesis. Q J Exp Psychol A 42: 513-527
  • Corina DP, Lawyer LA, Hauser P, Hirshorn E. 2013; Lexical processing in deaf readers: an FMRI investigation of reading proficiency. PLoS One 8 (01) e54696
  • Emmorey K. 2002. Language, Cognition, and the Brain: Insights from Sign Language Research. Mahwah, NJ: Lawrence Erlbaum Associates;
  • Emmorey K, Borinstein HB, Thompson R, Gollan TH. 2008; Bimodal bilingualism. Biling (Camb Engl) 11 (01) 43-61
  • Emmorey K, Bosworth R, Kraljic T. 2009; Visual feedback and self-monitoring of sign language. J Mem Lang 61 (03) 398-411
  • Fitzpatrick EM, Stevens A, Garritty C, Moher D. 2013; The effects of sign language on spoken language acquisition in children with hearing loss: a systematic review protocol. Syst Rev 2: 108
  • Forrin ND, Jonker TR, MacLeod CM. 2014; Production improves memory equivalently following elaborative vs non-elaborative processing. Memory 22 (05) 470-480
  • Forrin ND, Macleod CM, Ozubko JD. 2012; Widening the boundaries of the production effect. Mem Cognit 40 (07) 1046-1055
  • Geers AE, Pisoni DB, Brenner C. 2013; Complex working memory span in cochlear implanted and normal hearing teenagers. Otol Neurotol 34 (03) 396-401
  • Hamilton H, Holzman TG. 1989; Linguistic encoding in short-term memory as a function of stimulus type. Mem Cognit 17 (05) 541-550
  • Icht M, Bergerzon-Biton O, Mama Y. 2016; The production effect in adults with dysarthria: improving long-term verbal memory by vocal production. Neuropsychol Rehabil 1-13
  • Icht M, Mama Y. 2015; The production effect in memory: a prominent mnemonic in children. J Child Lang 42 (05) 1102-1124
  • Icht M, Mama Y, Algom D. 2014; The production effect in memory: multiple species of distinctiveness. Front Psychol 5: 886
  • Johnston T. 2004; W(h)ither the deaf community? Population, genetics, and the future of Australian sign language. Am Ann Deaf 148 (05) 358-375
  • Johnston T, Schembri A. 2007. Australian Sign Language (Auslan): An Introduction to Sign Language Linguistics. Cambridge, United Kingdom: Cambridge University Press;
  • Karchmer MA, Mitchell RE. 2003. Demographic and achievement characteristics of deaf and hard of hearing students. In: Marschark M, Spencer PE. Oxford Handbook of Deaf Studies, Language, and Education. New York, NY: Oxford University Press; 21-37
  • Lane H, Grodin M. 1997; Ethical issues in cochlear implant surgery: an exploration into disease, disability, and the best interests of the child. Kennedy Inst Ethics J 7 (03) 231-251
  • Lin OY, MacLeod CM. 2012; Aging and the production effect: a test of the distinctiveness account. Can J Exp Psychol 66 (03) 212-216
  • MacLeod CM. 2011; I said, you said: the production effect gets personal. Psychon Bull Rev 18 (06) 1197-1202
  • MacLeod CM, Gopie N, Hourihan KL, Neary KR, Ozubko JD. 2010; The production effect: delineation of a phenomenon. J Exp Psychol Learn Mem Cogn 36 (03) 671-685
  • Mama Y, Icht M. 2016; a Auditioning the distinctiveness account: expanding the production effect to the auditory modality reveals the superiority of writing over vocalising. Memory 24 (01) 98-113
  • Mama Y, Icht M. 2016; b Influence of retrieval mode on effects of production: evidence for costs in free recall. Can J Exp Psychol 70 (02) 177-185
  • Marmor GS, Petitto L. 1979; Simultaneous communication in the classroom: how well is English grammar represented?. Sign Lang Stud 23 (01) 99-136
  • McGuire PK, Robertson D, Thacker A, David AS, Kitson N, Frackowiak RS, Frith CD. 1997; Neural correlates of thinking in sign language. Neuroreport 8 (03) 695-698
  • Meyer TA, Svirsky MA, Kirk KI, Miyamoto RT. 1998; Improvements in speech perception by children with profound prelingual hearing loss: effects of device, communication mode, and chronological age. J Speech Lang Hear Res 41 (04) 846-858
  • Morford JP, Wilkinson E, Villwock A, Piñar P, Kroll JF. 2011; When deaf signers read English: do written words activate their sign translations?. Cognition 118 (02) 286-292
  • Newport E, Meier R. 1985. The acquisition of American sign language. In: Slobin D. The Crosslinguistic Study of Language Acquisition. Hillsdale, NJ: Erlbaum; 881-938
  • Ozubko JD, Hourihan KL, MacLeod CM. 2012; Production benefits learning: the production effect endures and improves memory for text. Memory 20 (07) 717-727
  • Ozubko JD, Macleod CM. 2010; The production effect in memory: evidence that distinctiveness underlies the benefit. J Exp Psychol Learn Mem Cogn 36 (06) 1543-1547
  • Paivio A, Lambert W. 1981; Dual coding and bilingual memory. J Verbal Learn Verbal Behav 20 (05) 532-539
  • Pisoni DB, Kronenberger WG, Roman AS, Geers AE. 2011; Measures of digit span and verbal rehearsal speed in deaf children after more than 10 years of cochlear implantation. Ear Hear 32 (1, Suppl) 60S-74S
  • Quinlan CK, Taylor TL. 2013; Enhancing the production effect in memory. Memory 21 (08) 904-915
  • Shafer VL, Garrido-Nag K. 2007. The neurodevelopmental bases of language, chapter 2. In: Hoff E, Shatz M. Blackwell Handbook of Language Development. Malden, MA: Blackwell Publishing; 21-45 ‏
  • Svirsky MA, Robbins AM, Kirk KI, Pisoni DB, Miyamoto RT. 2000; Language development in profoundly deaf children with cochlear implants. Psychol Sci 11 (02) 153-158
  • Taitelbaum-Swead R, Icht M, Mama Y. 2017; The effect of learning modality and auditory feedback on word memory: cochlear implanted vs normal hearing adults. J Am Acad Audiol 28 (03) 222-231
  • Wilson M. 2001; The case for sensorimotor coding in working memory. Psychon Bull Rev 8 (01) 44-57
  • Wilson M, Emmorey K. 1997; A visuospatial “phonological loop” in working memory: evidence from American sign language. Mem Cognit 25 (03) 313-320
  • Wilson M, Emmorey K. 1998; A “word length effect” for sign language: further evidence for the role of language in structuring working memory. Mem Cognit 26 (03) 584-590

Zoom Image
Figure 1 Proportion of correctly recalled words for the two presentation modes (written and manual) and production types (vocalization and signing), averaged for all participants. The error bars are standard errors of their respective means.