J Am Acad Audiol 2019; 30(01): 031-040
DOI: 10.3766/jaaa.17070
Articles
Thieme Medical Publishers 333 Seventh Avenue, New York, NY 10001, USA.

Quality and Readability of English-Language Internet Information for Tinnitus

Vinaya Manchaiah
*   Department of Speech and Hearing Sciences, Lamar University, Beaumont, TX
†   Department of Behavioural Sciences and Learning, The Swedish Institute for Disability Research, Linköping University, Linköping, Sweden
‡   Audiology India, Mysore, Karnataka, India
§   Department of Speech and Hearing, School of Allied Health Sciences, Manipal University, Manipal, Karnataka, India
,
Ashley L. Dockens
*   Department of Speech and Hearing Sciences, Lamar University, Beaumont, TX
,
Ashley Flagge
*   Department of Speech and Hearing Sciences, Lamar University, Beaumont, TX
,
Monica Bellon-Harn
*   Department of Speech and Hearing Sciences, Lamar University, Beaumont, TX
,
Jamie Hartwell Azios
*   Department of Speech and Hearing Sciences, Lamar University, Beaumont, TX
,
Rebecca J. Kelly-Campbell
¶   Department of Communication Disorders, University of Canterbury, Christchurch, New Zealand,
,
Gerhard Andersson
†   Department of Behavioural Sciences and Learning, The Swedish Institute for Disability Research, Linköping University, Linköping, Sweden
‖   Department of Clinical Neuroscience, Division of Psychiatry, Karolinska Institute, Stockholm, Sweden
› Author Affiliations
Further Information

Corresponding author

Vinaya Manchaiah
Department of Speech and Hearing Sciences, Lamar University
Beaumont, TX 77710

Publication History

Publication Date:
26 May 2020 (online)

 

Abstract

Background:

Because of the wealth of information available on the internet and increasing numbers of individuals relying on websites as a primary source of information for health-related questions, it is important that the readability of their content is within the comprehension level of most readers.

Objective:

The study evaluated the quality and readability of English-language Internet information for tinnitus.

Research Design:

Analysis of Internet websites on tinnitus.

Study Sample:

A total of 134 websites with tinnitus information.

Data Collection and Analysis:

Three key words (i.e., tinnitus, ringing in the ear, and buzzing in the ear) were entered in five country-specific versions of the most commonly used internet search engine in August 2016. For each of the 15 searches, the first 20 relevant websites were examined. After removing duplicates, a total of 134 websites were assessed. Their origin (commercial, nonprofit organization, government, personal, or university), quality (Health On the Net [HON] certification and DISCERN scores), and readability (Flesch Reading Ease score, Flesch-Kincaid Reading Grade Level Formula, and Simple Measure of Gobbledygook) were assessed.

Results:

Most websites were of commercial (49.3%) or nonprofit organization (38.8%) origin. Their quality and readability was highly variable. Only 13.5% of websites had HON certification. χ2 analysis showed that there was significant association between website origin and HON certification [χ2(4) = 132.9, p < 0.0001]. The mean DISCERN scores were 2.39. No association between DISCERN scores and website origin was found. Readability measures showed that on average, only people with at least 10–12 yr of education could read and understand the internet information for tinnitus in websites. Almost all the websites exceeded the most stringent reading level recommended for health information.

Conclusions:

The results highlight great variability in the quality and readability of health information, specifically for tinnitus in the internet. These findings underscores the need for stakeholders (e.g., web-developers, clinicians) to be aware of this and to develop more user-friendly health information on websites to make it more accessible for people with low literacy.


#

INTRODUCTION

Tinnitus has been defined as the perception of sound in the head or ears without an external stimulus ([Tunkel et al, 2014]). It is estimated that tinnitus affects between 10% and 15% of adults globally ([Baguley et al, 2013]; [Tunkel et al, 2014]). An estimated 10% of those affected report their tinnitus to be distressing ([Sindhusake et al, 2003]). There is no cure for tinnitus. However, clinicians provide informational counseling to help people cope with their perception of and reaction to tinnitus ([Folmer et al, 2004]; [Bauer and Brozoski, 2011]).

The current evidence reveals a lack of a viable pharmacological treatment for tinnitus suffers ([Baguley et al, 2013]). [Noble (2012)] points out that either a physical approach or a behavioral/psychological approach may be beneficial treatments for tinnitus. A physical approach aims to moderate or remove the tinnitus signal. A behavioral and/or psychological approach uses effective acoustic amplification to mask the tinnitus, potentially with the addition of biofeedback. Treatments using patient education, either through explaining how tinnitus arises or by helping individuals habituate to their tinnitus, has a stronger base of evidence to recommend them, in particular cognitive behavior therapy ([Martinez-Devesa et al, 2010]; [Hesser et al, 2011]). It follows that clinicians need to provide information in an easily understandable and usable way for their patients to help them achieve the best possible health outcomes ([Loumidis et al, 1991]). This information can come in the form of printed material, orally presented information, or online material.

As internet access, knowledge of use, and ease of use have increased in recent years, so has the availability of health-related information to the average health client. [Fox (2014)] reported that 79% of American internet users seek health-related information on the internet. According to [Couper et al (2010)], the internet is second only to direct clinician advice in terms of influence when making health-related decisions. With the ever-improving technology of smartphones and other “smart” devices allowing for nearly instantaneous answers to health-related questions, it is likely that the reliance on the internet as a source of health information will continue to increase. However, for clients to obtain and comprehend health-related information on the internet, the information must be presented at a level that can be easily understood. The National Adult Literacy Survey ([Kirsch, 1993]) reported that the average reading grade level (RGL) for American adults is about a seventh RGL. The RGL can be defined as the number of years of US education required to understand a text ([Ley and Florio, 1996]). Researchers in the field of health literacy have recommended that information accessible to the public should be written at about a fifth or sixth RGL ([Doak et al, 1996]; [Weiss and Coyne, 1997]).

Previous research assessing readability in the field of audiology and otolaryngology has examined the readability of patient-assessment questionnaires ([Kelly-Campbell et al, 2012]) and patient-reported outcome questionnaires ([Atcherson et al, 2011]), as well as otolaryngology websites ([Eloy et al, 2012]). Further research in the field of audiology has examined the readability of materials on the American Speech-Language-Hearing Association website ([Atcherson et al, 2014]), as well as the readability of websites dedicated to information on hearing loss ([Laplante-Lévesque et al, 2012]). In all the above-mentioned studies, the readability of the information examined was found to be well above the recommended fifth to sixth RGL, with readability in some cases even exceeding 17 yr of education ([Eloy et al, 2012]).

Because of the wealth of information available on the internet and increasing numbers of individuals relying on websites as a primary source of information for health-related questions, it is important that the readability of their content is within the comprehension level of most readers. In a systematic review of the literature, [Laplante-Lévesque and Thorén (2015)] reported poor readability of Internet information on hearing and hearing loss. [Joseph et al (2016)] analyzed patient education materials from six of the leading behind-the-ear hearing aid manufacturers and from popular hearing aid information websites. They found that the average RGL for patient education materials and professional organization websites was at a tenth-RGL, well above the current recommendations for readability. As such, they recommended a simplification of materials and websites that are directed toward health clients.

In addition to readability element, the quality of the website can be an assessed. Having the Health On the Net (HON) certification is an indication that the web developers adhere to good practice guidelines and principles recommended to having good quality information and formatting in the website ([Boyer et al, 1998]). While HON certification is gaining popularity, few web developers are aware of it and for this reason, only a small percentage of websites may have this certification. In addition, researchers and clinicians can use the DISCERN instrument for rating websites on various quality indicators (e.g., authorship; attribution; disclosure; how up-to-date the information is) ([Charnock et al, 1999]).

Moreover, the website origin may have some bearing toward the quality, accuracy, and readability of information provided ([Laplante-Lévesque et al, 2012]; [Hsu, 2017]). For example, websites from commercial, university, and government origins may have a higher quality, when compared with personal webpages or blogs. This could be due to the fact that commercial, university, and government organizations may have more resources to spend on websites when compared with websites of nonprofit or personal origin.

Accessibility of Tinnitus-Related Information

While hearing loss may be a more familiar topic for general people, according to Google Trends, “tinnitus” is more commonly searched topic on the internet when compared with “hearing loss.” Tinnitus is a relatively prevalent and somewhat complex complaint that often presents in patients alongside stress, anxiety, insomnia, or depression ([Fackrell et al, 2012]). [Atcherson et al (2011)] analyzed 15 patient-report outcome questionnaires on tinnitus and found that most of the questionnaires analyzed exceeded the recommended fifth to sixth RGL. General practitioners (GPs) in England have reported referring patients with tinnitus to favored websites as a preferred medium of treatment ([El-Shunnar et al, 2011]). [El Shunnar et al (2011)] reported an analysis of the quality of information contained in ten online sources commonly used by GPs in England using the DISCERN instrument ([Charnock et al, 1999]). Findings revealed that not a single website contained comprehensive information for GPs related to tinnitus assessment or management. However, even with a relative abundance of tinnitus-related information on the internet ([El-Shunnar et al, 2011]), studies examining the readability and quality of tinnitus-related websites are sparse.

The purpose of the current study was to assess the quality and readability of English-language internet information available about tinnitus.


#
#

METHOD

There was no ethical approval required for this study. The study design was inspired by previously published studies on quality and readability of internet information on speech and hearing disorders ([Laplante-Lévesque et al, 2012]; [Atcherson et al, 2014]; [Joseph et al, 2016]).

Search and Selection Strategy

A group of professionals (n = 6) and people experiencing tinnitus (n = 8) were asked to provide key words that they or their patients would use while searching for “tinnitus” information online. The primary key words were identified from the panel and entered in Google Trends (www.google.com/trends). Google Trends compiles any given keyword’s relative frequency as a search term in Google over time. From this trending analysis, three key words based on their frequency were identified. Those key words were: tinnitus, ringing in the ear, and buzzing in the ear.

These three key words were entered into five country-coded Top-Level Domains (ccTLDs) for which there was a Google country-specific search engine. These ccTLDs were Australia, Canada, India, United Kingdom, and United States. These ccTLDs were most frequently used English-language search engines. The five country-specific search engines were used to include diverse websites. Although there was much overlap in the websites found in each country-specific search, there were some new websites found in each country-specific search.

This process resulted in 15 separate searches (3 key words × 5 ccTLDs). For each search, the first 20 websites (to cover the first two pages in Google search) from each search that met the inclusion criteria were included in analyses. Duplicate websites (i.e., those identified by more than one search term or more than one ccTLD) were removed from the analysis. To be included in the analysis, the website must (a) be in English language, (b) provide information about “tinnitus” and its treatment, (c) not be identified as advertisement, news, images, and/or videos by Google; and (d) be classifiable as commercial, nonprofit organization, government, personal, or university.


#

Quality Assessment

Two measures were used in the quality assessment of the information in the websites.

HON certification ([Boyer et al, 1998]): The HON code of conduct helps standardize the reliability of medical and health information available on the World-Wide Web. Websites having HON certification are known to have higher reliability and credibility of information when compared with websites that do not have HON certification. Verification of HON certification is provided online at http://www.hon.ch/. Websites are entered in the search and are identified as certified or not.

DISCERN

The DISCERN scale has 16 items. The scores for each item can range between 1 and 5, and higher scores indicate better quality. Please note that DISCERN is not an abbreviation. Rating of 1 indicates that the answer to the item is a definite no (i.e., quality criterion has not been met at all), rating of 2–4 indicates answer to the item is partial (i.e., quality criterion has been met to some extent), and a rating of 5 indicate answer to the item is a definite yes (i.e., quality criterion has been completely met). Items 1–7 assess the publication reliability; items 8–15 assess how good the quality of information on treatment choice is; and item 16 assess the overall quality. One researcher rated each website based on the 16 items for quality rating ([Charnock et al, 1999]). A second researcher rated 60% of the websites and any disparities were discussed and a consensus reached. Both researchers used the DISCERN handbook and questionnaire in their analysis of each website. Before analysis of tinnitus websites, both researchers had completed the DISCERN rating for five websites on hearing loss, which were not a part of this study and discussed inconsistencies between their ratings.


#
#

Readability Assessment

Readability of the website text was assessed using Oleander Software (http://www.oleandersolutions.com/). There are number of available readability measures, but currently there is no standard for choosing readability formulas ([Breese and Burman, 2005]). For this reason, the three readability measures most widely used were selected ([Ley and Florio, 1996]). These included the following:

Flesch Reading Ease (FRE; [Flesch, 1948]) formula measures the readability of the text written between Grade 5 and college-graduate level. The FRE is measured as a reading ease score, with 0 indicating that the text is very difficult to read and 100 indicating that the text is very easy to read. The FRE assesses at least three 100-word passages from the text by taking into account the average words per sentence and the average syllables per word. The FRE is highly correlated with other readability measures ([Meade and Smith, 1991]).

The Flesch-Kincaid Reading Grade Level (F-K RGL) formula was adapted from the FRE, and reports the results in an equivalent RGL. The availability of F-K RGL as a Microsoft Word tool makes it convenient for health-care professionals. The FRE and F-K RGL formulas have been criticized as underestimating the RGL of a passage (and hence, would overestimate how readable the passage is) because they require that individuals reading at the calculated level only need to comprehend 75% of the text ([D’Alessandro et al, 2001]).

The Simple Measure of Gobbledygook (SMOG; [McLaughlin, 1969]) estimates the RGL based on an examination of ten consecutive sentences selected from the beginning, middle, and end of a text. The SMOG takes into account the number of words containing three or more syllables. It is valid between the 3rd and 19th RGL ([Kahn and Pannbacker, 2000]). It represents one of the most exacting readability measures ([Walsh and Volsko, 2008]) because it calculates RGL based on 100% comprehension of the text and is therefore a preferred measure by many health-care professionals ([D’Alessandro et al, 2001]; [Wang et al, 2013]; [Shoemaker et al, 2014]).


#

Data Analysis

Data were analyzed using the IBM SPSS Statistics 22 software. In the first instance, descriptive statistics were explored. In addition, the following statistics were used: χ2 test, t-test, analysis of variance (ANOVA), Pearson’s correlation, and interclass correlation coefficient. Assumptions of normality were tested and relevant data were transformed to achieve normality as necessary. An alpha level of 0.05 was used to determine significance for all statistical analyses, and Bonferroni correction was applied for multiple comparisons.


#
#

RESULTS

In total, 300 websites were assessed (3 key pair words × 5 ccTLDs × 20 first appearing websites meeting the inclusion criteria), and after excluding the duplicates, 134 websites were included in the analysis. [Table 1] provides information about website origin, summary of quality, and readability measures.

Table 1

Website Origin, Quality (HON Certification and DISCERN Scores), and Readability (FRE Score, F-K RGL Formula, and SMOG) for the 134 Websites

Percentage

(Mean ± SD [Range])

Origin (%)

 Commercial

49.30

 Nonprofit organization

38.80

 Government

5.20

 Personal

2.20

 University

4.50

Quality: HON certification

 Yes

13.40

 No

86.60

Quality: DISCERN scores

2.39 ± 1.2 (1–5)

Readability: FRE score

47.82 ± 13.4 (6–81)

Readability: F-K RGL formula

10.27 ± 2.5 (5.8–18.6)

Readability: SMOG

11.85 ± 2.2 (7.7–19)

Note: SD = standard deviation.


Origin

Each website’s origin was recorded: 66 (49.3%) of the websites were of commercial origin, 52 (38.8%) of the websites were of nonprofit organization origin, 7 (5.2%) of the websites were of government origin, 3 (2.2%) of the websites were of personal origin, and 6 (4.5%) of the websites were of university origin. χ2 analysis showed that distribution of websites based on their origin are not equal [χ 2 (4) = 127.3, p < 0.0001].


#

HON Certification

Only 18 of the 134 websites (i.e., 13.5%) had obtained HON certification. χ2 analysis showed that there was significant association between website origin and HON certification [χ 2 (4) = 132.9, p < 0.0001]. None of the websites that were of a personal or university origin had HON certification. However, 12%, 13.5%, and 42.9% of commercial, nonprofit organization, and government websites, respectively, had the HON certification. These results indicate that the websites that were of government origin were more likely to have HON certification compared to all other origins.


#

DISCERN Scores

[Table 2] provides details of DISCERN scores for each item. The items 14 (i.e., Is it clear that there may be more than one treatment choice?), 1 (i.e., Are the aims clear?), and 6 (i.e., Is it balanced and unbiased?) had the highest DISCERN scores, whereas the items 11 (i.e., Does it describe the risk of each treatment?) and 12 (i.e., Does it describe what would happen if no treatment is used?) had the lowest DISCERN scores.

Table 2

DISCERN Quality Criteria for Consumer Health Information on Treatment Choice ([Charnock et al, 1999])

Items

Mean (SD)

1. Are the aims clear?

2.75 (1.4)

2. Does it achieve its aims?

2.07 (1.1)

3. Is it relevant?

2.05 (1.2)

4. Is it clear what sources of information were used to compile the publication (other than the author or procedure)?

2.04 (1.4)

5. Is it clear when the information is used or reported in the publication was reported?

2.16 (1.5)

6. Is it balanced and unbiased?

2.75 (1.3)

7. Does it provide details of additional sources of support and information?

2.33 (1.3)

8. Does it refer to areas of uncertainty?

2.56 (1.2)

9. Does it describe how each treatment works?

2.26 (1.4)

10. Does it describe the benefit of each treatment?

2.19 (1.3)

11. Does it describe the risk of each treatment?

1.46 (1.0)

12. Does it describe what would happen if no treatment is used?

1.47 (0.9)

13. Does it describe how the treatment choices affect overall quality of life?

1.98 (1.1)

14. Is it clear that there may be more than one treatment choice?

3.23 (1.5)

15. Does it provide support for shared decision-making?

1.96 (1.2)

16. Based on the answers to all the above questions, rate the overall quality of the publication as a source of information about treatment choices.

2.39 (1.2)

Note: Mean and standard deviation (SD) for each items of the scale for 134 websites assessed.


The first 15 items of DISCERN provide information about a specific aspect of quality, whereas the item 16 is an overall quality measure based on the 15 different aspects. Hence, the following analysis was based on the overall quality (i.e., DISCERN item 16). For item 16, the low rating (i.e., 1) indicates serious or extensive shortcomings, moderate scores (i.e., 2–4) indicate potentially important but no serious shortcomings, and high ratings (i.e., 5) indicate minimal shortcomings. The total DISCERN scores were not normally distributed and presented a positive skew. Hence, logarithmic transformation of data was performed before the statistical analysis. However, the original DISCERN scores have been reported for the ease of interpretation.

The DISCERN scores ranged from 1 to 5 with a mean of 2.39. [Table 3] provides mean and standard deviation of DISCERN scores for tinnitus websites based on its origin. The ANOVA results [F (4, 129) = 0.85, p = 0.49] showed no difference in DISCERN scores based on the origin of websites.

Table 3

DISCERN Scores for Websites Based on the Origin (Mean and Standard Deviation [SD])

Website Origin

Mean (SD)

Commercial

2.23 (1.1)

Nonprofit organization

2.60 (1.2)

Government

2.57 (0.9)

Personal

2.00 (1.0)

University

2.33 (1.6)

The DISCERN ratings of all the 134 websites were performed by one of the researchers (A.L.D.) and another researcher (A.F.) performed randomly selected sample of websites (i.e., 60%). The interrater agreement for DISCERN was high as indicated by the interclass correlation coefficient of 0.86. This was comparable to earlier reports, which have reported interclass correlation coefficient of 0.82 ([Ademiluyi et al, 2003]) and 0.88 ([Laplante-Lévesque et al, 2012]).


#

Readability

[Table 4] provides details of readability measures based on website origin. FRE score had a mean of 47.82 with a range of 6–81. FRE scores can be translated into F-K RGL Formula. The F-K RGL Formula had a mean of 10.27 with a range of 5.8–18.6. The SMOG had a mean of 11.85 with a range of 7.7–19. These results suggest that on average, people required at least 10–12 yr of education to read and understand tinnitus websites. One-way ANOVA showed no significant difference in FRE score [F (4,129) = 0.35, p = 0.84], F-K RGL Formula [F (4,129) = 0.22, p = 0.93], and SMOG scores [F (4,129) = 0.07, p = 0.99]. These results suggest that the readability was not dependent on website origin. [Table 5] presents the percentage of tinnitus websites exceeding the recommended health literacy level of fifth to sixth RGL and the average US adult grade level of seventh to eighth RGL. Virtually all the websites exceeded the most stringent reading level recommended for health information. In addition, over 83% exceeded the average US adult RGL.

Table 4

Readability Scores for Website Based on the Origin

Readability measure

Website Origin (n) Mean (±SD)

All websites (n = 134)

Commercial (n = 66)

Nonprofit (n = 52)

Government (n = 7)

Personal (n = 3)

University (n = 6)

FRE score

47.82 (±13.4)

47.27 (±13.4)

48.54 (±13.4)

46.14 (±12.5)

55.33 (±10.9)

45.83 (±16.8)

F-K RGL

10.27 (±2.5)

10.43 (±2.7)

10.16 (±2.4)

10.3 (±2.7)

9.40 (±2.0)

9.85 (±2.4)

SMOG RGL

11.85 (±2.1)

11.91 (±2.2)

11.76 (±2.2)

11.98 (±2.3)

12.13 (±1.6)

11.61 (±1.8)

Note: SD = standard deviation.


Table 5

Percentage of Consumer Articles Exceeding the Recommended Health Literacy Levels and Those Exceeding the Average U.S. Adult Reading Grade Level

Grade Levels

FRE (%)

F-KGL (%)

SMOG (%)

Exceeds fifth to sixth grade (recommendation)

99.3

100

100

Exceeds seventh to eighth grade (U.S. adult average)

82.8

94.8

100

The FRE Score had moderate negative correlation with SMOG (Pearson’s correlation coefficient r = −0.47, p < 0.01), F-K RGL formula had a strong negative correlation with SMOG (r = −0.81, p < 0.01), and FRE Score had a strong negative correlation with F-K RGL formula (r = −0.81, p < 0.01).


#

Association between Quality and Readability

The association between quality and readability of websites was assessed using the Pearson’s correlation test between DISCERN scores and readability measures. DISCERN had a small negative correlation with FRE Score (r = −0.26, p < 0.01), had a small negative correlation with F-K RGL Formula (r = 0.20, p < 0.05), and had no statistically significant correlation with SMOG.


#
#

DISCUSSION

This study assessed quality and readability of English-language internet information available for tinnitus. Results were consistent with other research which has demonstrated that a large portion of internet health-care information is written at RGLs, which are too high for many American adults to comprehend ([Laplante-Lévesque et al, 2012]; [Laplante-Lévesque and Thorén, 2015]). The mean FRE score indicated that the reading level was difficult on tinnitus websites. The F-K RGL and SMOG RGL suggested the websites were written well above the recommended levels of fifth or sixth grade. Minimal differences existed across the website origin; in that, all websites exceeded the recommended readability levels. Moreover, both DISCERN and HON certification results indicated that the quality of websites was low.

Quality of Tinnitus Website Information

Even websites that meet the recommended readability levels need to maintain high-quality information. It has been documented repeatedly that both clients ([Couper et al, 2010]; [Fox, 2014]) and practitioners ([El-Shunnar et al, 2011]) often rely on the information obtained from websites. As such, it is imperative that the information contained on these sites is accurate, unbiased, and complete.

Examination of the tinnitus-related websites in the current study found the quality of information on the sites to be significantly lacking. HON certification was found for only 13% of the examined sites, indicating that most websites searched had no standardized quality certification, verifying that the information contained was accurate and unbiased.

Of the 16 items on the DISCERN, only item 14 (i.e., Is it clear there may be more than one treatment choice?) received an average rating of over 3. Four of the 16 items received an average rating of less than two. According to the DISCERN guidelines, a rating of 1 indicates that the quality criterion has not been met at all and a rating of 2–4 indicates that the publication only partially meets the quality criterion being considered. The overall quality of the websites (i.e., item 16) in this study received a mean score of 2.39, indicating that there were potentially important shortcomings in the searched sites.

Why do we see such a lack of high-quality websites related to tinnitus? In examining the types of websites, it was found that most searched sites were commercial in nature. Because there is no true, documented “cure” for tinnitus, management strategies are aimed at lessening the impact of tinnitus. In many cases, this is accomplished by making the tinnitus less noticeable or bothersome through the use of maskers, relaxation techniques, over-the-counter, herbal, or dietary supplements, home remedies, or other treatments. Different management strategies have led to a plethora of commercially based websites. In this study, many of these sites gave enough information on tinnitus to sell a product, but did not necessarily give sufficiently accurate and unbiased information. Further, the nature of tinnitus is such that each individual who suffers from it experiences it differently. Many websites shared personal experiences and success stories, which, while helpful in some cases, do not always give accurate and impartial quality information.

It should be noted that the lowest scoring items on the DISCERN were those relating to potential treatment risks, effects that treatments may have on quality of life, and impact if no treatment was used. As stated previously, possibly because of the commercial nature of many of the websites, treatment risks were often not discussed, potentially to sell a product. Those sites that were personal in nature tended to share success stories rather than discuss potential treatment risks. Even websites designed primarily for educational purposes often lacked a discussion on potential treatment risks, and effects that treatments may have on quality of life.

Overall, the quality of the websites examined in this study was generally poor. These findings suggest the need for improved accountability and accuracy for tinnitus-related websites. The standard of quality of internet information related to tinnitus (and other health-related matters) should be raised so that clients can feel confident in trusting internet information. Practitioners and clients alike should be aware of certifying agencies (e.g., HON) and quality measures when searching for tinnitus-related information.


#

Readability of Tinnitus Website Information

Raising the quality standard of tinnitus website information would be a somewhat futile undertaking if the readability levels of the websites remain above the recommended reading levels for adults. As stated previously, tinnitus is a complex complaint that often presents alongside other disorders, such as insomnia, depression, or anxiety ([Fackrell et al, 2012]). In addition, tinnitus may present differently from person to person, and management strategies may have differential results. Due to the inherent complexities in the symptomatology and diagnostic process, as well as the variability in management strategies, decreasing readability levels while maintaining high-quality information is no small task.

Findings in this study suggest that the vast majority of tinnitus-related websites are well above the recommended readability levels and not accessible to the average client. Why is the readability level for tinnitus websites (and other health-related websites) so high? [Laplante-Lévesque et al (2012)] stated that readability is the ease with which information can be read and understood. According to these authors, readability of a document is affected by both jargon (e.g., the use of words that are specific to a particular context and which may be misunderstood outside that context) and complex linguistic structures (e.g., polysyllabic words and lengthy sentences), both of which were commonly used in the tinnitus-related websites searched in this study.

Moreover, it important to understand the difference between “readability” and “health literacy.” The [WHO (2017)] defines health literacy as the cognitive and social skills that determine both the motivation and the ability of people to access health information and to understand and use that information. Using this definition, readability can be seen as one aspect of health literacy, but is not a proxy measure for health literacy.

When we examine the tinnitus websites carefully, it is clear that polysyllabic words such as tinnitus are very common. Having polysyllabic words in a passage by definition reduces the readability of the passage, as measured by the readability formulas that assess syllable count. That is, the passage will be more difficult to read for people with low reading abilities, compared with passages that contain fewer syllables (or syllables per word). However, repetition of words can potentially facilitate health literacy by promoting the understanding of the concept and bringing the jargon or polysyllabic word into the reader’s vocabulary. The only way to truly understand the effect of repeating polysyllabic or jargon words (such as tinnitus) on a reader’s health literacy is to obtain behavioral measures of comprehension (e.g., the Cloze test or a comprehension task).

This study was focused on readability. Improving readability can potentially make health information more accessible to people with low reading abilities, which is one aspect of health literacy. Future work needs to focus on the larger issue of promoting health literacy for people who experience tinnitus. It is not enough to simply improve readability. To improve health outcomes, the larger goal of improving health literacy must also be achieved.


#

Association between Quality and Readability

Some association between quality (i.e., DISCERN) and readability (i.e., FRE Scores, F-K RGL Formula) was noted. These results indicated that websites with higher quality also had lower readability, which suggest that some websites had accounted for both quality and readability of the information to facilitate comprehension of its content by its users.


#

Reconciling Quality and Readability of Tinnitus-Related Information

In examining these findings, it seems necessary to find a “middle-ground” to reconcile the need for high-quality, accurate information, while still maintaining a readability level that is consistent with that of the average client. How should this be accomplished with regard to tinnitus? As discussed previously, tinnitus is a complex complaint that, in most cases, requires a significant amount of description and discussion to maintain accuracy and quality. Readability, on the other hand, signifies relative ease of reading and understanding written text. Therefore, at least in the realm of tinnitus, quality and readability seem to be opposing factors. However, readability is influenced by factors such as word complexity and sentence length ([Laplante-Lévesque et al, 2012]). Therefore, there are simple steps that can be taken to ensure low levels of readability while still maintaining high-quality information. Terms should be clearly defined at the outset, and the use of polysyllabic, complex, jargon words should be avoided. Furthermore, the sentence structure should remain simple, eliminating the use of complex clauses and lengthy sentences. Accurate, high-quality information is still maintained in this way, but the complexity of the vocabulary and syntax is reduced, thereby lowering the readability level and allowing accessibility to the public. For example, take the following sentence: “While it is commonly referred to as ‘ringing in the ears’, tinnitus can manifest many different perceptions of sound, including buzzing, hissing, whistling, swooshing, and clicking” (www.ata.org). This sentence could be made easier to read by simplifying both the structure and word complexity. For example, “Tinnitus is often called ‘ringing in the ears’. But, it does not sound like ringing to everyone. Some people report that it sounds like buzzing, hissing, whistling, swooshing, or clicking.” By taking these steps, stakeholders (web developers, clinicians) can provide accessible information to their clients/patients, while still maintaining high quality of information required for tinnitus-related websites.


#

Study Implications

As stated earlier, there has been a tremendous increase in the number of persons seeking health information online. [Fox (2014)] reported that 79% of American internet users seek health-related information. Hence, this is an ideal time to consider how the health information for various symptoms and conditions are presented online. Findings of this study suggest the need for improvement in presentation of healthcare-related information regarding tinnitus.

In addition, one of the leading causes of tinnitus is noise exposure. Noise exposure is common in manual labor positions, and manual labor has been associated with low levels of education ([Unterrainer et al, 2001]). For this reason, it is likely that those with manual jobs who are highly susceptible to noise-induced tinnitus might have lower reading levels. Therefore, it is critical to consider the readability of tinnitus websites.


#

Study Limitations and Future Directions

The current study had several limitations. First, the use of Google as the only search engine may not represent all potential search methods. The Google search engine was chosen because it was the most commonly used (i.e., 81% of all the searches globally) internet search engine when compared to other search engines such as Yahoo, Baidu, and Bing ([Net Marketshare, 2017]). However, considering that there are multiple search engines, future research on this topic may be more inclusive. Second, while key words were selected to represent the most common searches, it is possible that information seekers might use other terms. Use of other key words could result in different information results. This needs to be further investigated.

Third, the website design and presentation of information is reported to influence the perception of credibility ([Robins et al, 2010]). This was not assessed in the present study. These factors could influence the understanding and knowledge attainment and should be investigated in future studies. Fourth, although three different and common readability measures were used, readability formulae do not assess all the dimensions related to the reading process ([Meade and Smith, 1991]). For this reason, other avenues need be investigated for a fuller picture of patient access and understanding. Fifth, quality and readability of websites were assessed; accuracy of the content of the websites was not assessed in fine detail. Future studies could include the assessment of content presented in these websites with experts rating the accuracy of content presented.

Sixth, while we used five country-specific search engines to include a wide range of websites, we are unsure what the recommended RGLs are for other countries (i.e., Australia, Canada, India, and United Kingdom). For example, as English is not the primary language in India, we anticipate that the recommended RGLs for health information may be lower when compared to United States. Hence, these results must be interpreted cautiously by taking the local language and health literacy levels into consideration. Finally, the present investigation only considered online sources. There still exists a great need to assess the health information for tinnitus through other sources (e.g., handouts, self-help materials, etc.). This should be considered in future studies.


#
#

CONCLUSIONS

The study evaluated the quality and readability of English-language internet information for tinnitus. The webpages highlight great variability in the quality and readability of health information. These findings highlight the need for stakeholders (e.g., web-developers, clinicians) to be aware of this and to develop more user-friendly health information on websites to make it more accessible for people with low literacy.


#

Abbreviations

ANOVA: analysis of variance
ccTLD: country-coded top level domain
F-K RGL: Flesch-Kincaid reading grade level formula
FRE: Flesch reading ease
GP: general practitioner
HON: health on the net
RGL: reading grade level
SMOG: simple measure of gobbledygook


#

No conflict of interest has been declared by the author(s).

  • REFERENCES

  • Ademiluyi G, Rees CE, Sheard CE. 2003; Evaluating the reliability and validity of three tools to assess the quality of health information on the Internet. Patient Educ Couns 50 (02) 151-155
  • Atcherson SR, DeLaune AE, Hadden K, Zraick RI, Kelly-Campbell RJ, Minaya CP. 2014; A computer-based readability analysis of consumer materials on the American Speech-Language-Hearing association website. Contemp Issues Commun Sci Disord 41: 12-23
  • Atcherson SR, Zraick RI, Brasseux RE. 2011; Readability of patient-reported outcome questionnaires for use with persons with tinnitus. Ear Hear 32 (05) 671-673
  • Baguley D, McFerran D, Hall D. 2013; Tinnitus. Lancet 382 (9904) 1600-1607
  • Bauer CA, Brozoski TJ. 2011; Effect of tinnitus retraining therapy on the loudness and annoyance of tinnitus: a controlled trial. Ear Hear 32 (02) 145-155
  • Breese P, Burman W. 2005; Readability of notice of privacy forms used by major health care institutions. JAMA 293 (13) 1593-1594
  • Boyer C, Selby M, Scherrer JR, Appel RD. 1998; The Health On the Net code of conduct for medical and health websites. Comput Biol Med 28 (05) 603-610
  • Charnock D, Shepperd S, Needham G, Gann R. 1999; DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 53 (02) 105-111
  • Couper MP, Singer E, Levin CA, Fowler Jr FJ, Fagerlin A, Zikmund-Fisher BJ. 2010; Use of the Internet and ratings of information sources for medical decisions: results from the DECISIONS survey. Med Decis Making 30 (05) (Suppl) 106S-114S
  • D’Alessandro DM, Kingsley P, Johnson-West J. 2001; The readability of pediatric patient education materials on the World Wide Web. Arch Pediatr Adolesc Med 155 (07) 807-812
  • Doak CC, Doak LG, Root JH. 1996. Teaching Patients with Low Literacy Skills. Philadelphia, PA: J.B. Lippincott Company;
  • El-Shunnar SK, Hoare DJ, Smith S, Gander PE, Kang S, Fackrell K, Hall DA. 2011; Primary care for tinnitus: practice and opinion among GPs in England. J Eval Clin Pract 17 (04) 684-692
  • Eloy JA, Li S, Kasabwala K, Agarwal N, Hansberry DR, Baredes S, Setzen M. 2012; Readability assessment of patient education materials on major otolaryngology association websites. Otolaryngol Head Neck Surg 147 (05) 848-854
  • Fackrell K, Hoare DJ, Smith S, McCormack A, Hall DA. 2012; An evaluation of the content and quality of tinnitus information on websites preferred by General Practitioners. BMC Med Inform Decis Mak 12 (01) 70
  • Flesch R. 1948; A new readability yardstick. J Appl Psychol 32 (03) 221-233
  • Folmer RL, Martin WH, Shi Y. 2004; Tinnitus: questions to reveal the cause, answers to provide relief. J Fam Pract 53 (07) 532-540
  • Fox S. 2014 Americans Feel Better Informed Thanks to the Internet. http://www.pewinternet.org/2014/12/08/better-informed/ . Accessed August 24, 2017
  • Hesser H, Weise C, Westin VZ, Andersson G. 2011; A systematic review and meta-analysis of randomized controlled trials of cognitive-behavioral therapy for tinnitus distress. Clin Psychol Rev 31 (04) 545-553
  • Hsu P-H. 2017 Readability of hearing related Internet information in traditional Chinese. Masters Dissertation, University of Canterbury. https://ir.canterbury.ac.nz/bitstream/handle/10092/13652/MAud%20thesis%20-%20Po-Hsiang%20%28Paul%29%20Hsu.pdf?sequence=1&isAllowed=y . Accessed October 6, 2017
  • Joseph J, Svider PF, Shaigany K, Eloy JA, McDonald PG, Folbe AJ, Hong RS. 2016; Hearing aid patient education materials: is there room for improvement?. J Am Acad Audiol 27 (04) 354-359
  • Kahn A, Pannbacker M. 2000; Readability of educational materials for clients with cleft lip/palate and their families. Am J Speech Lang Pathol 9 (01) 3-9
  • Kelly-Campbell RJ, Atcherson SR, Zimmerman KR, Zraick RI. 2012; Readability of audiologic self-report assessment tools. J Am Rehab Acad 45: 63-73
  • Kirsch IS. 1993. Adult Literacy in America: A First Look at the Results of the National Adult Literacy Survey. Washington, DC: Office of Education Research and Improvement, U.S. Dept. of Education;
  • Laplante-Lévesque A, Brännström KJ, Andersson G, Lunner T. 2012; Quality and readability of English-language internet information for adults with hearing impairment and their significant others. Int J Audiol 51 (08) 618-626
  • Laplante-Lévesque A, Thorén ES. 2015; Readability of Internet information on hearing: systematic literature review. Am J Audiol 24 (03) 284-288
  • Ley P, Florio T. 1996; The use of readability formulas in healthcare. Psychol Health Med 1 (01) 7-28
  • Loumidis KS, Hallam RS, Cadge B. 1991; The effect of written reassuring information on out-patients complaining of tinnitus. Br J Audiol 25 (02) 105-109
  • Martinez-Devesa P, Perera R, Theodoulou M, Waddell A. 2010; Cognitive behavioural therapy for tinnitus. Cochrane Database Syst Rev (09) CD005233
  • McLaughlin G. 1969; SMOG grading: a new readability formula. J Read 12 (08) 639-646
  • Meade CD, Smith CF. 1991; Readability formulas: cautions and criteria. Patient Educ Couns 17 (02) 153-158
  • Net Marketshare 2017 Desktop Search Engine Market Share. http://marketshare.hitslink.com . Accessed August 24, 2017
  • Noble W. 2012. Self-Assessment of Hearing. 2nd ed. San Diego, CA: Plural Publishing Inc.;
  • Robins D, Holmes J, Stansbury M. 2010; Consumer health information on the web: the relationship of visual design and perceptions of credibility. J Am Soc Inf Sci Technol 61 (01) 13-29
  • Shoemaker SJ, Wolf MS, Brach C. 2014; Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns 96 (03) 395-403
  • Sindhusake D, Mitchell P, Newall P, Golding M, Rochtchina E, Rubin G. 2003; Prevalence and characteristics of tinnitus in older adults: the Blue Mountains Hearing Study. Int J Audiol 42 (05) 289-294
  • Tunkel DE. et al. 2014; Clinical practice guideline: tinnitus. Otolaryngol Head Neck Surg 151 (02) (Suppl) S1-S40
  • Unterrainer J, Greimel KV, Leibetseder M. 2001; Are demographic and socioeconomic factors predictive for perceived tinnitus impairment?. Int Tinnitus J 7 (02) 109-111
  • Walsh TM, Volsko TA. 2008; Readability assessment of internet-based consumer health information. Respir Care 53 (10) 1310-1315
  • Wang LW, Miller MJ, Schmitt MR, Wen FK. 2013; Assessing readability formula differences with written health information materials: application, results, and recommendations. Res Social Adm Pharm 9 (05) 503-516
  • Weiss BD, Coyne C. 1997; Communicating with patients who cannot read. N Engl J Med 337 (04) 272-274
  • World Health Organization 2017 Health Literacy and Health Behaviour. http://www.who.int/healthpromotion/conferences/7gchp/track2/en/ . Accessed August 24, 2017

Corresponding author

Vinaya Manchaiah
Department of Speech and Hearing Sciences, Lamar University
Beaumont, TX 77710

  • REFERENCES

  • Ademiluyi G, Rees CE, Sheard CE. 2003; Evaluating the reliability and validity of three tools to assess the quality of health information on the Internet. Patient Educ Couns 50 (02) 151-155
  • Atcherson SR, DeLaune AE, Hadden K, Zraick RI, Kelly-Campbell RJ, Minaya CP. 2014; A computer-based readability analysis of consumer materials on the American Speech-Language-Hearing association website. Contemp Issues Commun Sci Disord 41: 12-23
  • Atcherson SR, Zraick RI, Brasseux RE. 2011; Readability of patient-reported outcome questionnaires for use with persons with tinnitus. Ear Hear 32 (05) 671-673
  • Baguley D, McFerran D, Hall D. 2013; Tinnitus. Lancet 382 (9904) 1600-1607
  • Bauer CA, Brozoski TJ. 2011; Effect of tinnitus retraining therapy on the loudness and annoyance of tinnitus: a controlled trial. Ear Hear 32 (02) 145-155
  • Breese P, Burman W. 2005; Readability of notice of privacy forms used by major health care institutions. JAMA 293 (13) 1593-1594
  • Boyer C, Selby M, Scherrer JR, Appel RD. 1998; The Health On the Net code of conduct for medical and health websites. Comput Biol Med 28 (05) 603-610
  • Charnock D, Shepperd S, Needham G, Gann R. 1999; DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 53 (02) 105-111
  • Couper MP, Singer E, Levin CA, Fowler Jr FJ, Fagerlin A, Zikmund-Fisher BJ. 2010; Use of the Internet and ratings of information sources for medical decisions: results from the DECISIONS survey. Med Decis Making 30 (05) (Suppl) 106S-114S
  • D’Alessandro DM, Kingsley P, Johnson-West J. 2001; The readability of pediatric patient education materials on the World Wide Web. Arch Pediatr Adolesc Med 155 (07) 807-812
  • Doak CC, Doak LG, Root JH. 1996. Teaching Patients with Low Literacy Skills. Philadelphia, PA: J.B. Lippincott Company;
  • El-Shunnar SK, Hoare DJ, Smith S, Gander PE, Kang S, Fackrell K, Hall DA. 2011; Primary care for tinnitus: practice and opinion among GPs in England. J Eval Clin Pract 17 (04) 684-692
  • Eloy JA, Li S, Kasabwala K, Agarwal N, Hansberry DR, Baredes S, Setzen M. 2012; Readability assessment of patient education materials on major otolaryngology association websites. Otolaryngol Head Neck Surg 147 (05) 848-854
  • Fackrell K, Hoare DJ, Smith S, McCormack A, Hall DA. 2012; An evaluation of the content and quality of tinnitus information on websites preferred by General Practitioners. BMC Med Inform Decis Mak 12 (01) 70
  • Flesch R. 1948; A new readability yardstick. J Appl Psychol 32 (03) 221-233
  • Folmer RL, Martin WH, Shi Y. 2004; Tinnitus: questions to reveal the cause, answers to provide relief. J Fam Pract 53 (07) 532-540
  • Fox S. 2014 Americans Feel Better Informed Thanks to the Internet. http://www.pewinternet.org/2014/12/08/better-informed/ . Accessed August 24, 2017
  • Hesser H, Weise C, Westin VZ, Andersson G. 2011; A systematic review and meta-analysis of randomized controlled trials of cognitive-behavioral therapy for tinnitus distress. Clin Psychol Rev 31 (04) 545-553
  • Hsu P-H. 2017 Readability of hearing related Internet information in traditional Chinese. Masters Dissertation, University of Canterbury. https://ir.canterbury.ac.nz/bitstream/handle/10092/13652/MAud%20thesis%20-%20Po-Hsiang%20%28Paul%29%20Hsu.pdf?sequence=1&isAllowed=y . Accessed October 6, 2017
  • Joseph J, Svider PF, Shaigany K, Eloy JA, McDonald PG, Folbe AJ, Hong RS. 2016; Hearing aid patient education materials: is there room for improvement?. J Am Acad Audiol 27 (04) 354-359
  • Kahn A, Pannbacker M. 2000; Readability of educational materials for clients with cleft lip/palate and their families. Am J Speech Lang Pathol 9 (01) 3-9
  • Kelly-Campbell RJ, Atcherson SR, Zimmerman KR, Zraick RI. 2012; Readability of audiologic self-report assessment tools. J Am Rehab Acad 45: 63-73
  • Kirsch IS. 1993. Adult Literacy in America: A First Look at the Results of the National Adult Literacy Survey. Washington, DC: Office of Education Research and Improvement, U.S. Dept. of Education;
  • Laplante-Lévesque A, Brännström KJ, Andersson G, Lunner T. 2012; Quality and readability of English-language internet information for adults with hearing impairment and their significant others. Int J Audiol 51 (08) 618-626
  • Laplante-Lévesque A, Thorén ES. 2015; Readability of Internet information on hearing: systematic literature review. Am J Audiol 24 (03) 284-288
  • Ley P, Florio T. 1996; The use of readability formulas in healthcare. Psychol Health Med 1 (01) 7-28
  • Loumidis KS, Hallam RS, Cadge B. 1991; The effect of written reassuring information on out-patients complaining of tinnitus. Br J Audiol 25 (02) 105-109
  • Martinez-Devesa P, Perera R, Theodoulou M, Waddell A. 2010; Cognitive behavioural therapy for tinnitus. Cochrane Database Syst Rev (09) CD005233
  • McLaughlin G. 1969; SMOG grading: a new readability formula. J Read 12 (08) 639-646
  • Meade CD, Smith CF. 1991; Readability formulas: cautions and criteria. Patient Educ Couns 17 (02) 153-158
  • Net Marketshare 2017 Desktop Search Engine Market Share. http://marketshare.hitslink.com . Accessed August 24, 2017
  • Noble W. 2012. Self-Assessment of Hearing. 2nd ed. San Diego, CA: Plural Publishing Inc.;
  • Robins D, Holmes J, Stansbury M. 2010; Consumer health information on the web: the relationship of visual design and perceptions of credibility. J Am Soc Inf Sci Technol 61 (01) 13-29
  • Shoemaker SJ, Wolf MS, Brach C. 2014; Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns 96 (03) 395-403
  • Sindhusake D, Mitchell P, Newall P, Golding M, Rochtchina E, Rubin G. 2003; Prevalence and characteristics of tinnitus in older adults: the Blue Mountains Hearing Study. Int J Audiol 42 (05) 289-294
  • Tunkel DE. et al. 2014; Clinical practice guideline: tinnitus. Otolaryngol Head Neck Surg 151 (02) (Suppl) S1-S40
  • Unterrainer J, Greimel KV, Leibetseder M. 2001; Are demographic and socioeconomic factors predictive for perceived tinnitus impairment?. Int Tinnitus J 7 (02) 109-111
  • Walsh TM, Volsko TA. 2008; Readability assessment of internet-based consumer health information. Respir Care 53 (10) 1310-1315
  • Wang LW, Miller MJ, Schmitt MR, Wen FK. 2013; Assessing readability formula differences with written health information materials: application, results, and recommendations. Res Social Adm Pharm 9 (05) 503-516
  • Weiss BD, Coyne C. 1997; Communicating with patients who cannot read. N Engl J Med 337 (04) 272-274
  • World Health Organization 2017 Health Literacy and Health Behaviour. http://www.who.int/healthpromotion/conferences/7gchp/track2/en/ . Accessed August 24, 2017