Evaluation of Vietnamese written materials for diabetes prevention and management in Ho Chi Minh City

Abstract

Introduction

Clear and effective health information is necessary for the prevention and control of diabetes in Vietnam, and written materials are playing an ever-greater role in educating patients. The Vietnamese government stated that access to health information is critically important in improving people’s health overall. However, there are few practical instructions for designing clear and easy-to-understand written materials. We aimed to evaluate the clarity and understandability of Vietnamese written materials on diabetes and healthy lifestyles.

Methods

Twenty-six examples of diabetes-related written materials were collected in Ho Chi Minh City, Vietnam. We created a Vietnamese translation of the Clear Communication Index (CCI), which was developed and validated by the United States Center for Disease Control and Prevention, and scored the collected materials.

Results

The mean CCI total score was 38.4%, and none of the materials achieved a score of 90% or above, which is defined as a sufficient CCI level. The evaluation items that scored particularly poorly were the use of visual content with annotation, placing the main message in the top section or paragraph, explaining what was unknown about the topic, explaining numbers with words, explaining the nature of risk, and mentioning both risks and benefits.

Conclusions

None of the collected materials reached the sufficient CCI score. Use of visual aids, summary of the main points, and explanation of numerical information were particularly lacking. We recommend wider application of the CCI scoring system to improve the quality of written health materials used in clinical practice in Vietnam, which requires promotion of health literacy at the organizational level.

Keywords:

• Health literacy
• health information
• diabetes mellitus
• healthy lifestyle
• Vietnam

Introduction

Preparing effective instructions for patients managing chronic diseases is a complicated task that requires a deep understanding of patients’ daily lives and health behaviors. Diabetes is one such chronic disease that occurs either when the pancreas does not produce enough insulin (blood glucose regulating hormone) or when the body cannot effectively use the insulin it produces. When the disease is not well controlled, resulting high blood sugar over time leads to serious damage to many of the body’s systems, particularly the nerves and blood vessels (World Health Organization, 2023). The disease can progress without being noticed, prompting it to be referred to as a ‘silent killer.’ According to the Vietnamese Ministry of Health’s circular for health literacy, 763/BYT-TT-KT (Vietnamese Ministry of Health, 2022), published in February 2022, providing effective health information to the public plays a critical role in changing people’s behaviors, which is beneficial for preventing disease and improving health overall. The paper also states that designing easy-to-understand information is one of the key factors in achieving improved health ­literacy among the public and preventing the development of common health problems, including non-communicable diseases such as diabetes mellitus.

Despite the importance of clear and concise health information being recognized in Vietnam (Vietnamese Ministry of Health, 2022), there is a distinct lack of suitable instructions and culturally tailored guidelines adapted to the Vietnamese language that explain how to design written health information from a health literacy perspective. This is where organizational health literacy comes in to fill the gap between information needs and accessibility. It is defined as “the degree to which organizations equitably enable individuals to find, understand, and use information and services to inform health-related decisions and actions for themselves and others (U.S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion, 2023).” We emphasize that concerted efforts among health professionals are needed to promote health literacy. Currently, most of the health information training for health professionals in Vietnam focuses on oral communication in groups or with individuals. Meanwhile, the demands on hospitals to provide health information are increasing, particularly following the publication of the above-mentioned circular (Vietnamese Ministry of Health, 2022). A rapid rise in the number of cases of non-communicable diseases (Thuy Duyen et al., 2020) has further increased the need to provide health information tailored to individual patients. With hospitals in Vietnam overloaded by rising numbers of patients (Tri et al., 2022), it is difficult for health professionals to spend sufficient time with each patient to provide detailed lifestyle instructions. This makes it even more important to distribute easy-to-understand written health information in clinical settings. However, the quality of written health information provided by health professionals and its use by patients, their families, and people in the wider community has not been assessed in Vietnam.

Internationally, various tools to assess the accessibility of written health information have been developed and used (Rudd, 2022). These include Suitability Assessment of Materials (SAM) (Doak et al., 1996), developed in the 1990s, and Patient Education Material Assessment Tool (PEMAT) (Shoemaker et al., 2014), developed more recently to assess both print and audiovisual materials. These tools provide valuable insight for rewriting and restructuring health information. The American Center for Disease Control and Prevention (CDC) developed and validated a health literacy assessment tool, the Clear Communication Index (CCI), to evaluate the clarity and understandability of written health materials (Centers for Disease Control and Prevention, 2013). It is an easy-to-apply tool including items to assess information on risks and numbers that are often contained in diabetes-related materials. Goto and colleagues translated and adapted the tool for Japanese public health settings (Goto et al., 2018), and have used it to assess and improve information about the health effects of environmental low-dose radiation exposure for community residents returning home after the Fukushima nuclear accident in 2011. The authors found that the CCI score increased after these materials were revised, although the score did not reach the “sufficient” level of 90% (Baur and Prue, 2014). Mani and colleagues applied CCI to health information relating to the COVID-19 pandemic, finding that only around 10% of written materials on COVID-19 infection reached the sufficient CCI level (Mani et al., 2021). Both cases show the usefulness of CCI for assessing health information when facing a health crisis.

Given the increasing need for quality written health information to help health professionals provide lifestyle instructions to patients in Vietnam, we aimed to evaluate the clarity and understandability of written materials that were available in Ho Chi Minh City, Vietnam. Our focus was on information relating to the prevention and management of diabetes.

Materials and methods

Materials

We collected diabetes-related written health information distributed at general hospitals in Ho Chi Minh City, as well as those written materials published on the webpages of Ho Chi Minh City Center for Disease Control and Prevention (HCDC), the National Institute of Nutrition, and the Vietnamese Association of Diabetes and Endocrinology. One of the co-authors, who had official access to local hospitals as a public servant, collected information from a convenience sample of secondary and tertiary hospitals in Ho Chi Minh City. These included eight specialized hospitals (specialized in traditional medicine, cardiology, ophthalmology, pulmonary and obstetrics and gynecology) and five general hospitals. All available written materials relating to diabetes were collected, regardless of format (leaflet, handbook, brochure), publication date, and topic.

Vietnamese version of Clear Communication Index (CCI)

For the purposes of this study, a Vietnamese–Japanese bilingual language specialist translated the Japanese version of the CCI into Vietnamese, confirming the accuracy of the translation word-by-word with one of the co-authors (AG) before it was finalized for use. We then applied this Vietnamese version of the tool to evaluate the clarity and understandability of the collected written materials.

The CCI consists of 4 parts. Part A (Core), for assessing clarity in general, has eleven questions (evaluation items) in the original English version. However, like the Japanese version, one of these items (“Do both the main message and the call to action use the active voice?”) was excluded in the Vietnamese version because of a basic grammatical difference in the use of passive and active voices in the English and Vietnamese languages. Part A of the Vietnamese version thus has ten evaluation items. The evaluation items that comprise Part B (Behavioral Recommendations), Part C (Numbers), and Part D (Risk) were kept the same as the original English version. All three parts (B, C, D) have three evaluation items each. In total, there are 19 items. Answer options were “Yes (1)” and “No (0),” and the total score was divided by the number of answers for items applicable for the assessed material. The CCI scoring sheet indicates that a score of 90% or more is defined as sufficient in terms of clarity and understandability (Baur and Prue, 2014).

Analysis

We categorized each of the collected written materials according to the publication year, topics (content) covered, and publication source. Each of the written materials was independently evaluated with CCI by three researchers. Then, the researchers discussed any discrepancies in scoring and came to an agreement. We conducted a descriptive analysis of the scores using Microsoft Excel 2010 software. Statistical analysis to examine differences in CCI total score by material category was conducted using the Wilcoxon–Mann–Whitney test and Kruskal–Wallis test. A p-value of less than 0.05 was considered significant.

Results

We collected a total of 26 written materials that were divided into five groups by topic: diet and nutrition for diabetes management (n = 9); prevention of diabetes complications (7); gestational diabetes (4); compilations of various topics including raising awareness about risks and complications, self-management, and model diet (4); and other topics (guidance for insulin usage and steps for diagnosing diabetes) (2). Regarding the publication source, nine of the collected materials were sponsored by for-profit companies, five were issued by the government, and twelve were published by the medical institutions themselves, such as the Departments of Nutrition and Endocrinology at the hospitals from which we obtained the materials. In terms of the format of the written materials, the majority were bi- or tri-folded leaflets (n = 20), four were handbooks, and two were single-sheet brochures.

Table 1 presents the distribution of CCI total scores for the assessed materials. The average CCI total score was 38.4%, and none of the assessed materials met the sufficient score of 90% or above. When testing differences in the total score by material category, we considered publication year, topics mentioned, and publication source (Table 2). Of note, there were nine written materials published before 2012 and seventeen published after 2012. There was no statistically significant difference between the median CCI total score of the two groups (33.3% and 44.4%, respectively, with a p-value of 0.220). Likewise, for other categories, we found no significant differences in the CCI scores.

Table 1. Distribution of CCI total scores of collected materials.

Total score	Frequency (n = 26)	(%)
≥ 90%	0	0.0
50–89%	6	23.1
40–49%	3	11.5
30–39%	10	38.5
< 30%	7	26.9

Table 2. CCI total score and characteristics of collected materials.

	Number of materials	CCI total score Median (min, max)	p value
Publication year
Within 10 years (2012 or later)	17	33.3 (26.0, 56.0)	0.22
10 years or older (before 2012)	9	44.4 (28.0, 56.0)
Topics
Diet and nutrition for diabetes management	9	38.9 (27.8, 55.6)	0.12
Prevention of diabetes complications	7	44.4 (31.6, 55.6)
Others*	10	33.3 (26.3, 50.0)
Publication source
For-profit companies	9	33.3 (26.3, 55.6)	0.32
Government	5	44.4 (31.6, 55.6)
Hospitals	12	33.3 (27.8, 55.6)

Wilcoxon-Mann-Whitney test or Kruskal-Wallis test was used.

* Others include topics such as gestational diabetes, insulin usage, and diagnosis procedures.

Table 3 shows the distribution of CCI assessment scores for each evaluation item. Three of the items in Part A were satisfied (were answered “yes”) by over half of the 26 assessed materials: calls to action for the primary audience (item 5), using bulleted or numbered lists (item 7), and using headings to organize the material (item 8). None of the assessed materials satisfied the following three items: using visual aids for the main message (item 4), summarizing the most important information in the first paragraph or section (item 9), and explaining what is authoritatively known and unknown about the topic (item 10). Most of the materials went straight into the detailed content without first providing the main message or a summary. Even when there were, visual aids were not used to convey or support the main message. Many were missing explanations for medical terms, such as blood glucose and food nutrients, some contained too much information without appropriate sectioning or bullet points, and none mentioned what was known and unknown about the topic.

Table 3. Distribution of Clear Communication Index (CCI) scores among collected diabetes-related health information in Vietnamese.

Evaluation items	Frequency of “Yes”	(%)
Part A: Core	(n = 26)
Main Message and Call to Action
1. Does the material contain one main message statement?	9	34.6
2. Is the main message at the top, beginning, or front of the material?	6	23.1
3. Is the main message emphasized with visual cues?	8	30.8
4. Does the material contain at least one visual that conveys or supports the main message?	0	0.0
5. Does the material include one or more calls to action for the primary audience?	26	100.0
Language
6. Does the material always use words the primary audience uses?	10	38.5
Information Design
7. Does the material use bulleted or numbered lists?	17	65.4
8. Is the material organized in chunks with headings?	24	92.3
9. Is the most important information the primary audience needs summarized in the first paragraph or section?	0	0.0
State of Science
10. Does the material explain what authoritative sources, such as subject matter experts and agency spokespersons, know and don’t know about the topic?	0	0.0
Part B: Behavioral Recommendations	(n = 25)
11. Does the material include one or more behavioral recommendations for the primary audience?	25	100.0
12. Does the material explain why the behavioral recommendation(s) is important to the primary audience?	3	12.0
13. Does the behavioral recommendation(s) include specific directions about how to perform the behavior?	7	28.0
Part C: Numbers	(n = 25)
14. Does the material always present numbers the primary audience uses?	15	60.0
15. Does the material always explain what the numbers mean?	4	16.0
16. Does the audience have to conduct mathematical calculations?^a	17	68.0
Part D: Risk	(n = 26)
17. Does the material explain the nature of the risk?	7	26.9
18. Does the material address both the risks and benefits of the recommended behaviors?	0	0.0
19. If the material uses numeric probability to describe risk, is the probability also explained with words or a visual?	0	0.0

^a Frequency of “No” is indicated for the reverse item.

The assessment scores were also varied in Parts B, C, and D. All of the assessed materials included behavioral recommendations (item 11). Reasons for (item 12) and ways to carry out the recommendations (item 13) were available in 12% and 28% of the materials, respectively. The proportion of materials presenting numbers appropriate for the primary audience (item 14) was 60%, and those not requiring the audience to carry out mathematical calculations (item 16) was 68%. The proportion that always explained the meaning of the numbers using words (item 15) was 16%. There were materials not giving explanations in words when using numerical percentages, and a few requiring calculations for body mass index (BMI) and energy calculation. The proportion of materials explaining the nature of the risks (item 17) was 26.9%, while none of the assessed materials explained risks and benefits for the behavioral recommendations (item 18) or explained risk-related probabilities with words or visuals (item 19).

Discussion

None of our collected written materials on diabetes reached the sufficient CCI level of 90% or higher. As mentioned above, the proportion of materials reaching the sufficient level was also 0% in a previous study in Japan (Goto et al., 2018), and 2% and 11% in studies in the United States (Mani et al., 2021; Porter et al., 2019). When we stratified the written materials into subgroups, we found no improvement in the CCI score among more recently published materials. When collecting the materials for this study, we found that older materials were still in use and were not up-to-date. In Vietnam, health literacy is still a new term. Research to date has mostly been conducted to assess the health literacy levels of people (Duong et al., 2019; Duong et al., 2017) rather than to strengthen the development and dissemination of health information. To our knowledge, the present study is the first in the country reporting results from assessing health information being used in medical practice by translating and using the Vietnamese version of CCI. Given that previous research reported improved communication competencies and understandability of information revised by health professionals attending a health literacy workshop (Honda et al., 2022; Goto et al., 2021), we recommend health literacy skills training for health professionals (Goto et al., 2018) that encourages wider use of the CDC’s CCI scoring system as a checklist to improve the clarity and understandability of written materials.

When looking closely at the individual evaluation items, seven out of ten items in the core part of the assessment (Part A) were satisfied by less than 40% of the collected materials. It should be noted that none of the assessed materials satisfied items 4, 9, and 10, and only 35% were clear about the main message. The results were similar to those of Porter and colleagues in their research evaluating written materials related to a behavioral intervention aiming to reduce sugar-sweetened beverage consumption (Porter et al., 2019). Their study showed low scores for the same items (4, 9, and 10). A positive aspect of our findings was that all materials included calls to action (item 5), while only 30% of materials in Porter’s study satisfied this criterion. As for numerical information (Part C), our study showed lower scores than those in Porter’s study (Porter et al., 2019). All materials assessed by Porter and colleagues used numbers appropriate for the primary audience and explained the meaning of those numbers in words (Porter et al., 2019). Among our collected materials, only 16% explained the meaning of numbers in words (Part C), and none explained numeric probabilities in words or with visuals (Part D). Greater use of visual aids (item 4), summarization of main points responding to the audience’s needs (item 9), and explanation (in words or with visuals) of numerical information (items 15 and 19) are recommended for health information provided to Vietnamese diabetes patients. In addition, regarding known and unknown information about a topic (item 10), what is known should be appropriately sourced with references, and what is unknown should also be pointed out and explained to readers.

In practice, however, these recommendations are not easy to achieve. Nearly half of the collected materials in the present study were developed by hospital departments internally, usually without the help of a professional designer or health literacy assessment tools. Additional resource input, such as careful editing by health professionals, is required to produce clear and accurate summaries of the main content and to explain numerical information. To achieve this, the promotion of organizational health literacy at hospitals is necessary as stated in the United States’ Healthy People 2030 (U.S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion, 2023). Similarly, the Vietnamese Ministry of Health issued a circular stating the benefits of health literacy (Vietnamese Ministry of Health, 2022). Our research can serve as a model case for forming a team of local physicians to assess and improve the health information that they use. Such efforts require the understanding and approval of the physicians’ workplaces. Further, studies similar to ours should be conducted at a wider scale to monitor improvements in health information provided to the public in Vietnam.

Of note, none of the materials in our study presented information from authoritative sources about both what was known and what was unknown about the topic (item 10). It is customary in Vietnamese culture for health professionals to only offer information that is known and clear without mentioning any points that require further research and clarification. Related to this, the assessment of Part B showed that all the materials assessed in our study included behavioral recommendations, and yet the vast majority omitted the reasons for the recommendations. Furthermore, in Part D, none of the materials explained the risks and benefits of the recommended behaviors. Such a limited scope in the information provided by health professionals might be linked to patient characteristics and service settings unique to Vietnam. Previous qualitative research in Ho Chi Minh City reported that patients were rather passive about their disease management and simply followed physicians’ instructions given in the short consultation time (Tri et al., 2022). The group recommended interactive patient–doctor communication, empowering patients to control their own health. Another survey in Vietnam reported that quality communication was associated with better service satisfaction (Vuong et al., 2017). Health literacy promotion should not only focus on improving the accessibility of information but also on the participatory development of the information, encouraging the audience to make an informed decision. In the case of dietary instructions in particular, patients need to be motivated to make changes in their daily habits. Improving information provision by and communication with health care providers would lead to better disease management (Ha et al., 2021).

Our study has identified that more attention needs to be paid to evaluating and improving written health information provided to diabetes patients in Vietnam. However, our study has several methodological limitations. First, the number of written materials that were analyzed was limited, and these were collected only at general hospitals in one city. Health information is issued by governments and various sources using a variety of methods, including digital tools. Second, there are multiple indices available to evaluate written materials. For example, a previous study recommends using a combination of CCI and the Suitability Assessment of Materials questionnaire (Nomura et al., 2021). Further research to evaluate more materials disseminated via different methods, using multiple evaluation indices, is recommended.

Conclusion

None of the Vietnamese written information on diabetes that we collected and assessed reached the sufficient CCI level. To improve the clarity and understandability of written health information, greater use of visual aids, a summary of the main points of the content, and explanations of numerical information are recommended. This requires the promotion of organizational health literacy. The first step should be to implement health literacy training for health professionals in Vietnam and to promote the evaluation and improvement of health information they use in daily clinical practice.

Author contributions

Phong Van Lam, Conceived the study question, concept and design, supervision of data collection, data analysis, data interpretation, writing the manuscript, data management, manuscript revision, supervision of study.

Aya Goto, Conceived the study question, concept and design, data analysis, data interpretation, manuscript revision, supervision of study.

Thanh Nhan T. Vuong, Data analysis, data interpretation, writing the manuscript, data management.

Anh Thu Q. Nguyen, Data analysis, data interpretation, writing the manuscript, data management.

Truc Dung Nguyen, Data collection.

Quynh Hoa Vu, Data collection.

Yokokawa Hirohide, Manuscript revision, supervision of study.

IRB approval

This work involved only the collection and analysis of health-related written materials, not in-person collection of data from human subjects. IRB approval was therefore not required.

Acknowledgement

We thank Oliver Stanyon for editing a draft of this manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The dataset generated and analyzed in the present study is available from the corresponding author on reasonable request.

Additional information

Funding

This work was supported in part by JSPS KAKENHI Grant Number 20K10539 (PI: Yokokawa Hirohide) and the Japan International Cooperation Agency Partnership Program “Promoting evidence-based patients-centered health services in southern Vietnam: University & medical association partnership initiative” from Japanese Fiscal Year 2017 to 2020 (PI: Aya Goto).