Health Literacy Measurement: A Proposed Research Agenda

Abstract

Although the field of health literacy is experiencing tremendous growth in terms of producing peer-reviewed journal articles and attracting practitioners, the foundation of that growth is potentially unstable. Despite a steady increase in their number, existing measures and screeners of health literacy are not based on an accepted conceptual framework and fail to align with the growing body of theoretical and applied work. Existing measures are mainly focused on assessing what individuals can read and understand in clinical contexts. This leaves important factors untested, such as how individuals use information, and how health professionals and systems communicate with patients. This article outlines key elements of a proposed research agenda focusing on development of a new, comprehensive approach to measuring health literacy.

Building a comprehensive approach to measurement of the social construct called health literacy may well be the most significant and necessary task facing health literacy research and practice.

In the spring of 2010 and again in the summer of 2011, the Literacy Information and Communication System's (LINCS) Health Literacy Discussion List hosted discussions on evaluation and measurement of health literacy. The participants for this discussion list include over 1,600 members from the United States and at least six other countries. Members include health professionals, adult literacy practitioners, librarians, employees of community-based advocacy organizations, health care delivery organizations, public health entities, and many other disciplines. The list community is recognized by many to be the largest and most comprehensive group of health literacy professionals. The community includes a broad range of perspectives from which to explore the complex and timely issue of measurement of health literacy.

Julie McKinney, Health Literacy Specialist at World Education, Inc., moderated the list, and both health literacy measurement discussions were guest moderated by Andrew Pleasant, Director of Health Literacy and Research at Canyon Ranch Institute.

After the first discussion in 2010, a survey was taken in which over 100 respondents demonstrated a fairly strong consensus on the current understanding of the underlying concept of health literacy and on the key elements that should be included in a new measurement tool. The results of that survey have been published separately (Pleasant & McKinney, 2011) and provide a solid basis for this article. Among the strongest areas of consensus found in that survey were three assertions that support the agenda proposed in this article:

1.	New measures of health literacy need to be based on sound theory.
2.	Researchers and practitioners need to be able to measure both sides of the health literacy equation—the health literacy of individuals and the health literacy of health systems and health professionals.
3.	A measure of health literacy must allow comparison across contexts including culture, life course, population group, and research setting.

During the second discussion, much of that initial consensus was confirmed. The second discussion also fully revealed the lack of satisfaction with current health literacy measures and screeners. Further, as the discussion delved into more complex underpinnings of health literacy, the extent to which the field has not been able to truly identify how people and health professionals find, understand, evaluate, communicate, and use information to make informed decisions—in other words, to use their health literacy—was further revealed. The lack of a broadly shared understanding about the causative structures underpinning the process of using health literacy to make informed decisions and the relationship between health literacy and behavior change was also revealed. This further compels us to suggest that now is the time to advance the field of health literacy by advancing a measurement approach to better apply the scientific process. The summaries and full transcripts of the two online discussions are available at the following links:

• Evaluating and Measuring Health Literacy: March 8–15, 2010

• http://lincs.ed.gov/lincs/discussions/healthliteracy/10Measures

• Evaluating and Measuring Health Literacy—Follow Up: June 13–20, 2011

• http://lincs.ed.gov/lincs/discussions/healthliteracy/11follow

Health literacy, it is broadly agreed upon, represents more than the ability to read health information. In a survey of participants on the electronic mailing list, conducted after the first discussion round, 97% of respondents agreed that we need to be able to measure the health literacy of health systems and health professionals as well as the health literacy of individuals (Pleasant & McKinney, 2011). However, current measures of health literacy test only a narrow range of reading and, occasionally, numeracy skills, and the vast majority of measures focuses on the competency of the individual rather than the provider or system (Agre, Steiglitz, & Milstein, 2006; Nielsen-Bohlman, Panzer, & Kindig, 2004; Rogers, Ratzan, & Payne, 2001; Schillinger & Davis, 2005; Schwartzberg, VanGeest, & Wang, 2005; Simonds, 1974; Zarcadoolas, Pleasant, & Greer, 2005, 2006). The lack of a comprehensive approach to measurement severely limits the ability to comparatively evaluate the growing number of health literacy initiatives in any depth beyond the superficial aspects of what happened (e.g., how many documents were rewritten or physicians trained). In contrast, a comprehensive approach to measurement focuses on causal processes to understand why change did or did not occur at the individual, community, or health care system level.

Existing health literacy measures include various versions of both the Rapid Estimate of Adult Literacy in Medicine (REALM) (Davis et al., 1991; Davis et al., 2006; Davis et al., 1993) and the Test of Functional Health Literacy in Adults (TOFHLA) (Gong et al., 2007; Parker, Baker, Williams, & Nurss, 1995), as well as the Health Activities Literacy Scale (HALS) (Rudd, Kirsch, & Yamamoto, 2004), the Newest Vital Sign (NVS) (Weiss et al., 2005), Wide Range Achievement Test Fourth Edition (Dell, Harrold, & Dell, 2008), Stieglitz Informal Reading Assessment of Cancer Text (SIRACT) (Agre et al., 2006), Medical Achievement Reading Test (MART) (Hanson-Divers, 1997), National Adult Reading Test (NART) (Uttl, 2002), Literacy Assessment for Diabetes (LAD) (Nath, Sylvester, Yasek, & Gunel, 2001), Nutrition Literacy Scale (NLS) (Diamond, 2007), the Short Assessment of Health Literacy for Spanish-speaking Adults (SAHLSA) (Lee, Bender, Ruiz, & Cho, 2006), an instrument targeting Canadian adolescents, a “talking touchscreen” approach, and the Demographic Assessment of Health Literacy (DAHL) (Hanchate, Ash, Gazmararian, Wolf, & Paasche-Orlow, 2008).

A new generation of health literacy assessment tools is appearing and includes items from the 2003 National Assessment of Adult Literacy (Baldi et al., 2009), the Health Literacy Skills Instrument (HLSI) (McCormack et al., 2010), and the Mandarin Health Literacy Scale (MHLS) (Tsai, Lee, Tsai, & Kuo, 2011). However, the health literacy questions and methodology for the 2003 National Assessment of Adult Literacy (NAAL) is not publicly available, and restricted access is difficult to obtain. The Agency for Healthcare Research and Quality (AHRQ) developed a “health literacy item set” for the Consumer Assessment of Healthcare Providers and Systems surveys (AHRQ, 2007). Also, the Joint Commission is embarking on an effort to develop health literacy standards as part of its hospital accreditation process.

A number of specific criticisms of existing health literacy measures are found in the academic literature (Agre et al., 2006; Aguirre, Ebrahim, & Shea, 2005; Baker, 2006; Buchbinder, Hall, & Youd, 2006; Friedman & Hoffman-Goetz, 2006; Johnston, Diab, Kim, & Kirshblum, 2005; Kalichman & Rompa, 2000; Morrow et al., 2006; Weiss et al., 2005; Zarcadoolas et al., 2005; Zun, Sadoun, & Downey, 2006). These include observations that existing measures of health literacy:

	are not designed to test or advance an underpinning theory of health literacy,
	are limited in approach to evaluating skills (e.g., some overly rely on the cloze formatted reading test, while others evaluate only word recognition and not understanding as well),
	lack cultural sensitivity and can exhibit bias toward certain population groups,
	are not directly useful for informing or evaluating health promotion and communication interventions (e.g., a pre-post design), curricula, policy, or schemes to pay physicians based on performance,
	place a problematic burden and potentially harmful label on patients being tested in clinical settings,
	do not evaluate spoken communication skills,
	do not consider health literacy as a public health issue,
	employ ambiguous item wording,
	don't measure an individual's ability to improve health or prevent illness and injury,
	do not adequately distinguish between people at very low and very high levels of health literacy,
	have not been subjected to rigorous psychometric analysis,
	have not been used in a consistent way,
	focus on a single dimension although health literacy involves multiple dimensions,
	may be biased toward those with recent experience with the health care system or content area, and
	use tools that vary and employ them in various ways, which makes it difficult to compare experiences or results across studies in order to definitively establish the relationship of health literacy to that of health status.

Further critiques and analysis of the existing state of the field of health literacy measurement and screening have been reported elsewhere by Pleasant & McKinney (2011). Clearly, no measurement of social constructs will ever be deemed perfect by all potential users—that is the tension that helps propel advancement in all scientific endeavors. The perfect should not be the enemy of the good; however, rigorous development based on an explicit theory, testing, and retesting to assure broad validity, reliability, and the absence of bias are requisite in the development of a new approach to measuring health literacy.

Current health literacy measures clearly made very important contributions to the study of health literacy and should not be downplayed. However, the field has advanced beyond that important beginning. The need for a new comprehensive approach to measuring health literacy becomes more urgent as complex social interventions and collaborative initiatives are steadily being put into place with the goal of improving health literacy skills in order to improve health.

One major change in the current understanding of health literacy is that it involves more than reading and understanding information. Health literacy is increasingly thought to include finding, understanding, evaluating, and communicating information, and using that information to make informed decisions (Coleman et al., 2011; Pleasant & McKinney, 2011). Evidence of this lies both in the number of endorsements to the Calgary Charter on Health Literacy, as well as the reports to the survey referenced earlier, in which over 90 percent of respondents consistently agreed with these key elements of the definition of health literacy initially proposed in the Calgary Charter on Health Literacy.

Another major change is that it is now increasingly accepted that health literacy competencies apply to both the information seeker (e.g., a patient) and the information giver (e.g., the health care provider, system, or disseminator of a public health message). This means that we must develop measures to test both these audiences, but said measures should be based on the same theory or conceptual framework of health literacy. As a result of these developments, new approaches to measuring health literacy need to address a broader array of skills and content areas, and must address the health literacy capacity of both information givers as well as information seekers in some fashion. In this regard, it is worth noting that some have protested the apparent overlap among understandings of health literacy, communication, patient-centered communication, and other related constructs. We suggest this critique is somewhat akin to a “straw man” argument. Consider the vast number of theories of health communication and health behavior. Most, if not all, overlap to some extent; and, instead of creating a negative effect, it is exactly that diversity and partial redundancy that has driven the field forward. Theory development, testing through gathering empirical data via measurement and experiment, and continual refinement are the hallmark of the scientific process.

Advancing Health Literacy: A Research Proposal

Ideally, a new approach to measuring health literacy will be based on rigorous methodology and address the broader array of skills and audiences reflected in the newer definition of health literacy. This approach should be grounded in the experiences of people when they find, understand, evaluate, communicate, and use health information and the experiences of information givers when facilitating these same elements. Currently, some projects have experienced success by assuring that the intervention is grounded in the best practices of health literacy and measuring health outcomes, for example, as a proxy—but that approach limits the ability to learn how health literacy actually works when it is present (Pleasant, 2011). A new approach to measurement clearly has the potential to inform the broader health research agenda, the design and assessment of specific interventions, policy needs, medical school, and health professional curricula, and the performance evaluation of health professionals as well as individuals seeking information. Some argue that researchers have the ethical responsibility to take health literacy in these directions (Freedman et al., 2009; Gazmararian, Curran, Parker, Bernhardt, & DeBuono, 2005). To do so would be a significant undertaking that would likely exceed the capabilities of any individual researcher or research group and would require significant support and collaboration. That research agenda should set out to build a comprehensive approach to measuring health literacy that reflects the following six attributes:

	Explicitly built on a testable theory or conceptual framework of health literacy. Existing measures are neither explicitly built from nor meant to test a model, conceptual framework, or theory of health literacy. Limited approaches to measuring literacy, such as the cloze format and word pronunciation, determine what is being evaluated rather than testing empirically generated theory. Although current health literacy measures are generally reliable (produce consistent results), over-reliance on these methods can cause their true validity (degree to which the test actually measures what it claims to measure) to be questioned. This issue has been compounded over time as new measures are validated against older ones.
	Multidimensional in content and methodology. Emerging theories define health literacy as a construct with multiple domains (Pleasant, 2011). Components of each distinct element of health literacy (e.g., finding, understanding, evaluating, communicating, and using information) should be reflected in a measure of health literacy (Nutbeam, 2000; Streiner & Norman, 1995; Zarcadoolas et al., 2005, 2006). This attribute also means that measures of health literacy should explore each of the core literacy skills—reading, writing, numeracy, speaking, and listening. Given the affordability and ease of use of emerging digital technologies, it is now more feasible, yet still challenging, to include elements of the spoken word into a methodology.
	Measure on a continual, not a categorical basis. There is not sufficient evidence to believe that health literacy is a categorical construct, such as job classification or religious denomination, yet the results of many current health literacy screening and measuring devices are treated in a categorical fashion (Kutner, Greenberg, & Paulsen, 2006; Pleasant, 2008). For example, in the report of the National Assessment of Adult Literacy, health literacy skills were categorized as Below Basic, Basic, Intermediate, and Proficient. A comprehensive approach to measuring health literacy should not assume a distinct case/non-case distinction or limited categories (e.g., below basic, basic). The approach to measurement should address health literacy as a continuous rather than categorical measure. If categorical labels are somehow necessary for individual studies, they can be constructed after the fact but should be based on a consensus process and analysis of outcome data. Results should then be reported both in a categorical and continuous fashion. An explicitly categorical measure can limit analysis and in the past has led to unproductive labeling of people (Pleasant, 2008).
	Treat health literacy as a “latent construct.” Health literacy is a latent construct for measurement purposes because it is not explicit (e.g., one cannot “see” health literacy) and varies across individuals and contexts (DeVillis, 1991). Latent means that the measure, “no matter how precise and elegant, is only an indirect approximation of an unobservable construct” (Atkinson & Lennox, 2006, Para. 5). This is true for many concepts that are regularly measured, such as general intelligence, physical pain, emotional well-being, materialism, depression, or social economic status (Bollen, 2002). A valid and reliable measurement scale can be constructed so that results are caused by health literacy but do not claim to represent all of health literacy (i.e., local independence and measurement equivalence; Raju, Laffitte, & Byrne, 2002). In practice, that means a new approach to measurement should systematically include multiple items drawn from the conceptual domains outlined by the underlying theory or conceptual framework. The domains should, at least initially, be equally represented in the resulting measure or measures until there is evidence to support a different approach. This approach will limit the size of a measure while still ensuring it reflects the range of skills and abilities that make up an individual's health literacy.
	Honor the principle of compatibility. The principle of compatibility is based on a strong body of empirical evidence that demonstrates that for a hypothesized relationship among attitudes, behaviors, and knowledge to hold true, they must be measured at equivalent levels in regard to action, target, context, and time (Azjen & Fishbein, 2005). This means that the wording of questions must be targeted at compatible contexts. For example, basing measurement of health literacy on an ice cream nutritional label does not meet the expectations of the principle of compatibility if the study is investigating the ability to navigate in a clinical setting. A solid understanding of the underlying theory and rigorous pretesting is thus necessary in order to obtain the correct balance between generally worded questions (which may not provide much insight into the reasons for any differences in performance) and specifically worded questions (which may seem less relevant to some). Health literacy is particularly challenging in this regard because it involves skills and abilities that are often applied in very specific health contexts (e.g., for different diseases as well as in clinical versus public health contexts) by very different individuals (e.g., physicians or patients) with, at times, very different goals (consider the wide range of definitions of what it means to be healthy). So whether the content of the measure or measures is appropriate to the context of the use must be considered in developing a new approach to measuring health literacy.
	Allow comparison to be commensurate across contexts including population groups, cultures, life courses, health topics, and research settings. A comprehensive approach to measurement of health literacy should allow comparison across a variety of settings. In practical terms, this implies that the same approach to measurement be useable for different populations, contexts, and disease-related topics. For example, there may be a core evaluation tool plus a variety of add-on modules that target specific issues such as diabetes, asthma, or navigating the health care system (Pleasant & Kuruvilla, 2008). Most important, core tools and add-on modules must be based on the same underlying conceptual approach to health literacy—essentially a family of health literacy measures. Individual research or evaluation projects could use the core evaluation tool and the appropriate add-on modules—thus not having to use the entire measure in all instances.
	Prioritize social research and public health applications versus clinical screening. There is a growing understanding that health literacy is a critically important determinant of public and individual health. Combined with the risk of harm in labeling individuals as “low health literate” in a clinical setting, this argues that resources in clinical settings would be better directed toward lowering barriers to access for all rather than identifying and labeling individuals (Agre et al., 2006; Baker, 2006; Zarcadoolas et al., 2006). A new comprehensive approach to measuring health literacy should address public health contexts and applications rather than simply create another assessment tool for clinical settings. The American Medical Association Foundation recently agreed and recommended that health literacy screening is not appropriate for routine clinical practice but is appropriate for research applications (Elliott, 2008).

Conclusion

In conclusion, building a new comprehensive approach to the measurement of health literacy may well be the next significant and necessary task facing health literacy research and practice. That is the sort of task best conducted with the explicit support of research and practice umbrella organizations such as, to name just a few examples, the World Health Organization, the National Institutes of Health in the United States, Health Canada (Santé Canada), the Agency for Healthcare Research and Quality in the United States, the Eurobarometer effort, and the National Health System in the United Kingdom. Clearly, if an international collaboration is created, this approach to measurement and the resulting tools will need to be tested across languages and cultures. One or several of these institutions could support a consultative research process that invites active and broad participation. Traditional top-down support and coordination is clearly necessary, but such an effort must also be careful to not lose contact with real-world concerns and experiences. The best experts in health literacy are the people who directly experience the barriers and benefits of health literacy. Involving them is perhaps best accomplished by including an explicit mechanism for engaging the public in the research process. Actively engaging the public (e.g., adult learners, patients) and policy makers will help assure validity, relevance, and utility for a new comprehensive approach to measuring health literacy. Broad engagement is also firmly aligned with a theoretical foundation that highlights the importance of literacy and health literacy as tools for empowering individuals and communities (Freire, 1970).

Active dialogue about health literacy across the many involved academic disciplines, combined with broad collaboration with individuals, community-based groups, government agencies, and all who have an interest in the role of health literacy as a determinant of health, will strengthen our collective ability to produce an accurate, valid, reliable, and useful approach to measuring health literacy. When completed, a new comprehensive approach to measuring health literacy will enable more effective efforts to identify needs, to confirm best practices, to build interventions that successfully address the role and effects of health literacy as a determinant of health, and to advance the public's health.

Acknowledgments

The authors wish to acknowledge the participation of all those on the LINCS Health Literacy Discussion List during the two discussions on health literacy measurement. Contributors' insights, critiques, and conversations, as well as their participation in the survey, are invaluable. We are truly grateful. We also wish to thank Dr. Michael Paasche-Orlow and the reviewers for their efforts in bringing together this collection of articles addressing health literacy, and the editorial staff of the Journal of Health Communication for their ongoing efforts to help advance health literacy. Andrew Pleasant thanks Jennifer Cabe, Executive Director of Canyon Ranch Institute, and the entire staff for their support during this effort and their work at all times to catalyze the possibility of optimal health for all people.