Addressing the “Other” Health Literacy Competencies—Knowledge, Dispositions, and Oral/Aural Communication: Development of TALKDOC, an Intervention Assessment Tool

Abstract

Most health literacy assessments evaluate literacy skills including reading, writing; numeracy and interpretation of tables, graphs, diagrams and charts. Some assess understanding of health systems, and the ability to adequately apply one's skills to specific health-related tasks or demands in health situations. However, to achieve functional health literacy, the ability to “obtain, process, and understand basic health information and services needed to make appropriate health decisions,” other health literacy dimensions should be assessed: a person's knowledge and attitudes about a health issue affects his or her ability to and interest in participating in his or her own care. In patient care settings, the abilities to listen, ask questions and check one's understanding are crucial to making appropriate decisions and carrying out instructions. Although literacy is a skill associated with educational attainment and therefore difficult to change in a short time, health education interventions can address health literacy domains such as knowledge, attitudes and oral communication skills. For this reason, an instrument that can assess these constructs is a valuable part of a health educator's toolbox. The authors describe the development and process and outcomes of testing a novel instrument targeted to assess HPV and cervical cancer health literacy competencies, TALKDOC, including its validation with the Health Activities Literacy Scale.

[Supplemental materials are available for this article. Go to the publisher's online edition of Journal of Health Communication for the following free supplemental resource: Appendix: Result of Internal Consistency Analysis: For Each Domain, Remaining Questions, Alpha, Response Options, and Correct Answer (in Bold). This appendix shows the remaining questions (and correct answers in bold) after 32 questions were eliminated.]

Strong evidence points to a link between literacy and health; low literacy is associated with both higher health costs and unfavorable health outcomes (Parker & Ratzan, 2010). Consumers are expected to make decisions with their providers, and to manage their own care. It is unfortunate that more than one third (36%) of U.S. adults have only a basic- or below-basic level of health literacy; up to 9 out of 10 adults cannot understand or use everyday health information effectively (Benjamin, 2010; Ratzan, 2010).

The acknowledged definition of health literacy is the “degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions” (Nielson-Bohlman, Panzer, & Kindig, 2004, p. 4; Ratzan & Parker, 2000). Constructs underlying this definition indicate that a functionally health literate person should have a complex, multidimensional set of dispositions, knowledge and skills that include reading, writing, numeracy (interpreting and analyzing numbers or data—in lab test results, tables, and graphs); listening to get information; oral and visual communication; problem solving and decision making. This person can/does understand health systems and topics, and can successfully handle health-related tasks or demands in health care situations (Dubow, 2004; Nielson-Bohlman et al., 2004; National Network of Libraries of Medicine, 2004; Paasche-Orlow, Wilson, & McCormack, 2010; Ratzan, 2010).

Rudd, Zobel, and colleagues (2004) noted that context influences how well individuals are able to use their health literacy abilities. Because patients need to navigate our increasingly complicated health system, health literacy tasks include knowing and acting on patient rights and responsibilities; completing medical, insurance and informed consent forms—while recognizing the implications for care; clearly describing symptoms, negotiating care, then understanding, recalling and following physician instructions for self-care. Beyond skills, motivation, attitudes, and self-efficacy contribute strongly to whether individuals handle health literacy-related tasks; one must be not only informed, but willing to seek the care they need (Benjamin, 2010; Nutbeam, 2008; Paasche-Orlow & Wolf, 2010; McCormack, Bann, Squiers, Berkman, & Squire, 2010; Rudd & Anderson, 2006).

In addition to context, functional health literacy is not only affected by individual abilities, but is also influenced by environmental factors such as culture and assimilation, race/ethnicity, education level, language facility, the mass media, technology access, the health condition and health care settings (Gee, 1996; Nielson-Bohlman et al., 2004). Being health literate assumes not just that generic literacy skills are applied in a health setting, but that individuals must often have multiple health condition-specific literacy competency. An individual may be health literate about diabetes, but health illiterate about cervical cancer, for example.

Health Literacy Assessment for Intervention Research

The Institute of Medicine (Nielson-Bohlman et al., 2004) called for new health literacy indicators and tools to better assess intervention-specific health literacy outcomes, noting that without a way to measure health literacy competencies, it is difficult to design and evaluate interventions meant to improve those skills. Health promotion/disease prevention programs designed without addressing related health literacy factors may well find their impact undermined.

Multiple assessment tools—whether for research, clinical use or field interventions – have been developed to assess various aspects of health literacy (Chew, Bradley, & Boyko, 2004; Clayman et al., 2010; Davis et al., 1993; Gazmararian, Beditz, Pisano, & Carreón, 2010; Hahn, Choi, Griffith, Yost, & Baker, 2011; McCormack, Bann, Squiers, Berkman, & Squire, 2010; Parker, Baker, Williams, & Nurss, 1995; Wallace, Rogers, Roskos, Holiday, & Weiss, 2006; Weiss et al., 2005). Yet, none of these general health literacy assessments have an exclusive set of indicators to evaluate interventions focused on improving a broad range of health literacy abilities, especially oral communication and listening. Few assessment tools are specific enough to adequately evaluate interventions focused on context- and health-condition specific cultural and conceptual factors that affect health literacy, or which explore the health literacy demands of a particular disease, such as cervical cancer (Helitzer, Hollis, Cotner, & Oestreicher, 2009).

The National Adult Literacy Survey (NALS), although considered the most comprehensive measure of literacy, requires a precise set of data analysis skills and is not available for general use (McCormack et al., 2010). Given this, the Health Activities Literacy Scale (HALS), created from selected NALS and International Adult Literacy Survey items (Rudd, Kirsch, & Yamamoto, 2004), identifies respondents' levels of certain health literacy skills in five areas: health promotion, health protection, disease prevention, health care, and systems navigation.

In each area, the HALS measures how well respondents process tasks (survey items), ranging in degree of difficulty from locating information (simplest Level 1 skill), through cycling, integrating, generating information, to analyzing information, formulating and calculating responses (Level 5). Task difficulty depends on the amount and location of relevant information in the text, length, and complexity of surrounding text, and the level of cognitive analysis needed. HALS evaluates three factors influencing health literacy level: (a) context and content of the information to be grasped; (b) presentation of information (prose, tables, charts, maps); and, (c) cognitive/analytic strategies needed to handle a task. Developed using a two-parameter logistic regression model from item response theory, the HALS provides mean scores correlated with categories of demographic variables: ethnicity/race, gender, education level, country of birth, health status, and income (Rudd, Kirsch, & Yamamoto, 2004). Although the instrument can measure functional health literacy in specific contexts, it has not been used to evaluate health literacy interventions. We theorized that if the HALS were effectively combined with an intervention and/or disease-specific health literacy evaluation tool, we could better assess health literacy skills and specific intervention-related knowledge and attitudes.

Cervical Cancer

Cervical cancer kills approximately 310,000 women annually; most (85%) cervical cancer cases occur in women living in less-developed countries. Though widespread screening with the Pap test has decreased the U.S. cervical cancer mortality rate approximately 75%, at least 12,000 cases of invasive cervical cancer are still diagnosed and 4,200 women still die from the disease annually in this country (American Cancer Society, 2007, 2010).

HPV infection, a common STI, is the prime risk factor for cervical cancer (Waller, McCaffery, Forrest, & Wardle, 2004; Wood, Shin, Duval, & Schmitt, 2006), and continues to be a major threat to women's (and men's) health. Its treatment expenses—economic and medical—cost the U.S. health care system up to $33,000 per person per year.

Extensive HPV vaccination coverage with either Gardasil or Cervarex, combined with effective health education and regular Pap test screening, could reduce the high rates of cervical cancer and its high treatment costs. Several factors—many related to health literacy— may impede achievement of this goal. Both vaccines are expensive, require a three-dose administration at different, specifically set times, and specify a young recommended age (11–16 years) for vaccination. In 2010, 32% of female adolescents 13–17 years of age had received all three doses (Centers for Disease Control and Prevention, 2011). Low utilization and new recommendations about the HPV vaccine for boys increase the need for HPV vaccine-specific health education interventions.

Parents/guardians perceptions of the HPV vaccine are critical for gaining consent to administer the vaccine to minor children. Studies examining parental and young adults' views indicate that prior experience with abnormal Pap test results and/or HPV infection are associated with greater knowledge of HPV; however, misunderstandings are widespread (even about the Pap test); and parent knowledge of HPV does not necessarily encourage vaccine acceptability. Major reasons for rejecting the vaccine include feeling their children were not at risk for HPV and having fears that it might cause children to begin early sexual activity. In addition, parents with lower education levels and/or those receiving public health services were more favorable toward vaccination (Black, Zimet, Short, Sturm, & Rosenthal, 2009).

When deciding whether to vaccinate their children, parents and guardians must apply health literacy skills to obtain trustworthy information, analyze or reject noncredible data, and possibly overcome certain attitudes or perceptions. Decision makers should be positively disposed toward vaccination, ready to implement action, and confident enough to discuss it with their physician. To assess these health literacy–related knowledge, dispositional, and oral/aural communication factors, this study designed, tested, and empirically validated a web-based assessment tool called TALKDOC, which intended to measure disease (HPV and cervical cancer)-specific health literacy.

Method

Instrument Development

All research activities and the TALKDOC assessment instrument, developed in phases, were approved by the University of New Mexico Health Sciences Center's Human Research Review Committee. In the formative stage, we identified the basic sets of knowledge and skills that an individual should have to understand, prevent, and screen for cervical cancer, to participate in treatment, and handle health/health system procedures to be considered health literate in cervical cancer. We defined the cervical cancer health literacy knowledge and tasks, engaged patients/community members in formative work, recorded patient/provider encounters, and consulted with an expert advisory board.

First, we outlined the cervical cancer prevention–related tasks/behaviors that an individual would have to undertake; these included identifying knowledge and skills needed to accomplish core tasks. Some tasks, such as making appointments for Pap screening—a health system navigation action—requires general health literacy skills (e.g., making any medical appointment). The team then refined the key attitudes, beliefs, and/or dispositions, such as self-efficacy and social norms supportive of vaccination, that an individual needs to undertake essential cervical cancer prevention and treatment tasks.

Second, we conducted formative interviews with five racially/ethnically diverse women with a past history of cervical dysplasia to gain their insight into the cervical cancer experience, and of how emotional, cultural, racial/ethnic, and contextual factors influence cervical cancer health literacy. We asked them to reinforce or negate the list of key tasks we identified, and we elicited factors that inhibited or promoted decisions and actions. We learned that Pap tests are painful for some women (a task-related physical and/or emotional aversion that must be overcome); that an empathetic physician can help in communication; and that concern about a family history of one disease, such as diabetes, may lead to neglecting prevention actions related to other diseases, such as taking regular Pap tests to prevent cervical cancer.

Third, with the informed consent of doctors and patients, we audiotaped five provider–patient discussions about Pap tests and STIs, which took place during gynecological visits. Five different providers agreed to be taped, and we asked for the consent of one patient from each provider, attempting to include patients from different age groups. Review of the tapes provided cues to the language used in these conversations and emotions expressed. Follow-up discussions with providers and patients provided greater detail about the dynamics of such health care talks, and the transcripts served as a basis for developing our instrument's oral/aural health literacy section.

Last, to structure and refine a core, underlying set of HPV/cervical cancer health literacy constructs/competencies for generating items, and to address instrument content validity, we convened an advisory board composed of medical experts, researchers and health care providers. Members included a microbiologist, adolescent medicine and STI specialists, and cervical cancer prevention researchers.

The advisory board first considered and categorized the major constructs or competencies needed to obtain, process, and understand basic HPV and cervical cancer prevention information into three domains—disease-specific knowledge, dispositions (attitudes and existential competencies such as self-efficacy), and communication skills. We used a modified Delphi technique (Cline, 2000) to prioritize the domain concepts to serve as the basis for generating assessment tool items.

The advisory board recommended that (a) the knowledge domain should include measurement of individual's knowledge about HPV transmission, infections, and vaccine; women's anatomy; cervical cancer risks and causes; Pap test screening and results; and safe sex and other forms of cervical cancer prevention; and understanding of how to navigate the health system; (b) dispositions should address self-efficacy and attitudes to prevention; and (c) communication skills should cover asking questions and listening abilities. On the basis of this input, our team agreed upon eight core informational categories to be addressed in each domain. These included definitions and concepts; causes and risk factors; information-seeking capacity; prevention and protection factors; secondary prevention/disease screening; treatment; cultural or emotional factors; and health system navigation.

Advisory board members suggested topics to be covered in the draft instrument. We prioritized essential constructs to measure and created a set of proposed questionnaire/instrument items, which were subjected to a second Delphi process.

The Nomological Network

Using the formative research findings, the research team created a nomological network (or chart; see Figure 1) that indicated the entire set of constructs (competencies) and categories—and their relationships—in all three health literacy domains to be addressed in the final instrument. Conceptualized by Cronbach and Meehl (1955) as a way of providing evidence for construct validity, a nomological network embodies the conceptual framework underpinning design of the assessment tool. It gives each factor or construct meaning, distinguishes each factor from the others, and demonstrates how they are related. A nomological network must contain at least two empirical constructs to be measured, theoretical propositions specifying linkages among constructs, and rules to operationalize measurements.

Figure 1 Nomological chart indicating constructs, categories, and their relationships in TALKDOC health literacy domains. (Color figure available online.).

Item Generation

The TALKDOC health literacy measurement tool (designed from the aforementioned process) consists of three interconnected categorical domains of indicators, measured through scaled or multiple-choice item responses that examine (a) context-specific health literacy knowledge; (b) dispositions of self-efficacy and prevention; and (c) communication/comprehension abilities.

We developed four scripts for the listening and oral communication section of TALKDOC using the knowledge gained from our original formative interviews and the review of transcripts of real-life physician–patient interactions. The scripts were acted out by the University of New Mexico actors trained to be standardized patients for medical students; all scenarios were professionally audiotaped. One script modeled a woman making a Pap test appointment (Tape A); the second, third, and fourth scripts presented different aspects of patient–provider clinical interactions—dealing with a genital wart diagnosis (Tape B), going through a routine Pap test (Tape C), and discussing an abnormal Pap test result (Tape D).

For each audio script, the research team prepared and cognitively tested a brief set of multiple-choice questions to test listening recall and understanding of the interaction being enacted. Thus, during the actual administration of this fifth component, unlike the other sections in which participants entered their own responses, for the oral/aural competencies, the data entry was completed by the researcher/interviewer. The interviewer would play the audiotaped scenarios (following along with the script text on the computer screen, which the participant could not see) and stop the tape at specifically indicated break points. Each research interviewer would read aloud the standard multiple-choice questions from the computer screen to each participant, as well as the four response options. The interviewer, upon request of the participant, would repeat the questions and potential responses as needed; however, the tape was never replayed. Once each participant stated aloud their response choice (i.e., A, B, C, or D), the interviewer would enter that response into the computer. Each interviewer thus orally asked each respondent the same nine multiple-choice questions prepared for each of the four scenarios. Multiple-choice answers were quantitative in nature; only one response per item was correct.

Some questions examined comprehension (“What was going on in this conversation?”). Others confirmed knowledge (“What is a pelvic exam?”), some checked recall of orally/aurally received information, and others determined whether a respondent could identify the most appropriate question a patient should ask at particular points in a conversation (“What question is Sandra most likely to ask at the end of this discussion?”). Additional questions such as “Not related to the tape, have you, personally, ever asked to have a medical appointment rescheduled?” and “Not related to the tape, have you, personally, ever asked a health care provider to explain a lab result to you?” explored respondents' own behaviors; responses used a 4-point Likert scale ranging from 1 (never) to 4 (always).

Our first instrument draft included more than 300 items. We conducted cognitive tests of the instrument (Willis, 2004) with five women, who reviewed the draft questionnaire, listened to the audiotaped scripts and responded to a series of think-aloud questions by an interviewer. Their responses elicited their cognitive analytical process in answering survey questions. The cognitive testing helped us reduce the number of items in the TALKDOC to 109 (see Table 1 for the topic areas and the numbers of questions included in each section).

Table 1. Item content areas and numbers of question per domain of instrument that was tested

Domain	Subcategories	Number of items
Demographics	Year of birth, highest level of school completed, marital status, language, ethnicity, location of birth, household income, employment	11 items
Experience and exposure	Cancer and Pap test experienceExposure to and trust in health information sources	18 items
Knowledge	Definitions, concepts, risk, prevention, protection, treatment, navigation	27 items
Disposition: Confidence	Self-confidence	5 items
Disposition: Attitudes to prevention	Importance and value of Pap tests, value of HPV vaccine	7 items
Oral communication	Comprehension and understanding, knowledge, and question asking	Tape A: Making a clinic Appointment: 10 itemsTape B: STD clinic visit: 9 items TapeC: Pap test visit: 12 itemsTape D: Abnormal Pap test visit: 10 items

We used the HALS to assess convergent validity. The HALS is a computerized, web-based assessment. To assure that computer literacy factors were not biasing our results, we created the TALKDOC in web-based format using the University of New Mexico–approved Opinio software (Object Planet, 1998–2011). In the testing process, participants followed the sequence of self-administering the HALS first, then TALKDOC; interviewers orally asked the communication section questions and entered the responses into the computer.

Participant Recruitment

We used different recruitment methods to identify female participants from various educational levels. We also wanted to oversample for the less-than-basic-literacy group (the U.S. Census indicates that one third of New Mexicans have a high school education or less). Our goal was to recruit 35% of women with less than an 8th-grade education, 35% with an 8th–12th-grade (including GED) level, and 30% who had achieved some college or more. We attended community health fairs; handed out study brochures; placed ads in the local newspaper and on craigslist.org; posted flyers in community sites such as women's health clinics (especially those serving low-income, racially diverse women), laundromats, beauty parlors, and nail salons; and the University of New Mexico hospital. In addition, we contacted English as a second language and adult literacy programs, encouraging women in the sessions to take part. Our brochures and flyers, written at a fourth-grade reading level, described the study and participant eligibility (ages 18–70 years) and gave contact information. Participant compensation ($25 gift cards) was offered, and those recruited were encouraged to tell a friend about the study. We also used a snowball sampling process—a raffle—to increase the number of recruited participants; women who referred other participants became eligible to have their name entered into a drawing for a $100 gift card.

Screening and Data Collection

We screened potential participants for education level, age, race/ethnicity, computer familiarity, and English language capability. Exclusion criteria included being younger than 18 years of age, having no computer skills, and not being conversant in English. Education and ethnicity data were used to create the sampling frame.

Participants provided demographic information (age, race/ethnicity, place of birth, education, employment, income, partnership status, cancer exposure, use of health information sources, and Pap test experience) and answered cancer related questions adapted from the National Cancer Institute's Health Information National Trend Survey (hints.cancer.gov; 2007 version). A research assistant helped respondents, as needed, navigate through the both computerized assessments. Four of the five TALKDOC sections were self-administered by the respondent, while the fifth section, oral/aural communication, required the research assistant to play the four recordings. The research assistant stopped the audiotapes at specific points, asked the respondent the relevant questions, and entered the responses into the computer. The total amount of time that was required for an individual to complete the assessment varied depending on several factors—their comfort with the material, the computer, and their level of education. This was especially true with the HALS, because the items become more difficult to answer as the participants progressed. Most participants did not take more than 30 to 35 minutes to complete the self-administered part of the TALKDOC; the oral/aural component (the tape was never replayed, but questions could be repeated) took anywhere from 28 to 40 minutes to complete (e.g., 7–10 minutes per tape), for a total of 60–75 minutes.

Analysis of Reliability and Validity

We used Cronbach's alpha as a measure of internal consistency and reliability as well as to eliminate questions from the analysis. Convergent validity between HALS test scores and TALKDOC knowledge, communication, and disposition questions was assessed with Spearman's correlations.

We combined the knowledge, communication, and disposition questions remaining after the analysis of internal consistency into four subscores. The knowledge subscore is the percentage answered correctly on 17 remaining knowledge questions; the communication subscore is the percentage answered correctly on 15 remaining communication questions.

The disposition self-confidence subscore is the average of responses to four questions reported on a 6-point scale ranging from 1 (not at all confident) to 6 (totally confident). The disposition attitudes to prevention subscore is the average of responses to two questions with responses 1, strongly disagree, to 6, strongly agree. For both disposition subscores, the average was stratified calculated into two groups, those whose average was less than 5 and those whose average was 5 or greater. We used cross-tabulations, Mantel-Haenszel chi-square and t tests to assess relations and significance of association between the scores and demographic experience and Pap test experience, exposure to information, and trust in information sources.

Results

Participant Characteristics

A total of 161 women participated in testing the TALKDOC (see Table 2 for the demographic characteristics of the sample). Female participants ranged in age between 18 and 60 years. The ethnic and racial distribution of the sample was similar to that of the state of New Mexico. Most women were born in the United States. The level of educational attainment was higher than hoped for; about one third of the sample had graduated high school or passed a GED; the remainder attended some sort of college, technical, or vocational school; and one third had graduated college or attained a graduate degree. Most respondents were employed for wages, although the median annual income was less than $35,000. Less than half of the sample reported being married or living with a partner.

Table 2. Participant demographics

Category	Variable	n	%
Year of birth	1946–19551956–19711973–1989	445463	263439
Ethnicity (distribution similar to that of New Mexico population)	American IndianHispanicNon-Hispanic WhiteMixed race, Asian, or African American	16518016	10324810
Location of birth	United StatesElsewhere	1509	94.35.7
Highest level of school completed	Grades 6–9High school diploma or GEDVocational or trade schoolSome collegeCollege degree (BA or BS)Graduate degree	9358523720	5.621.75.032.323.012.5
Employment	Employed for wages or self-employedOut of work, unable to work or retiredA homemaker or student, not working	1131929	70.511.918
Household income	$0–$34,999$35,000–$59,999$60,000–$99,000$100,000 or more	79342713	51.722.217.68.3
Marital status	Married or living with partnerNever marriedDivorced, widowed, or separated	744838	46.33023.8

Table 3 shows the participants' cancer screening experience and access to and use of health information sources. Almost all respondents had had at least one Pap test, and about two thirds had their last Pap test within the last year. More than half (52%) of those having had a Pap test had had an abnormal Pap test result. Most women reported having their last Pap test as a routine test. Motivation for getting their last Pap test included responding to a health system reminder, having had a prior abnormal test, and/or a recommendation from a doctor or family member. Although half of the sample said they would definitely have a Pap test within the next year, one quarter said they were only somewhat likely or unlikely to have one. More than half of the women had someone in their close family with cancer, including themselves. Most of the respondents reported getting information from the health section of a newspaper or magazine, the Internet, or TV news, or other sources. In addition, 126 (79%) had seen advertisements for the HPV vaccine, and 78 (62%) of these asked a health provider about the vaccine.

Table 3. Descriptive statistics

Question	Response category	n	%
Have you ever had a Pap test?	Yes	150	94.3
Have you ever had an abnormal Pap test result?	Yes	79	52
When did you have your last Pap test?	≤1 year ago	101	66.4
	1–2 years ago	24	15.8
	>2 year ago	20	13.2
	Do not know	3	2.0
	No Pap needed	4	2.6
Why did you have the last Pap test?	Routine test	125	89.3
	Last Pap not normal	13	11.0
	Notice sent to me	13	11.0
	Never had one before	5	4.4
	Health care provider told me to have one	46	38.3
	Female family member told me to have one	9	8.0
	Other (various*)	29	33.0
Do you plan to have a Pap test within the next year?	Unlikely	18	11.4
	Somewhat likely	19	12.0
	Very likely	34	21.5
	Definitely	87	55.1
Have you been told by a health provider that you have cancer?		12	7.5
Have you been told by a health provider that you have cervical cancer?		8	5.0
Has someone in close family been told by a health provider that they have cancer?		93	58
Respondent reads the health section of a newspaper or magazine		133	8
Respondent watches health reports on TV news		133	84
Respondent reads health information on the internet		122	76
Respondent has seen ads about the HPV vaccine		126	79
The ad made them ask their health provider about the vaccine		78	61.9
Where have the respondents seen the ad?	Radio	22	17
	TV	108	86
	Magazine	54	43
	Newspaper	30	24
	Internet	29	23
	Other	17	13

Internal Consistency

We aimed to reduce the number of items in the TALKDOC. Tests of internal consistency indicated that 32 questions could be eliminated.

Tests of association showed that education was significantly related (p < .001) to the percentage of knowledge questions answered correctly. Respondents with a high school education or less answered 57% (SD = 17) of the knowledge questions correctly, whereas those respondents with some college education answered 73% (SD = 11) of the knowledge questions correctly. Education was also significantly related (p < .001) to the percentage of oral communication questions answered correctly. Respondents with a high school education or less answered 77% (SD = 17) of the oral communication questions correctly, while respondents with some college education answered 89% (SD = 10) of the oral communication questions correctly.

No significant differences found between either the disposition self-confidence or the attitudes to prevention average scores by education (see Table 4) for the means and medians for the self-confidence and the attitudes to prevention subscores). There were no significant differences in knowledge, disposition, or oral communication scores by exposure to media, influence of information sources, having had a Pap test, or having had abnormal pap tests.

Table 4. Descriptive statistics for average response of and questions for disposition scores (self-confidence and attitudes to prevention)

Variable	n	M (SD)	Median (range)
Self-confidence	155	5.4 (0.8)	5.75 (2.75, 6)
• How confident are you that you can ask a health care provider questions about HPV or cervical cancer?	154	5.6 (0.9)	6.0 (1, 6)
• How confident are you that you can get a health care provider to take your concerns seriously?	154	5.3 (1.1)	6.0 (1, 6)
• How confident are you that you can understand what a health care provider tells you?	155	5.2 (1.0)	6.0 (1, 6)
• How confident are you that you are able to ask a health care provider to be more clear if you do not understand what he or she says?	148	5.6 (0.9)	6.0 (1, 6)
Attitudes to prevention	155	5.8 (0.7)	6.0 (1, 6)
• A Pap test is important for a woman my age.	154	5.7 (0. 8)	6.0 (1, 6)
• A Pap test is needed even if there is no family history of cancer.	154	5.8 (0.8)	6.0 (1, 6)

Convergent Validity

The HALS scores are divided into five levels of proficiency. Table 5 shows the definition of each of the five levels and the distribution of HALS test scores of the study respondents. Approximately one third of the respondents had a score that was lower than would be expected of those graduating from high school; one third had minimum-level proficiency; and the remaining third had high proficiency.

Table 5. Scores on Health Activities Literacy Scale (N = 150)

Level	Score range	Definition	n	%
1	155–225	Possess very limited to restricted literacy proficiencies	9	6
2	230–275	Below average proficiency of those graduating from high school	34	23
3	280–325	Can integrate information from relatively long/dense text or documents; minimum level literacy	59	39
4	330–375	High proficiency associated with challenging tasks in test	39	26
5	380–435	Highest proficiency associated with challenging tasks in test	9	6

Spearman's correlations between HALS test scores and knowledge were .51 (p < .01), and oral communication = .41 (p < .01), disposition: self-confidence = .09 (p = .24), and disposition: attitudes to prevention = –0.16 (p = .05).

Discussion

The study results show that the TALKDOC is a health literacy tool that can be used to measure constructs of knowledge, attitudes, and oral communication that are not currently addressed in other assessment tools but are necessary for functional health literacy. This type of health literacy assessment tool can be used to evaluate the effect, efficacy, and effectiveness of an intervention that addresses knowledge, dispositions, and/or oral communication.

The convergent validity assessment, comparing the TALKDOC with HALS, showed moderate correlations. This suggests that the TALKDOC is not redundant with HALS and that knowledge, dispositions, and oral communication are valid health literacy constructs to assess. Interventions that address these constructs can use the TALKDOC model with appropriate or needed revisions for health content.

The results of this study the importance of considering other constructs of health literacy in intervention assessment tools. Tests of association showed a relation between education levels and scores on our health literacy knowledge and oral communication tests. Literacy itself has been shown to be correlated with education; and literacy levels can be improved with training and education (Parker & Kreps 2005). Health literacy—especially certain constructs such as knowledge and oral communication skills—can be integrated into focused interventions. Taken together, the findings of this study suggest that the TALKDOC is an instrument suitable for measuring change from such an intervention. In this first test of the instrument, there was a sufficient knowledge gap so that improvement could be detected after an intervention. For oral/aural communication competence, there were ceiling effects, and we saw no change over time in the variable.

Additional research is needed to assess how conceptually different aspects of health literacy contribute to the overall construct of health literacy. In this study, we created and tested a tool that measures other indicators of functional health literacy—knowledge of a specific health condition or disease, oral/aural communication skills needed to learn about, negotiate treatment and follow directions to handle the health condition, and the attitudes or dispositions that facilitate or hinder health decision making and action. The study findings suggest that there is value in enhancing existing health literacy measures with those that measure different constructs of health literacy, such as those in TALKDOC.

Implications for Intervention Research

Health literacy is broader than literacy about health. The definition of literacy is “the ability to use texts to retrieve information” (White & McCloskey, Forthcoming). The definition of health literacy requires individuals to use information to make appropriate health decisions. Other domains of health literacy are important in real-life settings, such as knowledge of the subject matter, attitudes toward prevention, treatment, self-efficacy about asking questions or negotiating complex health systems, or filling out forms, and the ability to listen and pose questions to get more information in a dialogue about a health topic. These areas and skills should be the focus of health education interventions.

The research reported here is the first step to understanding the opportunities for measuring other health literacy competencies in addition to literacy. Findings suggest that this model instrument can be used to evaluate disease-specific interventions that aim to improve relevant constructs of health literacy, those which influence individuals' ability to make appropriate decisions about their health and health care.

Limitations

The most challenging aspect to testing the TALKDOC was participant recruitment. Our goal was to achieve a sample that mirrored the population of New Mexico, which includes a significant proportion of individuals who have below-basic literacy. However, these individuals are notoriously difficult to recruit. We tried to address common barriers by administering the instrument in evenings and on weekends, recruiting in places where these women might congregate, and inviting women to bring their friends. In addition, the total length of time needed to complete the consent process; demographic questionnaire; the HALS and the TALKDOC required from 40 minutes up to 2 hours. It was tiring for those women who required more time to finish; their responses may have reflected fatigue. Despite the length of time needed, most participants remained interested, engaged, and committed to completing the survey.

Acknowledgments

The work described in this manuscript was generously supported by the Eunice Kennedy Shriver National Institute of Child Health and Development (1 RO3 HD050402-01). We wish to thank the staff and student interns who participated in developing and implementing this assessment tool; the members of the Advisory Board who provided the content and ensured the scientific validity of the information in the instrument; and the members of the community who participated in the testing of the instrument. Also, the authors' sincere thanks go to Irwin Kirsch of the Center for Global Assessment Educational Testing Service, Princeton, NJ and to Rima Rudd of the Department of Society, Human Development and Health at Harvard University School of Public Health, Boston MA for their guidance, consultation and moral support and, in particular, for providing access to the Health Activities Literacy Scale.

Notes

*For example, I have had abnormal Pap test results in the past; it was time and I asked for it; school physical; to get birth control pills renewed; heavy bleeding and painful cramps; new sex partners.