Adapting Online Patient Decision Aids: Effects of Modality and Narration Style on Patients’ Satisfaction, Information Recall and Informed Decision Making

Abstract

Audiovisual and narrative information are often used in online decision aids. However, few studies have tested whether these strategies are more effective compared to other types of information. We tested the effect of these strategies on satisfaction with the information, recall and informed decision-making in a 2 (Modality: audiovisual vs. textual) x 2 (Narration style: narrative vs. factual) experimental design. Data was collected in an online experiment among 262 analogue cancer patients. Since most cancer patients are older people, we also assessed if the effectiveness of these strategies differs depending on the patient’s age. Data was analyzed using Structural Equation Modeling. Findings showed audiovisual modality had a positive effect on satisfaction. Moreover, audiovisual modality improved recall, both directly and indirectly via satisfaction, which subsequently resulted in better-informed decision-making. Narratives resulted in more satisfaction, but not better recall or informed decision-making. These effects were found in patients of all ages.

Patients are increasingly involved in their treatment decisions (Hawley & Jagsi, 2015). Making an informed decision, often defined as “a decision where a reasoned choice is made by a reasonable individual using relevant information about the advantages and disadvantages of all the possible courses of action, in accord with the individual’s beliefs” (Bekker et al., 1999, p. 1), is known to be difficult for patients. In order to make an informed decision about treatment options, patients need to gather and process a lot of complex information about possible treatments, and the benefits and harms of those treatments (Bekker et al., 1999; Gaissmaier & Gigerenzer, 2008; Zipkin et al., 2014). Therefore, supporting them in this process is often seen as warranted (Stacey et al., 2017).

Decision aids are generally online instruments to help people participate in medical decisions by weighing benefits and harms of different treatment options (Stacey et al., 2017). Decision aids have shown to be successful in improving knowledge, and informed decision-making of cancer patients (McAlpine, Lewis, Trevena, & Stacey, 2018; O’Brien et al., 2009; Stacey et al., 2017). However, there is also evidence that patients immediately forget large amounts of the medical information provided to them (Bol et al., 2014; Van Weert et al., 2011). In addition, it has been found that patients often do not experience online medical information as comprehensible, and online medical information is generally hard for the average patient to read (Daraz et al., 2018; McInnes & Haglund, 2011; Meppelink, van Weert, Brosius, & Smit, 2017). According to a review, most studies testing the effectiveness of decision aids do not report on readability (Stacey et al., 2017). However, based on other types of online medical information it seems unlikely that information in decision aids is more readable. Two strategies might be particularly useful to improve information processing and outcomes such as satisfaction, recall of information and informed decision-making: 1) using audiovisual information (instead of text) and 2) using narrative information (instead of factual information).

Even though audiovisual information is regularly used in decision aids (Brenner et al., 20162018; Dharmarajan et al., 2019; Manne et al., 2016), a review study shows inconclusive results regarding the effects of audiovisual information on healthcare decision-making outcomes such as treatment decisions (Winston, Grendarova, & Rabi, 2018). Therefore, it remains difficult to conclude whether audiovisual information is more effective than textual information in decision aids. Thus, the first aim of this study is to test the effects of information provision mode (audiovisual information vs. textual) in a patient decision aid on healthcare decision-making.

A reason for these inconclusive results could be that the effect of audiovisual information on health decision-making outcomes is mediated by the level of satisfaction with the information and subsequently, the amount of information that is recalled. Previous research showed satisfaction positively affects information recall (Bol et al., 2013, 2014; Park & Lim, 2007) and information recall is in turn an important factor in making informed healthcare decisions (Gaston & Mitchell, 2005). Few studies have tested whether audiovisual information in decision aids influences affective and cognitive outcomes such as satisfaction and information recall (Winston et al., 2018). However, studies on general online health information have shown positive effects of using audiovisual information on affective outcomes such as satisfaction with the information (Bol et al., 2013; Bol, Van Weert, de Haes, Loos, & Smets, 2015; Meppelink, van Weert, Haven, & Smit, 2015) and preference for the information (Laszewski et al., 2016), but also on cognitive outcomes such as information recall (Bol et al., 2013, 2015; Meppelink et al., 2015). Therefore, the second aim of this study is to test if the effect of information provision mode (audiovisual information vs. textual) on healthcare decision-making is mediated by satisfaction with the information and subsequently information recall.

Regardless of the mode used to present the information, information in decision aids is mostly presented in a factual style, for example, lists of numerical estimates (Trevena et al., 2013). When providing online health information in general, conversational narratives have shown to be effective in increasing patient satisfaction (Bol et al., 2013). On the one hand, researchers argue that using conversational narratives (rather than factual information) might also be effective in decision aids in terms of healthcare decision-making outcomes, such as decreasing decisional conflict (Volk et al., 2008). On the other hand, there are reasons to be critical regarding the incorporation of narrative information in decision aids (Schaffer et al., 2018;Schaffer &Zikmund-Fischer, 201). Empirical studies testing this strategy in decision aids are scarce and results have been mixed (Bekker et al., 2013; Dillard, Fagerlin, Dal Cin, Zikmund-Fisher, & Ubel, 2010). Therefore, the third aim of this study is to test the effectiveness of including conversational narrative information in a patient decision aid in terms of healthcare decision-making.

Since narrative information can lead to more satisfaction (Kreuter et. al., 2010; Mayer, 2002) and thus, indirectly to better information recall (Bol et al., 2013, 2014; Park & Lim, 2007), the fourth aim of this study is to test if the effect of narrativity (narrative vs. factual) on healthcare decision-making is mediated by satisfaction with the information and subsequently information recall.

When treating cancer, several options can be considered such as surgery, radiotherapy, chemotherapy or immunotherapy (Vansteenkiste et al., 2013). Actively involving patients and recognizing their preference has become more and more important in medical decision-making (Hawley & Jagsi, 2015; Levit, Balogh, Nass, & Ganz, 2013; Walker, Hipel, & Inohara, 2009). Therefore, in this study we focus on the effectiveness of audiovisual and narrative information in a decision aid aimed at cancer patients.

Cancer often occurs among older individuals (American Cancer Society, 2017; Dutch Cancer Society, 2011; Surveillance, Epidemiology, and End Results (SEER) Cancer Statistics Review 18, 2016), and these older patients are also expected to actively engage in treatment decisions. However, older patients can experience age-related declines in working memory and processing capacity (Bruine de Bruin, Parker, & Fischhoff, 2012; Finucane & Gullion, 2010; Mata, Schooler, & Rieskamp, 2007; Mata, vonHelversen, Karlsson, & Cüpper, 2012; Mata, vonHelversen, & Rieskamp, 2010), making it even harder to make decisions about their treatment. Thus, it is important to test whether the specific information provision strategies described above are also beneficial for older patients.

Theoretical Background

Outcomes of Information Strategies in Online Decision Aids

The effectiveness of audiovisual and narrative information will be tested based on three measures. The primary outcome measure in this study is informed decision-making. In addition, satisfaction with the information and information recall are taken into account as potential mediators.

Satisfaction with information can be seen as an important motivator for receivers to process information (Park & Lim, 2007). According to the elaboration likelihood model, receivers who are more motivated to process information, think more extensively about the issue (Petty & Cacioppo, 1986). Since the amount of cognitive effort put in to process information is an important factor for learning (Stanovich, 1999) and is linked to information recall (Cacioppo, Petty & Morris, 1983), we expect that patients who are more satisfied with the information will recall the information better at a later moment in time. In fact, previous studies have linked satisfaction with online health information to better information recall (Bol et al., 2013, 2014; Park & Lim, 2007).

In turn, correct recall of information is seen as an important prerequisite for informed decision-making (Gaston & Mitchell, 2005). In decision-making processes, obtaining information (Gaston & Mitchell, 2005) and later on correctly recalling that acquired information (Bopp & Verhaeghen, 2005), play an important role (Marteau, Dormandy, & Michie, 2001; McGuire, 1996), since patients may use this information to base their decision on. Therefore, it is expected that increasing patients’ satisfaction with the information provided in a decision aid can positively influence recall of that information. Subsequently, better information recall can positively influence informed decision-making.

Strategies to Provide Information in Decision Aids

Incorporating audiovisual and narrative information in decision aids is expected to positively influence information processing. As mentioned in the elaboration likelihood model one can follow two routes of information processing: a peripheral route and a central route (Petty & Cacioppo, 1983). The first route involves affective and associative thinking processes that ask for minimal cognitive effort resulting in a relatively temporary and susceptible attitude, whereas the central route asks for more deliberate thinking about the issue, asking for more cognitive capacity, which usually results in more long-term attitude formation that is predictive of future behavior (Petty & Cacioppo, 1983).

Regarding medical decision-making, it is unclear whether one of these information processing routes will result in better-informed decisions than the other. On the one hand, both modes can be considered important for decision-making, and some argue that high-quality decisions often occur when information is processed both affectively and cognitively (Peters, Diefenbach, Hess, & Västfjäll, 2008). On the other hand, especially central information processing leads to stable attitudes that are in line with future behavior and decisions (Petty & Cacioppo, 1983). To engage in central information processing, an individual not only needs to be motivated, but also able to process the information extensively which demands more cognitive effort and energy (Chen, Duckworth, & Chaiken, 1999; Stanovich & West, 2000). However, the human working memory has limited capacity to process and store information (Mayer, 2005, 2002; Sweller, 2011), which can be an inhibiting factor for processing information via the central route (Petty & Cacioppo, 1983). Audiovisual information and narrative information could be strategies to expand the cognitive capacity and enhance central information processing, without placing too much burden on the working memory.

Audiovisual Information

According to the Cognitive Theory of Multimedia Learning (CTML; Mayer, 2005), people process textual and audiovisual information through separate sensory systems, both of which are limited in their capacity to process information. This limited capacity can be expanded by using information that addresses more than one sensory system when processing information (i.e., hearing, seeing, feeling); This is called the modality effect (Sweller, 2011). When information is presented in audiovisual format, the information will be divided over the auditory and visual sensory systems instead of overloading one processing channel (Mayer, 2002; Sparks, Chang, & Chung, 2013).

Previous studies have shown that this “modality effect” of presenting audiovisual information increased satisfaction with the information (Dunn, Steginga, Rose, Scott, & Allison, 2004), also in online health communication (Bol et al., 2013, 2015). Patients experienced more positive feelings to audiovisual information compared to textual information (Bol et al., 2013). These positive feelings can in turn result in more motivation to process the information (Bekker et al., 2013) and therefore better recall (Petty & Cacioppo, 1986). Besides, since audiovisual information is processed via multiple sensory systems, information can be processed more easily (Mayer, 2002; Sparks et al., 2013), which is why we expect that audiovisual information will also directly affect information recall. In line with this expectation, previous studies have found that audiovisual information leads to more information recall compared to textual information (Bol et al., 2013, 2015; Meppelink et al., 2015). Therefore, we expect similar effects regarding satisfaction and recall in the context of patient decision aids, which in turn is showed to increase informed decision-making (Marteau et al., 2001; McGuire, 1996):

H1a: Compared to textual information, audiovisual information will result in more satisfaction and information recall, both directly and indirectly via satisfaction, which in turn will lead to more informed decision making.

Narrative Information

Narratives, or information presented in a personal, conversational style, are often liked better and experienced as more interesting (Bilandzic & Busselle, 2011; Kreuter et. al., 2010; McQueen, Kreuter, Kalesan, & Alcaraz, 2011); this is called the personalization effect (Kreuter et. al., 2010; Mayer, 2002). A previous study showed that narrative online health information increased satisfaction with the information compared to more factual information (Bol et al., 2013). Because of the positive feelings narratives evoke, satisfaction with narrative information might also increase motivation to process information (Bekker et al., 2013), which is expected to result in better recall (Petty & Cacioppo, 1986) and in turn more informed decision-making.

Additionally, we expect that individuals learn medical information better when presented in narrative format rather than factual format. One explanation is that the narrative information is structured following a chronological framework and information presented in an already existing framework will be remembered as part of this framework and is, therefore, easier to recall (Tulving & Pearlstone, 1966). According to Schank’s indexing theory, when receiving new information, individuals tend to link this to similar information already stored in their memory which allows them to remember this information better (Schank & Berman, 2002). When information is presented as a narrative, individuals can link the new information to elements of the narrative that are similar to ideas already stored in their memory helping them to organize complicated information (Koenig Kellas, 2008) and retrieve this information at a later date (Davidhizar & Lonser, 2003; Schank & Berman, 2002).

Studies on education methods and learning show promising results of incorporating narratives into teaching with regards to information recall (Kilaru et al., 2014; Kromka & Goodboy, 2019). Similarly, research on health information provision, both online and offline, have found a positive effect of narratives on satisfaction (Bol et al., 2013) and information recall (Bekker et al., 2013; Bol et al., 2015; Davidhizar & Lonser, 2003; Norris, Guilbert, Smith, Hakimelahi, & Phillips, 2005). A study that focused on decision aids specifically, showed an increase in knowledge of the information presented (Bekker et al., 2013). Because of the personal nature and structure of narratives described above, the following hypothesis is formulated:

H2a: Compared to factual information, narrative information will result in more satisfaction and information recall, both directly and indirectly via satisfaction, which in turn will lead to more informed decision making.

Audiovisual information and Narrative Information.

Both audiovisual information and narrative information in decision aids are expected to increase satisfaction, recall and lead to more informed decision-making, but combining them is expected to be most effective. As suggested by Bol et al. (2015), it can be expected that combining different information provision strategies show not only individual effects, but also synergistic effects which might optimize the effectiveness of the message even further. Previous research regarding online health information suggests that when audiovisual information is presented in a narrative style, satisfaction with the information and information recall increases, compared to audiovisual information in a factual style, textual information in narrative style, and textual information in factual style (Bol et al., 2013, 2015).

Because audiovisual information should be more effective than textual information and narrative information more effective than factual information, we expect the combination of audiovisual and narrative information to outperform the other combinations of modality and narration style:

H3a: Narrative information presented in audiovisual format will result in more satisfaction and information recall both directly and indirectly via satisfaction, which in turn will lead to more informed decision making than all other combinations of modality and narration style.

Age-related Impairmentsand Information Processing

Older patients generally possess less cognitive capacity, which may hinder adequate information processing. Therefore, making treatment decisions can become even harder for older patients. In line with this, studies showed that as patients get older they become less effective in several decision tasks that involve fluid cognitive ability (e.g., reasoning, problem-solving) (Bruine de Bruin et al., 2012; Finucane & Gullion, 2010; Mata et al., 2007, 2012, 2010), experience more difficulties comprehending factual and numerical information (Hibbard, Slovic, Peters, Finucane, & Tusler, 2001), experience more problems recalling medical information (Butow, Brindle, McConnell, Boakes, & Tattersall, 1998; Hillen et al., 2016; Jansen et al., 2008; Nguyen et al., 2018), use less information when making a decision (Johnson, 1993, Johnson & Drungle, 2000; Riggle & Johnson, 1996), and tend to make more immediate decisions regarding treatment options (Meyer, Russo, & Talbot, 1995; Meyer, Talbot, & Ranalli, 2007). These changes that come with aging become especially problematic when it is necessary to process new information (Zacks, Hasher, & Li, 2000), as is the case when making treatment decisions. Even though decision aids have been shown effective, effect sizes were smaller in studies where patients were older compared to studies including younger patients (van Weert et al., 2016). Thus far, the effectiveness of different information provision strategies within the context of decision aids has barely been tested among older patients (van Weert et al., 2016). Decision aids may not have been sufficiently adapted to the cognitive changes in information processing that come with aging (van Weert et al., 2016).

Previous research has shown that older adults experiencing cognitive declines in working memory preferred audio information over textual information (Wright et al., 2010). On the contrary, a study in which different information provision modes (i.e., instructional text, audio or video) were tested among older adults showed that older adults did not clearly prefer one mode over the other (Soroka et al., 2006). However, older adults often experience age-related sensory impairments, for example, worsened hearing or vision (Kiessling et al., 2003) Presenting information that addresses? more than one sensory system, such as audiovisual information, is supposed to be particularly important for this group of patients. Therefore, audiovisual information should lead to more satisfaction, and subsequently better recall, as the patient’s age increases.

According to the “Cognitive Aging Principle in Multimedia Learning”, in which elements of the cognitive load theory are combined with aspects of cognitive aging, aging can result in several cognitive declines which can be bypassed by using audiovisual information (Van Gerven, Paas, Van Merriënboer, & Schmidt, 2002; Paas, Van Gerven, & Tabbers, 2005 Audiovisual information could decrease cognitive overload by providing the information via multiple sensory systems. This is especially important for older adults taking into account their age-related cognitive declines (Van Gerven et al., 2002). In line with this, previous studies have shown that audiovisual information reduces cognitive load and the time to learn information for older patients (Van Gerven et al., 2003; Van Gerven et al., 2006). In the current study, we assumed that audiovisual information will lead to better recall directly and in turn more informed decision-making as the patient’s age increases.

As mentioned before, information can be processed via both systems (Petty & Wegener, 1999; Stanovich, 1999). However, individual differences can alter which processing system becomes more active. For example, when the cognitive capacity to process information via the deliberative processing system is low, one will rely more on the affective processing system (Peters et al., 2008; Peters & Slovic, 2007). Therefore, older patients experiencing age-related reduced cognitive capacity will rely more on affective information processing. In addition, emotional goals become increasingly important as the end of life nears (Carstensen, 2006). As a result, older patients rely more on affective information and personal experiences. Because narrative information induces the affective mode of information processing (Finucane, Alhakami, Slovic, & Johnson, 2000) and provides patients with personal experiences of the characters presented in the narrative (Winterbottom, Bekker, Conner, & Mooney, 2011), it might fit older patient’s information processing better (Peters, Hess, Västfjäll, & Auman, 2007). We expect that, because older patients have to deal with these changes in information processing, the positive effects of narrative information on satisfaction, and subsequently recall and informed decision-making become stronger as patients age.

H1b: The direct and indirect effects of audiovisual information on satisfaction, information recall and informed decision-making become stronger when the age of the patient increases

H2b: The direct and indirect effects of narrative information on satisfaction, information recall and informed decision-making become stronger when the age of the patient increases

H3b: The direct and indirect effects of audiovisual and narrative information combined on satisfaction, information recall and informed decision making become stronger when the age of the patient increases

Method

Design & Participants

For this study, we conducted an online experiment. A 2 (modality: audiovisual versus textual information) x 2 (narration style: narrative versus factual information) between-subjects factorial design was applied to test the hypotheses (see Figure 2). Ethical approval was provided for this study by the Review Board of the authors’ institute (2017-PC-7979).

Data were collected via an online panel named Flycatcher which is ISO20252 and 26362 certified. Participants were analog cancer patients, people with a personal history of being diagnosed with cancer (Van Vliet et al., 2012). All participants were asked if they had experience with or knowledge about the treatments that were discussed in the decision aid. If participants answered yes to these questions, they were excluded for further participation.

Procedure

Depending on the condition, the participant received either audiovisual or textual information and narrative or factual information. The participants were asked to imagine they had just been diagnosed with colorectal cancer and that they had to choose between two possible treatment options using the webpage of the decision aid to decide. After viewing the webpage, the participants completed questions regarding their demographics, satisfaction with the webpage, information recall, and informed decision-making.

Stimuli Materials

The stimuli created for this experiment were pages adapted from an existing decision aid for colorectal cancer. Figure 1 summarizes the development process and Appendix A describes the process in more detail. The webpage included information about two treatment options for colorectal cancer patients, chemotherapy consisting of Capecitabine (CAP) and chemotherapy consisting of Capecitabine and Oxaliplatin (CAPOX). The page provided comparable information about harms and benefits of both treatments regarding the effect on the tumor, administration of the treatment, and possible side effects. The development of the webpage was an iterative process with continuous communication between the researchers and an advisory team (consisting of oncologists, cancer patients, developers of the decision aid and representatives of the Association for Stomach, Liver and Gastro-intestinal diseases and the Foundation for Patients with Gastrointestinal Cancer) to ensure that the information provided was unbiased and included all necessary details for patients to make an informed decision.

Figure 1. Hypothesized model

Four different versions of the webpage were developed, which only differed in presentation mode and narration style. Great care was taken to ensure that the core information included the same key elements about the treatment options in all four versions. All four conditions included the core content about the treatments, but since the narrative conditions told the story of a fictional patient, contextual elements were added (i.e., a patient’s personal experiences with the treatment and with the side-effects). For the narrative conditions, patients who had experience with the CAP or CAPOX treatment were interviewed and based on their experience a scriptwriter formulated the extra contextual elements. Therefore, the absolute amount of information differed between the factual and narrative conditions, but the amount and content of the core information were exactly the same in all conditions (please see Appendix C for the amount of words percondition). In the narrative and factual audiovisual conditions, the same information was provided as in the narrative and factual textual conditions. However, in the audiovisual conditions the information was given in anarration accompanying the videos. In the videos, animations were shown that only supported the narrated information and did not add any new information compared to the textual conditions. In the audiovisual factual condition, the animations showed certain characteristics of the treatment (i.e., the administration of the treatment via an intravenous drip or pills). In the audiovisual narrative condition, these animations were also shown, but some additional animations were included that showed how a character (fictional patient) experienced the treatment (i.e., emotions regarding getting an IV or going to the hospital a lot). Please see Appendix C for the final versions.

Sample Characteristics

In total, 262 participants completed the online survey. Based on the amount of information provided, it was decided that participants had to spend at least 1 minute to read the information in the textual conditions and at least 4.5 minutes to watch the video in the audiovisual conditions. Participants who did not spend enough time on reading the information or watching the videos were excluded. This left 232 respondents (M_age = 63.98, SD_age = 12.33, range 21–90, 43.1%male).

Measures

Satisfaction with Information

Satisfaction with the information in the decision aid was measured with the Website Satisfaction Scale that consists of items measuring (WSS; Bol et al., 2015): satisfaction with the attractiveness of the information, satisfaction with comprehensibility, and satisfaction with emotional support (α =.91). All items were measured on a 7-point Likert scale, ranging from totally disagree (1) to totally agree (7).

Information Recall

To measure information recall, the Patient Information Recall Questionnaire (NPIRQ) (Jansen et al., 2008) was used. Nineteen open-ended questions were formulated, addressing for example the effects of the treatment on the patients’ condition and side-effects. A codebook was developed beforehand and two trained coders coded all open-ended questions. Participants could receive a maximum of two points for each answer. Sum score ranged from zero to 38 points. 10% (N = 24) of the total cases were coded by both coders to calculate inter coder reliability. Krippendorff’s Alpha was calculated for each question and ranged from .702 to 1.000 (M = .913) (Hayes & Krippendorff, 2007).

Informed Decision Making

Informed decision-making was measured by using the multi-dimensional measure of informed choice (MMIC) (Marteau et al., 2001). According to Marteau et al. (2001), a decision is informed when a person has adequate knowledge of the subject and his or her attitude fit their decision. Knowledge was measured with 18 multiple-choice questions about the treatments, for example, about what the treatments included. Scores ranged from 0 to 18. Adequate knowledge was defined as a knowledge score higher than the median, which was 66.67%. This method is in line with previous studies in this domain (e.g., Meppelink et al., 2015). Attitudes were measured with a four-item scale for each treatment option (α = .81 and α = .87), measured on a 7-point semantic scale (Marteau et al., 2001). Choice was measured by one question regarding which treatment the participant would choose. If knowledge was sufficient and the attitude for the chosen treatment was more positive than for the other treatment, participants scored 1 ‘informed’. If knowledge was insufficient and/or the attitude for the chosen treatment was less positive than for the other treatment, participants scored 0 ‘not informed’.

Background Characteristics

Age, gender and educational level were included in the questionnaire as background measures. Knowledge about colon cancer and knowledge about chemotherapy (“How much knowledge do you have about colon cancer/chemotherapy treatments for colon cancer?”) was measured on a 7-point scale ranging from not at all to a lot of knowledge. Having a medical education and amount of Internet use were measured as control variables. In the narrative conditions, identification with the characters was also measured as control variable using the VES scale of Visser et al. (2016).

Manipulation Check

To test if the manipulation of narration style was successful, six items were measured on a 7-point Likert scale that ranged from totally disagree (1) to totally agree (7). The questions addressed how much the information in the stimulus material was perceived as factual (3 items, α = .80) and narrative (3 items, α = .79). The manipulations were successful. The narrative conditions were perceived as significantly more narrative F(1, 261) = 114.31, p < .001, (M = 5.24) than the factual conditions (M = 3.69). In addition, the factual conditions were perceived as significantly more factual F(1, 261) = 10.93, p < .001, (M = 5.29) than the narrative conditions (M = 4.80).

Statistical Analyses

Chi-square tests were conducted to test whether background variables were equally distributed among the four experimental conditions. To see if these background variables were related to the dependent variables, independent samples t-tests were conducted. The model described in H1, H2 and H3 was tested with structural equation modeling (SEM) conducted in AMOS version 23. This was a suitable method because it allowed the complete model to be tested at once, instead of separately analyzing the hypotheses which could more easily result in finding effects based on chance. Following the two-step approach of Kline (2016), the measurement part of the model was tested first and then used as input for testing the structural part. Chi-square, the root mean squared error of approximation (RMSEA) and the comparative fit index (CFI) were used to test model fit.

Results

Randomization and Control Variables

Participants were equally divided over the four conditions regarding gender, χ2 (3, N = 262) = 5.72, p = .126; educational level, χ2 (27, N = 262) = 27.64, p = .43; knowledge about colon cancer χ2 (18, N = 262) = 13.24, p = .78; knowledge about chemotherapy χ2 (18, N = 262) = 4.88, p = .99; having a medical education χ2 (9, N = 262) = 5.62, p = .78; and amount of internet use χ2 (81, N = 262) = 90.96, p = .21. Identification with the character who shared experiences about CAP F(1, 126) = .22, p = .77 and the character that shared experiences about CAPOX F(1, 126) = .43, p = .69 did not differ between male and female patients. In general, participants identified with the CAP character (M = 2.78) as much as with the CAPOX character (M = 2.74), t(126) = −.84, p = .40. Finally, there were no differences between the audiovisual condition and the visual condition regarding identification with the CAP character talking about CAP F(1, 126) = .02, p = .89 and the character talking about CAPOX F(1, 126) = .10, p = .76. Age was treated as continuous variable in all analyses.

Model Testing

The model was tested following the two-step approach (Kline, 2011). The model consisted of both latent structures, following the item-based approach, and parceled structures, following the item-parcel approach (Matsunaga, 2008). Taking into account the dimensionality of the variables, parceling is considered an acceptable approach for SEM and can have various advantages over using latent structures only, such as the possibility to include less parameters and decrease the risk for Type I errors (Little et al., 2002). Parceling is especially utilized for SEM if there is already an understanding of the nature and dimensionality of the items that are being parceled (Little et al., 2002). Since satisfaction with the information is a multidimensional construct consisting of several items and we did not have a clear vision of the dimensionality of the items that might underlie the construct, satisfaction with the information was inserted in the model as a latent structure. Information recall was measured with an index score based on the number of correct answers and we therefore consider this variable as unidimensional construct. According to Little et al. (2002), constructs that are unidimensional in nature can be added to the SEM model as parceled construct. Informed decision-making was also included in the model as a parceled construct because, despite its multidimensional nature, since the method of constructing this variable was deliberately and heavily based on the theoretically acknowledged steps of Marteau et al. (2001). Because this means the dimensionality of the structure was already decided based on theory, this variable was unfit to include in the SEM model as a latent structure.

Following Kline (2011), we first tested the measurement part using Confirmatory Factor Analysis for satisfaction. This provided more insight into the relations between the items used and possible dimensions of satisfaction. The CFA model showed all items loaded on satisfaction. Therefore, we used this CFA model as starting point and added the other variables to set up the structural model. The extent to which the tested models fitted the data was evaluated based on several criteria. The model fit of the measurement model was good, which means that the measures loaded significantly on the factors and were valid and reliable, χ2 (15) = 22.707, p = .09, CFI = .99, RMSEA = .05 (Hooper, Coughlan, & Mullen, 2008). The structural model showed a significant chi-square, χ2 (104) = 161.08, p < .001, but the other commonly reported indices showed a better fit, CFI = .98, RMSEA = .05 (Hooper et al., 2008). Therefore, the model was accepted and used for further interpretation. Figure 2 illustrates the results found in the SEM analyses, including standardized coefficients and significance of each path.

Figure 2. Schematic overview of the phases of stimuli development

Figure 3. Model.

Model fit (N = 262): χ2 (104) = 161.08, p < .001, CFI = .98, RMSEA = .5. Estimates presented in the model are standardized estimates for each path. Gender was included in the model as control variable. Abbreviations: M = Modality (audiovisual information vs. textual information). S = Satisfaction. IR = Information recall. IDM = Informed decision-making. * p < .05. ** p < .01. *** p < .001

Effects of Modality (H1a)

Analyses showed that modality had an effect on satisfaction with the information with audiovisual information resulting in more satisfaction than textual information (β = .29, SE = .07, p < .001). Modality also had a positive effect on recall(β = 2.19, SE = .48, p < .001), which means that in the audiovisual condition, participants scored higher on recall than in the visual condition (please see Figure 3). Besides, we expected that audiovisual information would positively influence satisfaction with the information, which in turn would lead to better information recall and, subsequently, better-informed decision-making. In accordance with this, we found that more satisfaction resulted in more information recall (β = 1.49, SE = 44, p < .001) and that this led to more informed decision-making (β = .03, SE = .00, p < .001). The sequential indirect effect of audiovisual information on informed decision-making was also significant (β = .07, 95% CI [.03, .09], p = .02), meaning that the effect of audiovisual information was serially mediated by satisfaction and information recall. Therefore, H1a was supported.

Effects of Narration Style (H2a)

Results showed a positive effect of narration style on satisfaction with the information (β = .14, SE = .07, p = .04)and a marginal significant effect on information recall (β = −.86, SE = .47, p = .07) (please see Figure 3). Hence, H2a was supported regarding satisfaction with the information. In addition, it was expected that the positive effect of narrative information on satisfaction (compared to factual information), would, in turn, lead to better information recall and, subsequently, better-informed decision-making. This sequential indirect effect of narrative information on informed decision-making was not significant (β = −.02, 95% CI [−.04, .01], p = .17). Thus the effect of narrative information on informed decision-making was not mediated by satisfaction and subsequently information recall and H2a was rejected.

Interaction Effects of Modality and Narration Style (H3a)

There was no interaction effect of modality and narration style on satisfaction with the information (β = −.07, SE = .07, p = .28), but results show a margianlly signifcant interaction effect on information recall (β = −.92, SE = .47, p = .05). Additionally, it was hypothesized that combining audiovisual and narrative information would have a more positive effect on informed decision-making, via satisfaction and information recall, than other combinations. The sequential indirect effect of audiovisual, narrative information on informed decision-making, via satisfaction and information recall was significant (β = −.03, 95% CI [−.05, .00], p = .04), indicating a mediated interaction effect of audiovisual and narrative information. Hence, H3a was partly supported.

Interaction Effects of Modality and Age (H1b)

Analyses showed a marginally significant interaction effect of modality and age on satisfaction with the information (β = −.13,SE = .07, p = .06). However, there was no significant interaction effect of modality and age on recall (β = .65, SE = .48, p = .17). In addition, we expected that the mediation effect of modality on informed decision-making via satisfaction and recall would be greater as the patient’s age increased, but age did not affect the sequential indirect effect (β = −.02, 95% CI [−.05, .00], p = .113). Therefore, H1b was rejected.

Interaction Effects of Narration Style and Age (H2b)

There was no interaction effect of narration style and age on satisfaction with the information (β = −.03, SE = .07, p = .69) and information recall (β = .21, SE = .47, p = .66). Hence, narrative information did not become more effective as the patient’s age increased. Again, it was expected that this mediation effect of recall on informed decision-making would become stronger with an increase in age, but the sequential indirect effect was not stronger when patient’s age increased (β = 0.00, 95% CI [−.03, .03], p = .85). Thus, H2b was rejected.

Interaction Effects of Modality, Narration Style and Age (H3b)

No three-way interaction effect of modality, narration style and age was found on satisfaction (β = −.01, SE = .07, p = .94), recall (β = .14, SE = .47, p = .77) and informed decision-making (β = .01, SE = .03, p = .72). This implies that both audiovisual conditions resulted in better recall and informed decision-making, regardless of the narration style of the information and that this does not depend on age. We also expected that this mediated effect on informed decision-making via satisfaction and recall would become stronger with age. The sequential indirect effect of audiovisual information, narrative information and age on informed decision-making, via satisfaction and recall was not significant (β = −.00, 95% CI [−.02, .03], p = .703). Therefore, H3b were rejected.

Conclusion &Discussion

The aim of this study was to test the effectiveness of two strategies of information provision in decision aids. Audiovisual and narrative information was tested in terms of satisfaction with the information, information recall, and informed decision-making, taking the patient’s age into account. We expected these strategies would become more beneficial when age increased.

Results revealed that audiovisual information resulted in more satisfaction with the information and better information recall, which in turn resulted in more informed decision-making compared to textual information. Contrary to our expectations, these effects appeared in patients of all ages and did not become stronger as age increased. Even though the positive effect of audiovisual information on information recall matches research that has focused on online health information provision in general (Bol et al., 2013, 2015; Meppelink et al., 2015), the current study showed that this positive effect also holds in the context of decision aids.

We did find an effect of narrative information on satisfaction with the information, but no effect on information recall. This is inconsistent with some previous studies where narrative information positively affected information recall (Bekker et al., 2013; Bol et al., 2015; Davidhizar & Lonser, 2003; Norris et al., 2005; Shaffer, Focella, Scherer, & Zikmund-Fisher, 2016). Nevertheless, a number of previous studies show mixed results regarding the effects of narrative information in decision aids on recall (Bekker et al., 2013). A reason for these mixed results could be that decision aids often provide patients with information about multiple treatment options. This makes the information considerably different than the general health information about one specific treatment option studied in previous research. It could be that narratives work for health messages that focus on the thoughts and actions of one character, but not for more complicated information in which the patient is expected to compare information regarding different treatment options or sympathize with the situation of two (or more) patients, who both made a different (often contrasting) choice, as seen in decision aids.

Furthermore, Shaffer et al. (2016) mention identifying with the character in the narrative as an important factor to increase feelings of transportation and immersion. Additionally, the few studies that have shown a positive effect of narrative information on recall, noted the importance of identifying with the main character in the narrative (Kreuter et al., 2007; McQueen et al., 2011). In our study, patient’s age or gender did not affect how they identified with the characters, but identification with the characters was relatively low on average. The main character in the audiovisual narrative condition was an animated figure, who was not clearly male or female. Patients might have identified themselves more with a more realistically designed character, a real person and/or a gender equivalent character. Future studies should investigate whether identifying with the main character in a narrative is a boundary condition for better information recall.

Moreover, a key feature of a narrative is that it provides not only factual information, but also contextual information (Escalas, 1998). This automatically means that the narrative conditions consisted of more information than the factual conditions. A previous study on recall of information provided during consultations with oncologists found that the more information was presented, the harder it became to recall this information (Jansen et al., 2008). Adding narrative, contextual elements to already complex information extends the amount of information and could therefore also complicate the processing of this information instead of making it easier. Even though in the current study we tried to limit the amount of contextual information and keep the core information in all conditions similar, it could be that the possible positive effect of narrative information is neutralized by excess contextual information. This could be especially the case for older patients who already experience more difficulties distinguishing the main information from side issues in a body of content (Park, 2000). Future research should investigate if narrative information is less effective for certain complexity levels of information within the context of healthcare decision-making.

We expected that audiovisual and narrative information would become more effective as patients’ age increased, but we did not find any results to support this assumption. However, while interpreting these results, we should emphasize that informed decision-making was operationalized and measured as a deliberative decision where the patient had to possess sufficient knowledge about the different treatment options. Some scholars in the field of informed decision-making use a different definition and argue that affective processing is also important in making an informed decision (Peters et al., 2008). It is reasonable that narrative information does support older patients in affective decision-making processes, but that we were unable to uncover such results because the measure used in the current study possibly did not take into account the affective part of healthcare decision making. Future research should consider using other measures that might provide a more complete image of informed decision-making.

Another reason for the absence of the expected age effect could be that having control over the pace of the information received becomes more important with aging. Previous research has suggested that recall in older patients increases when they spend more time processing the information (Bol et al., 2015). In line with this, Meppelink et al. (2015) suggested the importance of self-pace audiovisual information for older individuals with low health literacy. Future research could study whether self-paced audiovisual information results in stronger effects for older patients.

Moreover, the data were collected via an online panel. Participants in this panel actively decided to be part of the panel and to fill out online questionnaires on a regular basis. Therefore, the participants included in this study were experienced internet users. There were no age differences regarding patients’ internet use, whereas in real life aging probably plays a role in how the internet is used (Statistics Netherlands, 2013a). As we might not have reached the most vulnerable older individuals, this might explain why we did not find the expected age differences. We recommend including more vulnerable older patients in future studies, for example, in field studies with real patients who need to make a treatment decision.

Lastly, the sample of this study was relatively small. To evaluate whether the nonsignificant results regarding narrative information and recall, as well as the rejected interaction effects between modality, narrativity, and age were caused by a lack of power we conducted post hoc power analyses. First, to test if we had enough power to test the fit of the SEM model we used the method by MacCallum, Browne, and Sugawara (1996) in which sample size, RMSEA fit indices and degrees of freedom of the model are taken into account to calculate the power. We set the RMSEA value on .08 to test the power since this is seen as satisfactory (MacCallum et al., 1996). The outcomes of this power analyses (α = 0.05) were .99; thus, we can conclude that there was enough power to test the fit of the hypothesized model. In addition, we conducted a post hoc power analysis based on the sample size, the number of predictors in the model and the R2 value of recall (Cohen, 1988; Soper, 2020), to check if there was enough power for rejecting the effects of narrativity and the interaction effects on recall. The results show power (α = 0.05) to be .99 for testing these hypotheses. Therefore, we expect that the absence of significant effects of narrativity, narrativity and age, and modality and age, is not caused by a lack of power.

Besides methodological limitations, it is also important to point out a limitation regarding the choice of analyses. While using Structural Equation Modeling allows the complete model to be tested in one time, decreasing the possibility of finding effects based on chance, this means that multiple tests are conducted on the same set of variables, increasing the probability of falsely accepting hypotheses. Researchers have not yet reached consensus about how to handle this problem. Several error controlling measures have been suggested (Benjamin, Hochberg, & Kling, 1994; Cribbie, 2000), but it seems like more recently researchers have argued that these measures are overly conservative for Structural Equation Modeling (Cribbie, 2007; Dalrymple & Zimmerman, 2008). Therefore, no error-correcting measures were taken in this study and it is important to consider the possibility of false positives in our study, or type I errors, that could come with Structural Equation Modeling while reading the results of the current study.

Despite the above-mentioned limitations, our study provides insight into strategies that can optimize information in online decision aids. Even though we did not find that effects become larger when people age, we did find positive results of audiovisual information in general. Therefore, incorporating audiovisual information in decision aids seems a promising communication strategy to increase the satisfaction with the information, information recall, and informed decision-making for patients of all ages. From a theoretical viewpoint, our findings prove the importance of being satisfied with information and correctly recalling the information for making an informed decision about cancer treatment. Including narrative information could be considered, but developing high-quality narratives can be expensive and time-consuming. Therefore, developers should carefully consider the pros and cons of including narrative information in the decision aid.

Acknowledgments

We wish to thank Dr. A. Mels (Management MLDS), Dr. M. Koopman (Internist-oncologist UMC), Dr. J. Portielje (Internist-oncologist LUMC) and A. Willems (Chairman SPKS) for their advice in different phases of this study.

Additional information

Funding

This work was supported by the ZonMw [516000510].

Appendix A.

Development of the stimulus materials

The stimuli created for this experiment were adapted parts of a webpage derived from an existing Dutch decision aid for colorectal cancer, developed by The Association of Stomach, Liver and Gastro-intestinal diseases (Maag-Lever-DarmStichting). The content of the webpage included information about two treatment options for colorectal cancer patients, namely chemotherapy consisting of Capecitabine (CAP) and chemotherapy consisting of Capecitabine and Oxaliplatin (CAPOX). The information on the webpage addressed comparable information about harms and benefits of both treatments, including the effect of the treatment on the cancer, on the patient’s condition and survival chance, how the treatment is administered to the patient, and what side-effects can occur.

A version of the webpage was developed for each condition, so in total four different versions were developed that differed in presentation mode and narration style of the information. For development of the webpages, the authors collaborated with an advisory team that consisted of medical oncologists, cancer patients, developers of the decision aid, the Association for Stomach, Liver and Gastro-intestinal diseases and the Foundation for Patients with Gastrointestinal Cancer. The development of the webpage was an iterative process with continuous back and forth communication between the researchers and the advisory team. Figure 1 gives an overview of the development process.

In the factual conditions, the information was presented in a more formal style in which statistical information was provided without contextual information. We started with the development of the textual information for the factual textual condition, which was nearly the same as the original text in the decision aid. After this version was definite, we adjusted the information to make it suitable for the narrative conditions. The narrative information consisted of the same information as in the factual condition, combined with more contextual information about the personal lives and treatment experiences of two fictional patients. The first fictional patient shared experiences about the CAP treatment and the second one about the CAPOX treatment. This was done to keep the information as much as possible the same between the factual and narrative condition, while only manipulating narration style of the information. The information was presented in chronological order and was causally related, which are characteristics for narratives. The added contextual information was based on information provided by (ex-)cancer patients who received one of the two treatments. In total eight (ex-)cancer patients, who had chosen either the CAP or the CAPOX treatment in the past, were interviewed. These patients were recruited by means of an invitation via an online panel including cancer patients founded by an academic hospital in the Netherlands. Besides, patients were recruited via the Association for Patients with Gastro-intestinal Tract (SPKS) and The Association of Stomach, Liver and Gastro-intestinal diseases. The interview guide included questions about the diagnosis, why the patient chose one treatment option over the other, and how the chosen treatment impacted the patient’s life. Subsequently, the transcribed information derived from these interviews were sent to a scriptwriter, who used the information of all interviews to compose two narratives; one for each treatment. The scriptwriter created a fictional, gender neutral patient for each treatment. The narrative was written from this patient’s perspective and told in first-person. We kept the webpage as much as possible the same in design as the factual webpage, so the page consisted of a text and two images per treatment option. The two images were the same as in the factual condition. Because of the added contextual information in the narrative condition, the information consisted of more words (1.222 words) than in the factual condition (752 words).

After having developed the textual information for both the factual and narrative conditions, we started developing the animated videos for the audiovisual conditions. The animated videos were development in collaboration with a professional company (BehandelingBegrepen), whose expertise is to make medical information more understandable for patients by means of animations. To ensure that the characters were appealing to both younger and older cancer patients, 21 different characters were pretested. Data for this pretest were collected by means of an online survey (N = 38) among participants who were again recruited via the panel, and a focus group (N = 9) among visitors of a community center for older people. Based on the results of this pretest, we selected an animation style for this experiment that was most similar to the animation style that scored best in the pretest.

The same textual information as in the textual conditions was narrated during the animation, while the animation depicted the events described in the textual information. Thus, the text of the spoken narration was exactly the same as the text in the textual conditions, so that only presentation mode of the information was manipulated and not the spoken content. The text was narrated in a men’s voice. The animated videos in the factual conditions mainly showed moving images that depicted the information, for example, a tumor, pills and a hospital. In the animated videos for the narrative conditions, the fictional patients that were described in the narratives were pictured as animated characters. All images were developed with a computer by the professional company, so no videos of real people were used. The characters used in the narrative animated videos were gender neutral to enhance equal identification with the characters among both men and women. The factual and narrative animated videos were respectively 2.35 minutes and 3.45 minutes long.

Appendix B.

Hyperlinks to stimulus materials

Audiovisual Factual: http://www.oca22017.nl/oca2742992740293/

Audiovisual Narrative: http://www.oca22017.nl/oca2748858210/

Textual Factual: http://www.oca22017.nl/oca275937583992/

Textual Narrative: http://www.oca22017.nl/oca2238864927/

Appendix C.

Explanation exclusion based on watching/reading time of stimulus materials

Textual conditions: pretests of the textual stimulus materials revealed the minimal amount of reading time it took the fastest reader was 1 minute.

Audiovisual conditions: The minimum amount of time that was needed to get the essence of the video was calculated. Please see Table 1 below. All videos included an introductory and closing part in which no information was provided regarding the key elements Besides, there was overlap in the key information provided in the CAP and the CAPOX videos in both conditions. For calculation of the exclusion criteria, the total time of both videos was taken and the duration of the introduction, closing and overlapping parts were subtracted. This resulted for both conditions in a total of 4.5 minutes in which the key elements were discussed.

Table 1. Length of videos used as stimulus materials

Condition	Treatment Option	Length videos/Amount of words	Total time per condition/Total amount of words
Factual/Audiovisual	CAP	2.50 minutes	5.37 minutes
	CAPOX	2.47 minutes
Narrative/Audiovisual	CAP	3.56 minutes	7.42 minutes
	CAPOX	3.46 minutes
Factual/Textual	CAP	382 words	754 words
	CAPOX	372 words
Narrative/Textual	CAP	618 words	1222 words
	CAPOX	604 words