The use of contextual cues to improve warning symbol comprehension: making the connection for older adults

Abstract

This study teased apart the effects of comprehensibility and complexity on older adults' comprehension of warning symbols by manipulating the relevance of additional information in further refining the meaning of the symbol. Symbols were systematically altered such that increased visual complexity (in the form of contextual cues) resulted in increased comprehensibility. One hundred older adults, aged 50–71 years, were tested on their comprehension of these symbols before and after training. High comprehensibility–complexity symbols were found to be better understood than low- or medium-comprehensibility–complexity symbols and the effectiveness of the contextual cues varied as a function of training. Therefore, the nature of additional detail determines whether increased complexity is detrimental or beneficial to older adults' comprehension – if the additional details provide ‘cues to knowledge’, older adults' comprehension improves as a result of the increased complexity. However, some cues may require training in order to be effective.

Practitioner Summary: Research suggests that older adults have greater difficulty in understanding more complex symbols. However, we found that when the complexity of symbols was increased through the addition of contextual cues, older adults' comprehension actually improved. Contextual cues aid older adults in making the connection between the symbol and its referent.

Keywords:

• warnings
• safety symbols
• ageing
• training

1. Introduction

As the US working population continues to age (see, e.g. statistics produced by the Administration on Aging 2010), it has become increasingly important to understand how age-related changes in cognitive processing may impact the workplace. One critical area is the understanding of safety communications. Often, safety communications rely on symbols because they can potentially convey large amounts of information in minimal space, be seen from further distances than text, be remembered better than text and communicate to target populations of varying language backgrounds. Although there are clear advantages for incorporating symbols into warning communications, the effectiveness of symbols has only been inconsistently observed with older adults – that is, some studies indicate that older adults have greater difficulty than younger adults in understanding warning symbols (e.g. Collins and Lerner 1982; Easterby and Hakiel 1981; Hancock, Rogers, and Fisk 1999; Lesch 2003, Lesch et al. 2011; Morrell, Park, and Poon 1990; Zwaga and Boersema 1983).

We (Lesch et al. 2011) have argued that inconsistent findings regarding age-related differences in warning symbol comprehension may be due to the use of different comprehension measures, different types of symbols and varying definitions of ‘older’ populations across studies. Regarding comprehension measures, we have suggested that poor comprehension by older adults on open-ended tests of warning symbol comprehension may reflect an age-related decline in verbal fluency, rather than a decline in warning symbol comprehension. Consequently, we developed an alternative measure of comprehension that minimises verbal output – the semantic relatedness judgement task (see Lesch 2005). In this task, participants view a warning symbol paired with a verbal label that either matches the symbol's meaning or does not. The correct verbal label and the distractor appear on separate trials and the participant's task is to decide, as quickly as they can, whether the text matches the meaning of the symbol and then rate the level of their confidence in their judgement. A symbol is considered as understood if the participant accepts the correct answer and rejects the incorrect answer. Level of confidence is used to further discriminate comprehension levels. Using this task, a series of studies (Lesch 2003, 2004, 2008a, 2008b) indicated that older adults (aged 55–70 years) have poorer comprehension than younger adults (aged 18–35 years), but that comprehension can be improved through the use of accident scenario training, in which a symbol is paired with an accident scenario that further expands on the nature of the hazard, the required or prohibited actions and the consequences of failure to comply. Although this line of research documented age-related difficulties in warning symbol comprehension and the effectiveness of accident scenarios in addressing those difficulties, it did not address the issue of whether older adults have a specific difficulty with certain types of symbols or whether they have a more generalised problem in comprehending symbols. However, other research has indicated that less familiar symbols are more poorly understood than more familiar symbols (e.g. Hancock et al. 2004; see also Ng and Chan 2007).

Based on a review of age-related declines in cognitive abilities, we (Lesch et al. 2011) suggested that older adults should be more likely than younger adults to experience difficulty with more complex symbols and less comprehensible symbols (i.e. symbols for which the meanings are estimated not to be easily guessed without prior knowledge). An effect of complexity is expected on the basis of reduced inhibitory efficiency (i.e. a reduced ability to ‘shut off’ irrelevant information) (Hasher et al. 1991, 168, see also Zacks, Radvansky, and Hasher 1996) or on the basis of a reduced ability to selectively attend (e.g. Alain, Ogawa, and Woods 1996; Allen, Weber, and Madden 1994; McCalley, Bouwhuis, and Juola 1995), whereas an effect of comprehensibility is expected on the basis of an increased difficulty in forming associations between previously unrelated entities (see Luo and Craik, 2008). In this study (Lesch et al. 2011), younger and older adults' comprehension of symbols varying in terms of rated familiarity, complexity and comprehensibility were tested before, and after, accident scenario training. Consistent with earlier studies, it was found that older adults have greater difficulty than younger adults in comprehending warning symbols and that accident scenario training improves comprehension. The more interesting question concerned the impact of symbol characteristics.

The impact of symbol characteristics was examined through regression analyses, which indicated that familiarity and comprehensibility were strong predictors of pre-training comprehension by both younger and older adults. However, for older adults, the effect of comprehensibility was moderated by complexity – that is, increases in complexity resulted in a decreased effect of comprehensibility. Training eliminated the effects of symbol characteristics for younger adults, whereas older adults' comprehension continued to be significantly influenced by comprehensibility, indicating that comprehensibility is critical for learning by older adults. These results suggest that older adults may have particular difficulty in understanding high complexity and low comprehensibility symbols. However, symbol characteristics did not vary independently in this study and it tended to be the case that high complexity symbols were also low comprehensibility.

In an attempt to further tease apart the effects of complexity and comprehensibility, this study used existing symbols that were systematically altered to produce symbol triplets that simultaneously increased in complexity and comprehensibility. To produce these triplets, it was sometimes necessary to (1) increase an existing symbol's complexity, while also increasing its comprehensibility, (2) decrease a symbol's comprehensibility, while maintaining its complexity and (3) increase a symbol's complexity, while decreasing its comprehensibility. We were particularly interested in determining whether contextual cues could be incorporated into the symbols to improve their comprehensibility, despite a corresponding increase in complexity. We had noted in an earlier study (Lesch 2004) that older adults were able to learn the meaning for the symbol for cancer-causing substance, despite its low-rated comprehensibility. We argued that after being told the meaning of the symbol, older adults were able to make use of a verbal retrieval cue incorporated in the symbol – the broken circle represents cell mutation, as well as the letter ‘c’ for ‘cancer’ (see Figure 1). This study represents a formal test of that hypothesis and explores the possibility that the amount of information (i.e. complexity) contained in the symbol may not be critical, but, rather, whether or not that information is relevant to determining the meaning of the symbol.

Figure 1 The symbol for cancer-causing substance contains the verbal retrieval cue ‘c is for cancer’.

Complexity and comprehensibility were determined through expert ratings of the symbols (procedure to be described in detail below). Older adults (aged 50–71 years) were tested on their comprehension of these warning symbols both before and after receiving accident scenario training. Comprehension was measured by accuracy of, and confidence in, judgements on the semantic relatedness task. It was expected that comprehensibility would be most critical in determining older adults' comprehension of warning symbols because it was expected that symbols that incorporated contextual cues would help older adults ‘make the connection’ between the symbol and long-term memory representations, despite the increased complexity of the symbol. The ability to form new associations (e.g. in learning a new language) tends to decrease with age (Luo and Craik 2008). It was expected that the inclusion of contextual cues would help older adults in forming associations between (unfamiliar) symbols and stored knowledge. We were also interested in determining how contextual cues might interact with training – that is, do cues require training to be effective? Or would training, which should increase the familiarity of the symbols, eliminate any benefit associated with contextual cues?

2. Methods

2.1. Participants

One hundred older adults (51 females and 49 males), between the ages of 50 and 71 years, were recruited through local newspaper/online advertisements and received $40 for their participation. All participants were native English speakers with normal, or corrected to normal, visual acuity. Prospective participants who reported that they took medication or had a health condition that might influence their performance in the study were excluded. The mean age of the participants was 62 (SD = 4.6) years. Although our ‘older adults’ might be more appropriately described as ‘middle-aged’ (see Nichols, Rogers, and Fisk 2003), this age range was selected to represent older working age adults. Furthermore, this age range is similar to that used in a series of studies (Lesch 2003, 2008a, 2008b; Lesch et al. 2011), where we have previously observed age-related differences in warning symbol comprehension, thereby facilitating comparison of results across studies. All study procedures were approved by the Liberty Mutual Research Institute for Safety's Institutional Review Board.

2.2. Apparatus

Four personal computers (2.8 GHz processor) and 21-inch LCD monitors were used for data collection. The experimental programme was developed and executed using E-Prime 2.0 software (Psychology Software Tools, Inc., Sharpsburg, PA, USA). The program measured all valid keyboard responses and reaction times from stimulus presentation onset to participant keyboard response.

2.3. Stimuli

Thirty-two warning symbol triplets, each representing a single warning concept (i.e. cancer-causing substance), were created such that the members of each triplet were expected to vary in terms of their complexity and comprehensibility. As a starting point, symbols were obtained from a manufacturer of safety labels/signs with additional symbols being obtained from Dreyfuss (1984) and Modley (1976). These symbols were then modified as required to satisfy the conditions of the experiment. The symbols represented a variety of industries including (but not limited to) medical, chemical, construction, laboratory and manual materials handling. The symbols were presented in a 15 cm × 15 cm area and subtended ∼15° of visual angle.

The symbols within each warning concept triplet were designed to vary from low to medium to high comprehensibility–complexity. The biohazard symbol, modified to include a syringe, is an example of a modification that was intended to increase the complexity and the comprehensibility of the symbol (see Figure 2). Alternatively, the symbol for cancer-causing substance was rotated in order to maintain its complexity, but decrease its comprehensibility – the rotation of the symbol was expected to eliminate the verbal cue contained within the symbol (‘c’ is for ‘cancer’) (see Figure 2). Several different types of contextual cue were incorporated: verbal/symbolic cues (i.e. ‘c’ is for ‘cancer’ or inclusion of the arrow in the symbol for crush hazard), colour cues (i.e. the use of the colour red to indicate ‘heat’ as in hot surface and protect from heat), human form cues, which consisted of the inclusion of the human form or parts of the body for reference (i.e. as for the symbols for safety shower and cancer-causing substance), general contextual cues (i.e. the addition of the box for the symbol for protect from heat and the wrench for read instructions) and multiple cues for those symbol variants, which included multiple cue types (Figure 3).

Figure 2 Examples of symbol modifications intended to vary complexity and comprehensibility. For biohazard, the addition of the syringe was intended to increase both complexity and comprehensibility. For cancer-causing substance, the rotation of the symbol was intended to eliminate the verbal cue ‘c is for cancer’, while maintaining the same level of complexity.

Figure 3 Examples of the different contextual cues incorporated in the experiment.

An initial attempt was made to obtain ratings of familiarity, complexity and comprehensibility of the symbols from study participants. However, it was found that the participants' ratings lacked face validity, in that they appeared to be driven primarily by the comprehensibility of the items – that is, highly comprehensible symbols that were more visually complex were actually judged by the participants to be less complex. It was also the case that less familiar symbols (e.g. symbols created for the experiment) were rated as highly familiar if they were highly comprehensible (Figure 4). Therefore, we decided to obtain ratings from experts.

Figure 4 Examples illustrating lack of face validity in participants' ratings of stimulus characteristics.

Three experts with backgrounds in experimental psychology rated the symbols in terms of familiarity, complexity and comprehensibility. Prior to completing the rating tasks, the experts participated in a discussion of the meaning of the concepts ‘comprehensibility’ and ‘complexity’. It was agreed that complexity should reflect the number of visual features/characteristics contained within the symbol whereas comprehensibility should reflect the extent to which there was sufficient information contained within the symbol to identify its meaning without any prior experience with the symbol, or any other additional information. Rating type was blocked with familiarity ratings obtained first, followed by complexity ratings, and then by comprehensibility ratings. On each trial, the expert viewed three alternative symbols for a given concept. To judge familiarity, the experts received the following instructions: ‘How often have you encountered each of these symbols? Please judge how often you have encountered these symbols using the numbers from 1 (never) to 5 (frequently)’. To judge complexity, the experts received the following instructions: ‘How visually complex is each of these symbols? Please judge how visually complex you think each of the symbols are. Use the numbers from 1 (not at all complex) to 5 (extremely complex) at the top of the keyboard to indicate your rating. Please note that visual complexity is not related to the recognisability or meaningfulness of the symbols. Please base your judgements solely on visual characteristics of the symbols’. For comprehensibility, the experts were given the warning concept (i.e. cancer-causing substance) and were instructed: ‘Pretend you are seeing each of these symbols for the first time and you know nothing about them. How likely do you think it is that you could guess the meaning of each symbol? Please judge how likely you think it is that you could guess the meaning of each symbol if you knew nothing about it. Use the numbers from 1 (not at all likely to guess) to 5 (extremely likely to guess) at the top of the keyboard to indicate your rating.’ The results of the expert rating task are presented in the Results section.

Each warning symbol concept had an associated verbal label (i.e. ‘cancer-causing substance’). However, the use of the semantic relatedness judgement task necessitated the development of distractor labels. Although the study design involves comparison of symbols within a warning symbol concept to tease apart complexity and comprehensibility, it was necessary to control the degree of plausibility of distractors across symbols within a given warning concept triplet. The cleanest way of doing so would be to use the same unrelated distractor for each member of the warning concept triplet. Therefore, distractors related to the warning concepts were developed and then randomly re-paired with other warning concepts and then checked to ensure that no ‘related’ pairings had resulted by chance. It should be noted that although the use of unrelated distractors facilitates comparisons within warning concept, it would also be expected to decrease the difficulty of the semantic relatedness task (i.e. it should be easier to reject unrelated distractors than to reject plausible distractors).

There were also 16 filler warning symbol concepts that received accident scenario training and were tested for comprehension. Seven of these filler items had reading comprehension questions associated with them to help ensure that participants were reading the accident scenarios.

2.4. Procedure

Upon arrival, participants read and signed an informed consent form. Participants were seated in front of a computer and were told that the experiment was concerned with how people understand warning symbols. The study consisted of four main sections (administered in successive order): pre-training comprehension, training, demographic questionnaire and post-training comprehension. The experimental session lasted ∼90 min.

2.4.1. Pre-training comprehension

Comprehension of the warning symbols was assessed via the semantic relatedness paradigm (Lesch 2003), in which the participant views a symbol paired with a verbal label and is asked to decide whether the verbal label conveys the meaning of the symbol.

Six stimulus (counterbalancing) lists were composed such that each participant viewed only one symbol from each warning concept triplet and equal numbers of each comprehensibility–complexity level. Order of presentation within each stimulus list was randomised. Each symbol within a given counterbalancing list was viewed twice by participants – once with a label that conveyed the meaning of the symbol (i.e. the correct label) and another with a label that did not convey its meaning (i.e. the distractor label), resulting in a total of 96 randomly presented trials; 32 experimental items × 2 and 16 fillers × 2. The participant's task was to determine, as quickly as possible, whether the label conveyed the meaning of the symbol by pressing ‘Yes’ or ‘No’ on the keyboard. Immediately following their Yes/No response, the participants reported their level of confidence in their decision from 1 (not at all confident) to 5 (certain).

2.4.2. Training

In the training phase of the study, participants viewed each symbol with its verbal label (1500 ms), followed by an accident scenario which described an accident or ‘close-call’ related to the hazard indicated by the symbol (the display was terminated by the participant's button-press), followed by a second presentation of the symbol with its verbal label (i.e. its referent) (1500 ms). The accident scenarios further elaborated on the nature of the hazard depicted by the symbol, the recommended actions, as well as the possible consequences of failing to perform these actions (see Figure 5). The accident scenarios were derived from accident reports from a number of online sources (see references) including the US Department of Labor Occupational Safety & Health Administration Accident Report Fatal Facts, the National Institute for Occupation Safety and Health Alerts, the US Department of Labor Mine Safety and Health Administration Safety Hazard Alerts and the Centers for Disease Control and Prevention's Morbidity and Mortality Weekly Report. Earlier research (e.g. Lesch 2008a) indicated that the presentation of an accident scenario during training provides an additional benefit beyond provision of the verbal label alone. On seven filler trials, the accident scenarios were followed by a question on the content of the accident scenario to encourage participants to actively read the associated text.

Figure 5 Symbol for eyewash location with its associated accident scenario.

2.4.3. Demographic questionnaire

The demographic questionnaire asked participants their age, gender, level of education and years in the workforce. They were also asked whether they had ever worked in an occupation in which there were serious hazards to life and limb and whether they (or someone they know) had ever been involved in a work-related accident. Participants responded to the questionnaire immediately following training in order to provide a brief delay between training and post-test and to prevent rehearsal of the to-be-learned information.

2.4.4. Post-training comprehension

Participants were tested again on the meaning of the symbols using the same paradigm used in the pre-training comprehension test.

3. Results and discussion

3.1. Accident scenario comprehension

On average, participants obtained 87% correct on the questions associated with the accident scenarios on the filler trials suggesting that they were actively reading the accident scenarios. There was no effect of gender on comprehension performance.

3.2. Expert ratings

There was strong agreement in experts' ratings of comprehensibility (mean Pearson's r = 0.88) and complexity (mean Pearson's r = 0.83). However, nine warning concept triplets were removed from the data analysis to ensure complete agreement among the experts' rank ordering (derived from ratings) of their members in terms of complexity and comprehensibility, because the relative complexity/comprehensibility (within a triplet) is critical to the comparisons to be made here. The remaining 23 warning concept triplets contained symbols which increased in comprehensibility, while at the same time increasing in complexity. Therefore, there was a low comprehensibility–complexity symbol, an intermediate comprehensibility–complexity symbol and a high comprehensibility–complexity symbol for each warning concept triplet. Analysis of the expert ratings indicated that the low, medium and high comprehensibility–complexity conditions significantly differed in their mean comprehensibility and complexity ratings (all ps < 0.05) (Table 1). The experts' ratings of familiarity indicated that, on average, they were somewhat familiar with the symbols examined (mean = 2.0 and SD = 0.99). The experts were very knowledgeable about cognitive processes such as visual information processing, but were not intended to be experts on warning symbols, in particular.

Table 1 Mean (SE) expert comprehensibility and complexity ratings as a function of comprehensibility–complexity condition.

	Low–low	Medium–medium	High–high
Comprehensibility	2.2 (0.2)	3.1 (0.2)	4.3 (0.2)
Complexity	2.0 (0.2)	3.0 (0.1)	4.0 (0.1)

3.3. Warning symbol comprehension

To assess warning symbol comprehension, two dependent measures focused on accuracy of responses: (1) percent correct (where ‘correct’ was defined as correct acceptance of the target and correct rejection of the distractor) and (2) composite confidence scores which incorporated confidence ratings together with the comprehension responses. Confidence ratings were first transformed to a scale from 1 to 10. For target trials, ‘no’ responses were assigned values from 1 to 5 with ‘certain-no’ responses receiving a score of 1 and ‘not at all certain-no’ responses receiving a score of 5. ‘Yes’ responses received scores from 6 to 10 with ‘not at all certain-yes’ responses receiving a score of 6 and ‘certain-yes’ responses receiving a score of 10. Therefore, for trials in which the correct answer is ‘yes’, ‘certain-no’ responses get the least credit, a ‘not at all certain-no’ receives somewhat more credit and ‘certain-yes’ responses receive the most credit. A similar procedure was used to calculate composite confidence scores for the distractor trials. The two scores were then averaged to obtain an overall composite confidence score.

3.3.1. Pre- versus post-training comprehension performance

To assess the effects of training, ANOVA were conducted on percent correct, average composite confidence (the average of the composite confidence scores to target and distractor trials), target composite confidence and target reaction times with the within-subject variable test (pre-training and post-training), the between-subjects variables gender (male and female) and counterbalancing list (1–6). The same pattern of results was observed for all dependent measures: a significant main effect of test session indicating increased accuracy and faster reaction times following training. However, there were no main effects of gender or counterbalancing list and no significant interactions with these variables.

To summarise, prior to training, mean comprehension was only 68% correct, but increased to 91% correct following training, F(1,88) = 291.35, p < 0.01, partial η² = 0.77, whereas the mean average composite confidence score increased from 8.0 to 9.5, F(1,88) = 512.10, p < 0.01, partial η² = 0.85, and the mean target composite confidence score increased from 7.3 to 9.5, F(1,88) = 415.22, p < 0.01, partial η² = 0.83. Reaction times to targets were reduced from 4.9 s, prior to training, to 2.5 s, after training, F(1,88) = 364.46, p < 0.01, partial η² = 0.81. Therefore, consistent with our earlier research (Lesch 2003, 2004, 2008a, 2008b; Lesch et al. 2011), accident scenario training resulted in improved accuracy, greater confidence in responses and decreased reaction times to warning symbols. One caveat is that the current design did not allow for the assessment of an effect of re-testing. However, in earlier research using a similar training paradigm, it was demonstrated that there was substantial improvement in performance beyond that which could be attributed to repeated testing alone (Lesch 2003, 2008a).

3.3.2. Symbol characteristics

Comprehensibility versus complexity. The accident scenario training significantly improved the comprehension of the warning symbols; however, of greater interest, is the effect of symbol characteristics (complexity and comprehensibility) on initial comprehension and learning of warning symbols by older adults. For these analyses, we focused on target composite confidence because distractors could be relatively easily rejected due to their relative implausibility. Consequently, the distractor trials would be expected to dilute any effects observed. In addition, the predictions regarding the effects of symbol characteristics on reaction times are unclear because increased complexity might be expected to lengthen initial visual processing, whereas increased comprehensibility might be expected to shorten the decision stage. Therefore, changes in reaction times as a function of symbol characteristics are not considered here.

In our earlier study (Lesch et al. 2011), it tended to be the case that comprehensibility was highly negatively correlated with complexity – therefore, we had to rely on regression analyses to try to determine whether the high comprehensibility or the low complexity of our items predicted higher comprehension rates. This study overcame this issue by systematically altering symbols in order to break down that negative relationship – components were added to the existing symbols that should increase both complexity and comprehensibility, such that there would be a positive relationship between the two. Note that it was hypothesised by Lesch et al. (2011) that older adults should have greater difficulty with more complex symbols due to age-related declines in selective attention and inhibitory efficiency. However, if an advantage of comprehensibility is observed for this set of symbols (which were also high complexity), it would suggest that the nature of the additional detail determines whether increased complexity is detrimental or beneficial to older adults' comprehension – that is, if the additional details provide ‘cues to knowledge’, older adults' comprehension should improve as a result of the increased complexity.

It was hypothesised that prior to training, the most complex symbols would produce higher target composite confidence scores than low and medium complexity symbols because increased complexity for these symbols is the result of additional contextual cues – cues which should facilitate comprehension. Alternatively, it may be the case that the increased visual complexity would inhibit comprehension of the symbols – that is, the increased complexity may prevent, or override, a benefit of increased comprehensibility. Lesch et al. (2011) found an effect of comprehensibility that decreased as complexity increased.

Training would be expected to increase the familiarity of the symbols as well as to strengthen the association between the symbols and associated information. Therefore, it may be the case that training would eliminate any benefit associated with cues. Alternatively, there may be some cues that require training in order to be effective (i.e. ‘to make the connection’ between the symbol and long-term memory representations).

A significant main effect of comprehensibility–complexity condition (low, medium and high) indicated that mean target composite confidence increased with increasing comprehensibility and complexity, F(2,87) = 98.87, p < 0.01, η² = 0.69 (see Table 2). A significant test × comprehensibility–complexity condition interaction, F(2,87) = 94.85, p < 0.01, η² = 0.69, reflected smaller differences between low versus high and medium versus high comprehensibility–complexity conditions at post-training than at pre-training, t(99) = 11.35, p < 0.01 and t(99) = 8.96, p < 0.01, respectively. Prior to training, high comprehensibility–complexity symbols received significantly higher mean target composite confidence scores (8.3) than did medium comprehensibility–complexity symbols (6.4), which, in turn, received significantly higher mean target composite confidence scores than did low comprehensibility–complexity symbols (5.9), all ps < 0.05. Following training, high comprehensibility–complexity symbols received higher target composite confidence scores (9.8) than did low (9.1) or medium (9.4) comprehensibility–complexity symbols, t(99) = 5.56, p < 0.01 and t(99) = 3.11, p < 0.01, but low did not significantly differ from medium, p>0.05. The mean target composite confidence scores significantly increased for all three comprehensibility–complexity conditions following training, all ps < 0.05, see Table 2. There were no significant main effects of, or interactions with, gender or counterbalancing list. Therefore, these variables were not included in subsequent analyses.

Table 2 Mean (SE) target composite confidence scores as a function of comprehensibility–complexity condition and test.

	A	B	C
	Low–low	Medium–medium	High–high	Mean
Pre-training	5.9 (0.2)	6.4 (0.2)	8.3 (0.1)	6.9 (0.1)
Post-training	9.1 (0.1)	9.4 (0.1)	9.8 (0.1)	9.4 (0.1)
Mean	7.6 (0.1)	7.9 (0.1)	9.0 (0.1)

Type of contextual cue. The members of warning concept triplets differed not only in terms of comprehensibility and complexity but also in terms of how increased comprehensibility and complexity were implemented – that is, in terms of the types of contextual cues that were added to increase comprehensibility and complexity. To examine whether some cues were more effective than others in improving comprehension, target composite confidence scores were subjected to an ANOVA with the within-subject variables test (pre- vs. post-training), cue type (verbal/symbolic, colour, human form, general contextual and multiple) and cue presence (absent vs. present). Twenty-one warning symbol concepts contributed to these analyses since the remaining two did not vary contextual cue. Table 3 shows the warning concepts contributing to comparisons across cue type. As can be seen in Table 3, most comparisons were made across symbols that only differed in the relevant cues. For example, for cancer-causing substance, B is compared with A to assess the effect of a verbal cue since these two symbols only differ in the presence of the letter ‘C’ – that is, A is the same as B, visually, except that it has been rotated in an attempt to remove the verbal cue. To test the effect of a body cue, C is compared with B because they only differ in that cue (and share the verbal cue). If body cues are effective in increasing comprehension, then it would be expected that symbols that contain these cues will have greater comprehension scores than symbols that do not include these cues. This prediction is indicated in the table as C>B. Note, however, that comparison of C with A does not provide a test of a verbal or body cue as C differs from A in terms of both these cues. Therefore, this comparison is classified as testing the effect of multiple cues. However, it was sometimes the case that the symbols differed in other respects, in addition to the presence/absence of cues. This is particularly the case for triplets involving a shift from a relatively abstract representation (or relationship) to a more concrete representation – for example danger of avalanche and no entry. These shifts tended to be coded in terms of the addition of ‘multiple cues’ since the most comprehensible–complex symbols contained several of the cue types, even though the least comprehensible–complex symbol might not share its basic form. Although these classifications were corroborated by the experts' judgements, it remains the case that these comparisons may be qualitatively different from those simply involving changes in cues.

Table 3 Warning concepts contributing to comparisons over cue type with predictions.

Warning concept	Low (A)	Medium (B)	High (C)	Verbal	Body	General context	Colour	Multiple
Cancer-causing substance				B>A	C>B			C>A
Crush hazard				B>A	C>B			C>A
Corrosive substance					B>A	C>B		C>A
Danger of avalanche					B>A		C>B	C>A
Electric fence					B>A	C>B		C>A
Entanglement hazard					C>A
Eyewash				C>B
Fire alarm								C>A
Handle with care					B>A	C>B
Hot surface					C>A		B>C	B>A
Inhalation hazard					B>A		C>B	C>A
Keep frozen				C>A
No entry					C>B			C>A
No standing						B>A C>A C>B
Oxidising chemical				C>B			B>A	C>A
Pressurised system					C>B	B>A		C>A
Protect from heat						B>A	C>B	C>A
Radiation					B>A	C>B
Read instructions				C>B		B>A		C>A
Safety shower				C>B	B>A			C>A
Shock hazard					B>A C>A

There was a significant main effect of test, F(1,65) = 232.14, p < 0.01, η² = 0.78, indicating higher mean target composite confidence scores following training (mean = 9.4 and SE = 0.17), than prior to training (mean = 6.7 and SE = 0.13), and significant main effects of cue, F(4,260) = 7.01, p < 0.01, η² = 0.10 and cue presence, F(1,65) = 97.17, p < 0.01, η² = 0.60, indicating that symbols containing contextual cues received higher mean target composite confidence scores (mean = 8.5 and SE = 0.14) than symbols not containing contextual cues (mean = 7.7 and SE = 0.11). There was also a significant cue type × cue presence interaction, F(4,260) = 3.71, p < 0.01, η² = 0.05 and a test × cue type × cue presence interaction, F(4,260) = 5.21, p < 0.01, η² = 0.07. Planned comparisons indicated that prior to training, symbols containing human form cues, general contextual cues, colour cues and multiple cues received higher target composite confidence scores than symbols without those cues, all ps < 0.05. Following training, only symbols containing general contextual cues and multiple cues received higher mean true composite confidence scores than comparable symbols which did not contain those cues, p < 0.05. Verbal cues did not provide any significant benefit before or after training (Figure 6).

Figure 6 Mean (SE) target composite confidence scores as a function of cue type, cue presence and test.

To examine whether some cues were more effective than others in improving comprehension, true composite confidence difference scores were computed between symbols in which a cue was present and comparable symbols without the cue. These difference scores were subjected to an ANOVA with the within-subject variables test (pre- vs. post-training) and cue type (verbal/symbolic, colour, human form, general contextual and multiple).

There was a significant main effect of test, F(1,65) = 60.18, p < 0.01, partial η² = 0.48, indicating that symbols with cues demonstrated a greater benefit over symbols without cues prior to training (mean = 1.3 and SE = 0.13) than after training (mean = 0.3 and SE = 0.07). There was also a main effect of cue type, F(4,260) = 5.58, p < 0.01, partial η² = 0.08 and a significant test × cue type interaction, F(4,260) = 3.68, p < 0.01, partial η² = 0.05. Planned comparisons indicated that, prior to training, human form cues provided greater benefit relative to verbal cues and multiple cues provided greater benefit than verbal cues, general contextual cues and colour cues, all ps < 0.05. Following training, human form cues, general contextual cues and multiple cues all provided greater benefit than colour cues, all ps < 0.05. It was also the case that human form, colour and multiple cues provided a greater benefit prior to training than after, all ps < 0.05.

All cue types, with the exception of verbal cues, provided a significant benefit in terms of improved comprehension. Therefore, we decided to take a closer look at the warning concepts which varied in the availability of verbal/symbolic cues. Seven warning symbol concepts incorporated verbal cues (cancer-causing substance, keep frozen, oxidising chemical and read instructions) or symbolic cues (crush hazard, eyewash and safety shower) (Figure 6). At pre-test, a planned comparison indicated that there was no advantage for the cancer-causing substance symbol containing the letter ‘c’ as compared with the symbol that was rotated in an attempt to remove the cue. The symbol that placed the pre-existing cancer-causing substance symbol within a human form produced a significantly higher mean composite confidence score (mean = 5.0) than the rotated symbol (mean = 2.9) or the symbol containing the verbal cue (mean = 3.5) (p < 0.05). It was expected that the symbol containing the verbal cue would perform better than the rotated symbol; however, the difference was not statistically significant. One possibility is that some participants detected the ‘c’ within the rotated symbol and were able to use it as a cue, despite the rotation, thereby reducing the observed difference between the two symbols. Although the pattern was as predicted following training, there were no statistically significant differences.

For keep frozen, the inclusion of the verbal cue produced a significantly higher mean target composite confidence at pre-test (mean = 6.8) relative to the same symbol without the cue (mean = 5.0) (p < 0.05). However, there was no significant difference following training.

For oxidising chemical, the inclusion of the verbal cue did not produce a significantly higher mean target composite confidence at pre-test (mean = 6.7) relative to the same symbol without the cue (mean = 6.3), but did following training (9.7 vs. 8.8; p < 0.05), suggesting that, in this instance, training was required to ‘make the connection’.

For read instructions, there was no significant difference between the symbol with the verbal cue and the symbol without (these two symbols also contained an additional contextual cue in the form of a wrench). However, it was the case that the symbol with no additional cues performed better than those with additional cues. One possibility is that, not only is there no benefit of additional cues when the symbol is relatively unambiguous, but additional detail may actually be a source of confusion/distraction in these instances. This is consistent with the hypothesis that older adults may have greater difficulty in processing symbols when they contain irrelevant details (details that do not serve to further refine the meaning of the symbol).

For crush hazard, there was no significant difference between the symbol with the symbolic cue (in this case, a directional arrow) and the symbol without.

Both the eyewash and the safety shower symbols incorporated a cross (‘+’) as an indicator of ‘first aid’. The ‘+’ aided comprehension for eyewash prior to training (p < 0.05), but not after, and did not significantly improve comprehension for safety shower either before, or after, training.

To summarise, benefits of verbal/symbolic cues were not reliably observed across warning concepts. In some instances, the cues appeared to require training in order to be beneficial – that is, for ‘ox’ to benefit comprehension of oxidising chemical, one must know, or be able to guess, what ‘ox’ stands for. If the participants were unfamiliar with the concept of ‘oxidation’, then the accident scenario would help fill-in gaps in their knowledge and enable them to form the link between the verbal cue ‘ox’ and the warning concept oxidising chemical. Therefore, some verbal cues may be less transparent than others and require training to be fully effective. We also suggested that verbal cues could be a source of distraction or confusion when the symbol is already relatively unambiguous.

4. General discussion

This study was designed to tease apart the effects of comprehensibility and complexity on older adults' comprehension of warning symbols by manipulating the relevance of additional information (i.e. complexity) in further refining the meaning of the symbol. Symbols within warning concept triplets were systematically altered such that increased visual complexity (in the form of contextual cues) resulted in increased comprehensibility (as determined by expert ratings). It was found that high comprehensibility–complexity symbols were better understood (i.e. produced significantly greater target composite confidence scores) than were low- or medium-comprehensibility–complexity symbols. Therefore, it is the nature of the additional detail that determines whether increased complexity is detrimental or beneficial to older adults' comprehension – that is, if the additional details provide ‘cues to knowledge’, older adults' comprehension improves as a result of the increased complexity.

Training interacted with comprehensibility–complexity condition such that, prior to training, target composite confidence scores significantly increased with increasing comprehensibility–complexity but, following training, only high comprehensibility–complexity symbols received higher target composite confidence scores than the other conditions. However, it was the case that mean target composite confidence scores significantly increased for all three comprehensibility–complexity conditions as a function of training.

We were also interested in whether different types of cues would be differentially effective and whether contextual cues might interact with training – that is, do some cues require training to be effective? Or does training eliminate any benefit associated with some contextual cues? Prior to training, all cues except verbal cues provided a benefit relative to symbols without those cues. However, following training, only symbols containing general contextual cues and multiple cues received higher mean true composite confidence scores than comparable symbols, which did not contain those cues. It is unclear why there were differential effects of training for the general contextual cues and the multiple cues. However, as noted earlier, the comparisons for the multiple cues were sometimes qualitatively different from the other cues because they sometimes represented a shift in the basic form of the symbol (and not just the addition of cues). General contextual cues formed a sort of catch-all category for cues that were not easily classified into one of the other categories. However, they often took the form of an object that could be acted upon (e.g. the addition of the box to protect from heat was intended to indicate that the box should not be allowed to be heated by the sun and the addition of the wrench to read instructions was intended to indicate that the instructions would provide information about assembly of a product, using a wrench). It may be that cues that indicate these sorts of relationships are more informative than other types of cues. Future research should investigate this possibility as well as alternative classification schemes. In any case, training eliminated the benefit associated with some types of cues, but not others. It also appears to be the case that the even greater level of complexity achieved through the addition of multiple cues was insufficient to be detrimental to older adults' comprehension – despite much greater complexity, comprehension was significantly improved. Again, it appears to be the nature of the additional information that is critical, not the degree of complexity.

Closer examination of the symbols that incorporated verbal cues (the only cue type that did not produce a significant benefit in the form of improved comprehension) showed that some of these cues were only beneficial following training (‘ox’ for oxidising chemical), suggesting that some cues require training for older adults to ‘make the connection’. Other verbal cues, however, appear sufficiently transparent, so as not to require any training (e.g. keep frozen). We also speculated that, in some instances, the addition of a verbal cue was unnecessary and perhaps confusing (e.g. read instructions). However, additional research is needed with a larger symbol set to further test these hypotheses.

There are several limitations to this study. First, we were unable to use our participants' ratings of familiarity, comprehensibility and complexity because there was evidence of a bias in their responses such that high comprehensibility items tended to be considered both highly familiar and very simple, despite clear indicators otherwise. A related finding was reported by Forsythe, Mulhern, and Sawey (2008) who found that familiarity and learning influenced subjective complexity ratings for nonsense shapes – as familiarity increased as a function of learning, rated complexity decreased. Consequently, we decided to rely on trained experts' ratings of complexity and comprehensibility to classify symbols and experts made their judgements in the context of warning concept triplets (i.e. judgements could be considered relative to other members of the triplet). However, we could not use experts' ratings for familiarity, because familiarity is clearly dependent on an individual's experience, rather than on characteristics inherent in the symbol. The pattern of results, however, suggests that participants were not highly familiar with the symbol set because comprehension was relatively poor prior to training, but significantly improved following training. Training serves to increase familiarity with the symbol and to clarify the connection between the symbol and its real-world referent.

Another limitation is that we only studied older adults. However, our earlier study had indicated a different pattern of results as a function of age group (younger vs. older) such that older adults' comprehension, both before and after training, was influenced by comprehensibility and complexity, whereas younger adults' comprehension was influenced by comprehensibility prior to training, but not after. This study aimed to increase our understanding of the role of comprehensibility and complexity in older adults' comprehension of warning symbols and found that comprehension can be improved by the inclusion of contextual cues in symbols that would otherwise be low comprehensibility. However, we have no direct evidence that younger adults' comprehension would also benefit from inclusion of contextual cues.

It should also be noted that the method of obtaining comprehensibility ratings (i.e. first providing the meaning and then asking experts how likely they would be to guess that meaning) might have introduced a ‘hindsight’ bias – the finding that participants tend to assign a higher likelihood of occurrence to outcomes that they have been told are true (Fischoff 1975). However, it seems unlikely that this bias, if present, would operate differentially across the different symbols within a given warning concept triplet.

The results of this study clarify the roles of complexity and comprehensibility in comprehension of warning symbols by older adults. Although earlier research indicated that older adults had greater difficulty with high complexity and low comprehensibility symbols (Lesch et al. 2011), we found that it is the nature of the added complexity that is critical – increased complexity aids older adults' comprehension if the additional information (in this case, contextual cues) serves to further refine the meaning of the symbol. If the additional information is redundant or unnecessary in further refining the meaning of the symbol, it may be distracting or confusing to older adults (as appeared to be the case with read instructions). It should also be noted that there are instances, in which greater detail and complexity can be detrimental to legibility – for instance, when there is limited space and the symbol requires reduction in size. This is an important concern for older adults.

In another study of the effects of sign characteristics on training effectiveness, Chan and Ng (2010) failed to observe effects of familiarity, concreteness, simplicity, meaningfulness and semantic closeness. However, their study differed from ours in significant ways: (1) participant population (younger adults vs. older adults), (2) type of training and (3) method of assessing comprehension. Another potential difference lies in the definition of constructs. In this investigation, comprehensibility is seen as distinct from the abstract–concrete dimension, in which the comprehensibility of abstract items can be increased through the addition of contextual cues that do not necessarily increase the concreteness of the symbol. Also, the method of obtaining ratings can influence results. We found that our participants' ratings of familiarity and complexity were influenced by the comprehensibility of the symbols. Forsythe, Mulhern, and Sawey (2008) noted a confounding of complexity and familiarity in image processing. McDougall, Curry, and de Bruijn (1999) found, using 239 icons from various sources, that icon familiarity, concreteness, meaningfulness and semantic distance were all closely inter-related. The strengths of this study include the minimising of bias through the use of expert raters and the disentangling of the effects of complexity and comprehensibility.

In conclusion, our results indicate that symbols should be designed to be comprehensible as possible (i.e. to have a high probability of having its meaning guessed) and that, in some instances, added complexity can increase comprehensibility. We found that several different types of contextual cues could be used to improve older adults' comprehension and that these cues were effective even without training. However, particular abstract concepts (e.g. keep frozen and cancer-causing substance), which are difficult to represent symbolically, may need to rely on verbal cues, which is the only cue type that did not receive strong support from the results of this study. A closer examination of individual symbols suggested that the verbal cues varied in the extent to which they required training to be effective. Therefore, as a first line of defense against comprehension difficulties, symbols should be designed so as to incorporate contextual cues to their meaning; however, care should be taken to ensure that the additional information is necessary and not redundant. Cues should help to make the new knowledge ‘old’, or already known, by providing a clearly recognisable and familiar piece of information that relates the new information (the warning symbol) to already known information in long-term memory. In some instances, training may be required to make the connection.

Acknowledgements

The authors thank Peg Rothwell for her assistance in modifying existing symbols and in manuscript preparation. The authors also thank YoonSun Choi for her assistance in data processing and analysis, Luci Simmons for her assistance at various stages and Marvin Dainoff, Angela DiDomenico and Michelle Robertson for their comments on earlier versions of this manuscript.