Visual map and instruction-based bicycle navigation: a comparison of effects on behaviour

Abstract

Cycling with a classic paper map was compared with navigating with a moving map displayed on a smartphone, and with auditory, and visual turn-by-turn route guidance. Spatial skills were found to be related to navigation performance, however only when navigating from a paper or electronic map, not with turn-by-turn (instruction based) navigation. While navigating, 25% of the time cyclists fixated at the devices that present visual information. Navigating from a paper map required most mental effort and both young and older cyclists preferred electronic over paper map navigation. In particular a turn-by-turn dedicated guidance device was favoured. Visual maps are in particular useful for cyclists with higher spatial skills. Turn-by-turn information is used by all cyclists, and it is useful to make these directions available in all devices.

Practitioner Summary: Electronic navigation devices are preferred over a paper map. People with lower spatial skills benefit most from turn-by-turn guidance information, presented either auditory or on a dedicated device. People with higher spatial skills perform well with all devices. It is advised to keep in mind that all users benefit from turn-by-turn information when developing a navigation device for cyclists.

Keywords:

• Navigation
• spatial ability
• map
• ageing
• cycling

1. Introduction, Part A. Navigation

1.1. Paper and electronic map navigation

For a very long time navigating in traffic in unfamiliar environments could only be realised by consulting a printed map, either by the drivers themselves or by a passenger. This was the standard way of navigating, not only in cars, but also on bicycles. Only in the 90s of the previous century navigation with ego-position indication (egocentric maps, e.g. Crundall et al. 2011) became available to the public at large thanks to GPS (Global Positioning Systems). Nowadays satellite navigation systems have become almost a standard feature in cars, and these systems allow for giving turn-by-turn directions. Another development since the turn of the century is smartphone navigation. Instead of a stand-alone navigation device, route information is displayed on the phone, and provides route guidance for car drivers, cyclists and pedestrians.

The question arises whether these new navigation devices support cyclists better than a paper map. Do people make fewer errors, do they appreciate the way information is presented, and is there an optimal way to present route information? Also, differences in preferred information presentation between for example different age groups may be present.

1.2. Spatial knowledge acquisition

One of the issues that has received attention in the past is the relation between spatial abilities and navigation performance (Rodes and Gugerty 2012). Spatial ability refers to the ability to represent, transform, generate and recall symbolic, non-linguistic information (Linn and Petersen 1985). Spatial ability is needed to translate the two-dimensional map image into a three-dimensional one in order to navigate successfully. With sufficient spatial ability, or sufficient spatial knowledge acquisition (Huang, Schmidt, and Gartner 2012), one can still find one’s way when navigating in conditions where the system fails (e.g. freezes or runs out of battery).

Siegel and White (1975) (see also Huang, Schmidt, and Gartner 2012; and Löchteveld 2014) distinguish three levels of spatial knowledge: 1. Landmark knowledge, 2. Route knowledge and 3. Survey knowledge. Landmark knowledge refers to knowledge about noticeable objects (landmarks) at fixed locations in the environment. Route knowledge refers to a sequence of landmarks when following a route. Survey knowledge refers to the ability to integrate landmarks, distances and routes from different experiences into a mental map. This should enable people to find new routes that have not been taken before and that do not depend on landmarks (Löchteveld 2014). In general a sequence is planned and this sequence is monitored, i.e. compared with landmarks passed. Spatial ability is important for this. Normally navigating with electronic devices leads to good route knowledge, but to less satisfactory survey knowledge (Krüger, Aslan, and Zimmer 2004; Münzer et al. 2006). Using paper maps however, leads to better survey knowledge and very good route knowledge. Having to encode and transform the information from a paper map may be the reason for this (Münzer et al. 2006).

With electronic maps drivers make less navigation errors and reach their destination faster (Lee and Cheng 2008). An important difference between paper maps and electronic maps is that track-up (or heading up) maps adapt what is displayed to the position of the user and the direction of travel (e.g. Burnett and Joyner 1996). This is beneficial for navigation performance, as Rodes and Gugerty (2012) report that high levels of spatial ability can aid effective navigation with a track-up map. With regard to effects of age, studies have shown that spatial ability and navigation may decrease with increasing age (Moffat 2009; Techentin, Voyer, and Voyer 2014). Therefore, when studying effects of navigation aids it is useful to consider the age of the user.

The use of navigation devices on bicycles is likely to differ from the use of these devices in a car. This applies to the physical position where the navigation device is placed. According to Dancu, Zlatko, and Fjeld (2014) a map projection on the road in front of the wheel with a viewing distance of 2.5 metres would be the optimal location to display navigation information. This is close to the normal field of view and moreover, it enables fast focussing and requires little head tilting. Most standard commercial devices are however attached to the handlebar. How a navigation device is used also differs. Use of a navigation devices on a bicycle is also different from use in a car as on a bike these are likely to be more frequently used for recreational purposes than for business use, for example for finding the location of a meeting. Smartphone navigation on a bicycle may introduce dual task performance issues similar to those found while reading or writing text messages on a smartphone while cycling (Ahlstrom et al. 2016; De Waard, Westerhuis, and Lewis-Evans 2015; De Waard et al. 2010, 2014; Goldenbeld et al. 2012; Kircher et al. 2015). Smartphone use while cycling is particularly mentally demanding (De Waard et al. 2014) and often the increased demands of a secondary (telephone) task are met by slowing down (e.g. De Waard et al. 2010). A reduced cycling speed could therefore reflect that navigation demands are high.

1.3. Present study and aim

In the present study, navigating with electronic navigation devices was compared with navigating from a paper map in terms of differences in subjective effort and performance. Included are a comparison of the effects of whole route vs. turn-by-turn information presentation, and effects of modality of information presentation. The systems are first evaluated in terms of ergonomic properties and are presented in section 2.

2. Part B. Evaluating navigation technology

2.1. Devices

Four types of navigation support were compared in the present study: (1) A paper map, (2) A moving map displayed on a smart phone (Google Maps™ in visual format), (3) Auditory route guidance (Google Maps™’ auditory output) and (4) A dedicated system with flashing lights (The Hammerhead™). The paper map was a print out of the route created by Google Maps on A4 size paper, put in a holder attached to the handlebar (Figure 1(a), (b)). In the moving map condition Google Maps was used in default setting, the way it is probably used by most people, which meant the programme zoomed in and out of the map depending on location (Figure 1(c), (d)). However, in this condition auditory feedback was switched off. In the Auditory condition only auditory route information was made available (directions and street names), the telephone was put upside down in the holder. Either earbuds or a loudspeaker was used. The Dedicated lights device (Figure 1(e), (f)) is a T-shaped device that leaves out all textual information. Bright flashing coloured LEDs that are clearly visible even in bright sunlight indicate direction; colour and frequency of flashes change when the location of a turn is approached.

Figure 1. Devices included in the study. (a) Paper map setup and (b) example of the route that had to be followed. (c) Moving Map (visual), (d) route guidance information as displayed. (e) Dedicated lights (Hammerhead™) indicating a right turn. (f) indicating an upcoming right turn (picture taken from https://hammerhead.helpshift.com/a/hammerhead-one/?s=light-patterns&f=what-does-each-light-pattern-mean where all patterns are shown).

2.2. Ergonomic considerations

For in-vehicle systems such as radios and navigation systems, design recommendations have been made by the American National Highway Traffic Safety Administration (NHTSA 2010) and by the European Human Machine Interface Task Force (Godthelp et al. 1998). According to these guidelines, primary task performance (i.e. driving), should not be impaired by the secondary task. While operating a navigation system, the driver has to be able to steer the car with at least one hand, have the ability to interrupt the navigation task at any time, have clear view of the display, and the driver’s main focus should be on the road. As display design and driver’s focal point are inherently related, these two recommendations have been tested as a separate entity in automobiles (Broström, Bengtsson, and Aust 2016). However, as individual differences in glance duration have been found, a change of these guidelines is expected. Therefore, objective design factors have not yet been included. Moreover, these guidelines do not include recommendations for the use of these devices on bicycles.

Nevertheless, Human Factors guidelines for visual displays provide recommendations that can be applied to bicycle navigation systems. The Visual Display guidelines are organised into five themes: basics, legibility, understandability, organisation and content (Green et al. 1994).

(1)	Basics: the information displayed is to be kept at the absolute minimum necessary. Adding information should not lead to a reduction in the size of the items. The display should be in the line of sight, which should be realisable by attaching navigation devices to the handlebar.
(2)	Legibility: Legibility is influenced by the size of the display; a larger display contains larger, and therefore more readable, letters. As a minimum the text should be 6.4 mm in height. Character height is set by the so-called Bond Rule, according to which the characters’ height divided by the viewing distance should be at least 0.007 radians. Following this rule the text height should be 6.4 mm at a distance of 0.9 m. The distance between head and handlebar is likely to be smaller than 90 cm, thus the legibility criterion is met if the letters are at least 6.4 mm in height. Other factors that contribute to legibility are contrast and colours. In order to minimise glare, light characters should be used on a dark background. The colours of the characters should be differentiable. Further, if used in different light conditions, these conditions have to be taken into account. Thus, the display should be adaptable in luminance to match changes in lighting environment.
(3)	Understandability: Only commonly known abbreviations and symbols should be used, such as ‘ave’ for avenue and an arrow to follow the indicated direction.
(4)	Organisation: Elements that should be perceived together should be placed next to each other, according to the proximity principle. This effect can be emphasised by matching the colour and background colour of those elements. Hierarchical order is generally expressed by placing the most important information on the upper left side of the display.
(5)	Content: Drivers using a navigation system are most likely not familiar with the area and the street names, therefore the navigation system should display the names identical to the street signs. For example ‘Friesestraatweg’, instead of its numerical code, in this case N370, should be displayed. In cases where both pieces of information can be found on street signs, both should be indicated by the navigation system. Lastly, distances should be shown accurately and should be updated as the driver moves. The driver needs to be able to recognise the environment on the display as the environment outside the vehicle.

It is important to state that these guidelines for in-vehicle navigation are limited to information display format, not content. The user should be able to read the information. What is displayed however, is not part of these guidelines.

A study by Lavie, Oron-Gilad, and Meyer (2011) supports these guidelines. Moreover, they investigated the levels of abstractness and aesthetics that influence the perceived usability of electronic maps. They found that more abstract maps were rated as more aesthetically pleasing and also had shorter search times. They conclude that drivers have less difficulty finding their way with abstract maps including simple directional information. When providing more detailed information, a more detailed map with landmarks is needed to ensure good performance. In line with Lavie, Oron-Gilad, and Meyer (2011), Sonderegger and Sauer (2010) also found a halo effect of aesthetics, or in other words ‘what is beautiful, is good’. Participants indicated how they liked the design of a product and claimed to like the most appealing one for its usability, when in fact they made more mistakes with this product than with less appealing products. This line of study shows that users may prefer the product that is not optimal for their current needs and favour a more ‘beautiful’ product.

Three of the four systems evaluated in this study were scored on Human Factors guidelines based on the General Guidelines for Visual Displays (Green et al. 1994); The Moving Map, The Dedicated lights, a T-shaped (potentially aesthetically appealing) display that gives directions through LED light indication (Figure 1(e), (f)), and the classical paper map. The auditory version of Google maps is omitted from this comparison as no information is displayed there.

In Table 1, points were assigned to the devices with respect to the earlier mentioned Human Factor guidelines. In the bottom row, the score for each device is presented. Moving Map has 23 out of 40 possible points, whereas the Dedicated lights received 15 points. Comparing the scores of both devices, it can be seen that the Dedicated lights device omits many recommended elements, as indicated by the score 0. Yet, this device excels at the requirements that it does meet, as indicated by the maximum score of 5. Because the Dedicated lights completely omits any map features such as street names and the maps itself, the categories of understandability, organisation and content scored zero. However, the contrast of colourful LEDs against a black background fully satisfies the criterion of Contrast. Also the Minimum Information criterion was met by showing directions in the line of sight.

Table 1. Moving Map, Dedicated lights and paper map compared with a point system (0-40) on Visual Human Factors Guidelines (based on the General Guidelines for Visual Displays, Green et al. 1994).

	Moving map	Dedicated lights	Paper map
Basics
Line of sight	5	5	5
Minimum of information	3	5	1
Legibility
Letter size	2	0	3
Contrast	3	5	3
Understandability
Abbreviations	5	0	5
Organisation
Hierarchical order	5	0	0
Content
Street names	5	0	5
Movement	5	0	0
Total score	33	15	22

Notes:

5 = criterion fulfilled, 1 = somewhat fulfilled, 0 = not applicable.

The rating was performed by the authors.

The Moving Map follows the recommendations to a larger extent, yet there is space for improvement. Assuming that the distance head-handlebar is 0.5 m on average, the letters are too small for a, at present common, 4.6 inch smartphone screen. The letters would therefore have to be increased in size by 2 mm in order to meet the Bond rule. Further, white letters on a green background do not create optimal contrast. In order to achieve such contrast, a darker colour, or shade of green, should be chosen. On the other hand, Moving Map displays the directions in an hierarchical order, presents the street names and adapts the map to the user’s movement in direction and speed. All three devices require a similar line of sight that Dancu, Zlatko, and Fjeld (2014) indicate as ‘Phone View’.

3. Method

3.1. Participants

Thirty-one participants in two age groups were recruited via advertisements and word of mouth. The two age groups were restricted to cyclists aged 20–30 years and those aged 60+ years. Data for one of the participants were not analysed as the equipment failed in all conditions. A total of 30 participants completed the tests; 16 participants were classified as ‘young’ with an average age of 23.1 years (SD 2.0, range 21–28, 9 males) and 14 participants were classified as ‘older’ (average age 68.6, SD 4.3 years, range 60–75, 5 males).

3.2. Procedure, conditions and material

Upon arrival at a sports hall nearby the experimental location, participants provided informed consent and completed a questionnaire on demographics. They received information on the four conditions and devices they would cycle with, and completed the Paper Folding Test (Ekstrom et al. 1976, see measures).

The Dedicated lights device, being the device none of the participants had experience with, was mounted on the handlebar of the participant’s own bicycle. On the route to the starting point, a ride of about 10 min, participants could experience the way information was displayed on this device.

A within-subjects design was applied; all participants completed all four conditions. Their bicycle was equipped with the different devices and a Contour +2 GPS camera (see also Figure 1). The four conditions were balanced in order across participants, but the order of the four routes was the same for all. Conditions are displayed in Table 2. In none of the conditions the participants programmed or operated the device themselves. A Sony Xperia Z1 Compact (Android version 5.1.1) was used for the Moving Map and auditory conditions. In the auditory condition the phone was attached to the handlebar upside down, so only auditory information was available. If participants were used to using their earbuds while cycling, they were allowed to use these. In other cases the loudspeaker function of the phone was used. In the Moving Map condition the auditory information was turned off. No internet connection was available in the Moving Map and auditory conditions in order to prevent the software from recalculating the route. In the Dedicated lights condition the device was connected via Bluetooth to an iPhone6 (iOS 9.3.2) with internet connection. On the telephone the Hammerhead app (version 2.2.0) ran. For all conditions the routes had been prepared beforehand on a computer using Google Maps (https://maps.google.com) and were imported in gpx format.

Table 2. The four conditions and type of information provided.

Condition		Turn-by-turn	Preview route	Ego position
1 Paper map	Paper map [reference]	No	Yes, full	No
2 Moving map	Moving map	Partly	Limited	Yes, visible
3 Auditory	Auditory guidance	Yes	No	Yes, used
4 Dedicated LEDs	Dedicated lights (LEDs)	Yes	No	Yes, used

Each route was about 2 km in length (1.6–2.3 km, see Figure 1(b) for an example) and contained between 10 and 14 turns. For safety reasons the experiment was performed in a quiet neighbourhood in Groningen (Vinkhuizen). For some of the analyses, such as speed, tracks of 160 m were selected where participants received navigation instructions while cycling. On these tracks it was made sure that their course and speed would not be affected by external factors such as the presence of other traffic participants. At the end of the experiment participants were debriefed and received a € 10 voucher for their participation. Ethical approval for the study was obtained by the Ethical Committee Psychology of the University of Groningen.

3.3. Measures and analyses

3.3.1. Paper folding test

The Paper Folding Test (Ekstrom et al. 1976) is a 20-item test that measures spatial ability. Participants have to mentally fold and unfold pieces of paper that were punched. The end score is calculated by subtracting 0.2 times the number of incorrectly answered items from the number of items answered correctly, leading to scores between −4 (all items wrong, low spatial ability) to +20 (all correct). The test was completed in 2 × 3 min.

3.3.2. Errors

The route contained many turns allowing for error analyses. A test leader followed the participant at about 10 m. distance and noted missed and wrong turns taken. Errors were assessed over the whole route. In case of an error that was not fixed by the participants themselves, the test leader would intervene.

3.3.3. Speed

Speed was calculated on the basis of GPS data of the Contour +2 ™ camera. As a back-up the participant also carried a GPS logger (iGotU GPS tracker GT120) storing the same GPS data. This was only used in case of camera storage failure. Data were exported in gpx format with the Contour programme StoryTeller (version 3.6.2.1043) and were imported in Excel 2010. As sometimes roads had to be crossed and the experiment was carried out on a public road, there was no control over the presence of other traffic participants. Sometimes these had right of way. To minimise this influence, speed was calculated on the earlier mentioned tracks of 160 m where participants received one directional message.

3.3.4. Eye fixations (frequency & duration)

The camera was directed towards the participant’s face (see Figure 1). On the same tracks as described under speed, eye fixations in the direction of the device were scored. This was done by importing the video into the programme ELAN (version 4.9.4). Fixations at the device were manually marked in this programme, and the number of fixations and the duration of fixations were calculated.

3.3.5. Subjective mental workload

At the end of each route a test leader would await the participant. Participants were then asked to rate the amount of invested mental effort during the ride on the Rating Scale Mental Effort, RSME (Zijlstra 1993). The RSME is a unidimensional scale that consists of a line running from 0 to 150 with anchor points denoting the amount of effort required at irregular intervals . A rating of 2 coincides with ‘no effort’, 37 with ‘some effort’, 85 ‘great effort’ and 112 ‘extreme effort’ while the scale runs beyond this point up to 150. Participants gave a rating by marking a point on the line.

3.3.6. User experience and preference

At the end of each condition participants filled out the User Experience Questionnaire (Schrepp, Hinderks, and Thomaschewski 2014). This scale contains 26 items with bipolar labels, such as ‘obstructive-supportive’ and ‘difficult to learn-easy to learn’, all on a 7 point scale. Items load on six subscales: attractiveness, perspicuity, dependability, efficiency, novelty and stimulation. Lastly, participants were asked which navigation device they preferred.

3.4. Analyses

IBM SPSS Statistics 20 was used for statistical analyses. Analyses concentrated on the differences between conditions and the two age groups. A Chi-square test was conducted for comparing proportions of the age groups in terms of their experience with the devices and how often they cycle. Either repeated measures ANOVA or MANOVA (when sphericity was violated) was used to compare performance with the different devices.

4. Results

For some participants data were missing. The Dedicated lights device did not work three times, data collection in the auditory condition did not work for two participants and data registration in the Paper map condition failed for one participant. Data of at least 24 (13 young, 11 older) participants were complete.

4.1. Demographics

All participants of both age groups cycled more than once per week, while more than 50% of them cycled on a daily basis. Frequency in which they used a smartphone differed between groups, all younger participants used their smartphone every day, eight out of 14 older cyclists used a smartphone more than once per week, but six of the older participants never used a smartphone (, p = 0.002). More older than younger participants had no experience with using a smartphone to navigate; 12 (86%) older versus 4 (25%) of the younger participants (, p = 0.004). The majority of participants never used a paper map to navigate while cycling; (9 young (56%) and 10 older participants (71%), , p = 0.047).

4.2. Paper folding test

Scores on the Paper Folding Test can range between -4 (low spatial ability) to +20 (high spatial ability. Younger participants had a higher average score than older participants, 11.8 (SD 2.9) vs. 4.5 (SD 3.4), (F(1,28) = 40.72, p < 0.001,  = 0.593).

4.3. Errors

Most errors were made in the Paper Map and Auditory conditions (differences between conditions: (F(3,75) = 6.20, p = .001,  = 0.199). In these conditions the test leader also had to intervene most frequently. Post-hoc pairwise comparisons revealed that the number of errors made while cycling with the Moving Map was significantly lower than in the Dedicated lights and auditory guidance conditions. However, compared to the paper map condition no differences were found (Table 3). No age effects in average number of mistakes were found (F(1,25) = 0.420, NS,  = 0.017).

Table 3. Per condition and age group the average (Ave) number of mistakes and stops to look at device or map and the number of times the test leader had to intervene. SD is indicated in brackets.

	Ave. no. mistakes			Ave. no. stops		Ave no. interventions
	Young	Older	All	Young	Older	Young	Older
Paper map	0.75 [0.93]	1.86 [1.46]	1.27 [1.31]	0.44 [0.89]	1.21 [2.08]	0.75 [1.07]	1.79 [1.42]
Moving map	0.06 [0.25]	1.36 [2.85]	0.67 [2.02]	0.00 [0.00]	1.43 [4.33]	0.06 [0.25]	1.00 [1.96]
Auditory	1.44 [1.26]	1.15 [1.28]	1.31 [1.26]	0.56 [1.03]	0.00 [0.00]	1.44 [1.37]	0.92 [1.04]
Dedicated LEDs	0.80 [0.68]	0.46 [0.66]	0.64 [0.68]	0.27 [0.59]	0.00 [0.00]	0.73 [0.70]	0.54 [0.66]

4.4. Speed

Average speed on the selected representative 160 metre tracks, where speed was not affected by other traffic and where one instruction about an upcoming turn was given, did not differ significantly between conditions ((λ = 0.805, F(3,19) = 1.53, NS,  = 0.195). The difference in cycling speed between the age groups did not reach the level of statistical significance (F(1,21) = 3.79, p = 0.065,  = 0.153, Figure 2).

Figure 2. Average Speed per condition in km/h on the 160 m selected track. Error bars reflect Standard Error.

Notes: N = 23 . PaperMap = Paper Map, MovingMap = Moving Map (Visual info displayed on a smartphone), Auditory = only the Auditory route information, and DedicatedLEDs = Dedicated lights device.

4.5. Eye fixations (frequency & duration)

Eye fixations at the device were scored from a video of the selected 160 m tracks. Differences in the mean number of fixations at the device did not differ significantly on the 5% level between the two age groups (F(1,22) = 3.46, p = 0.076,  = 0.136, Figure 3). Between conditions, differences in mean number of fixations were found (F(3,66) = 41.16, p < 0.001,  = 0.652), post hoc pairwise comparisons revealed that these were due to not looking at the device in the Auditory guidance condition.

Figure 3. Mean number of fixations at the device on the selected 160 m track. Error bars reflect SE.

Notes: PaperMap = Paper Map, MovingMap = Moving Map (Visual info displayed on a smartphone), Auditory = only the Auditory route information, and DedicatedLEDs = Dedicated lights device.

Apart from the number of fixations, glance duration was also studied, as was the total percentage of time looking at the device. Average duration per fixation is shown in Figure 4. Again no differences between the age groups were found (F(1,22) = 1.53, NS,  = 0.065). Significant differences in duration were found between conditions ((λ = 0.251, F(3,20) = 19.87, p < 0.001,  = 0.75). Post hoc tests showed that all conditions, with the exception of Moving Map and the Dedicated lights condition, differed significantly (p < 0.05). Average fixation duration was longest for the paper map condition.

Figure 4. Average duration of fixations in seconds per condition (Error bars reflect SE).

Notes: PaperMap = Paper Map, MovingMap = Moving Map (Visual info displayed on a smartphone), Auditory = only the Auditory route information, and DedicatedLEDs = Dedicated lights device.

Percentage of time looking at the device (and thus not looking at the road) was also determined (Figure 5). Again no main effect of age was found (F(1,23) = 2.40, NS,  = 0.09). With regard to differences between conditions (λ = 0.105, F(3,21) = 59.41, p < 0.001,  = 0.895), only looking behaviour in the Auditory condition differed significantly from the other three conditions. Overall the percentage time spent looking at the map or device is around 25%.

Figure 5. Percentage of time looking at the device or map.

Notes: Error bars reflect SE. PaperMap = Paper Map, MovingMap = Moving Map (Visual info displayed on a smartphone), Auditory = only the Auditory route information, and DedicatedLEDs = Dedicated lights device.

4.6. Subjective mental workload

The ratings of the young participants were not statistically significantly higher than those from the older participants (F(1,22) = 3.50, p = 0.075,  = 0.137, Figure 6). Differences between conditions were however found (λ = 0.417, F(3, 20) = 9.33, p < 0.001,  = 0.583), with the paper map condition being rated as more mentally effortful than all the other conditions. Ratings between those other conditions did not differ from each other.

Figure 6. Average ratings on the Rating Scale Mental Effort.

Notes: Error bars reflect SE. PaperMap = Paper Map, MovingMap = Moving Map (Visual info displayed on a smartphone), Auditory = only the Auditory route information, and Dedicated LEDs = Dedicated lights device.

4.7. Preference/evaluation/user experience questionnaire

Multiple comparisons revealed the following differences:

Attractiveness: Paper map scored lower than the other three conditions

Perspicuity: Paper map was rated lower than Dedicated lights and Moving Map, but did not differ from the Auditory condition.

Efficiency: Paper map was rated lower than Dedicated lights and Moving Map, but did not differ from the Auditory condition.

Novelty: All pairs differ significantly, with the exception of the Moving Map and the Auditory condition. The Dedicated lights condition scored highest.

Stimulation: Paper map scored lower than the other three conditions

Total: Paper map scored lower than the other three conditions

Overall, the User Experience Questionnaire shows that the paper map is not as attractive, stimulating or efficient and novel as any of the other modes of information presentation. (Table 4). The Dedicated lights scored high on novelty, as could be expected. It also received the highest average score for attractiveness. None of the participants preferred the paper map as navigation means, whereas the new Dedicated lights proved to be very popular, in particular amongst the older cyclists (two out of three older cyclists preferred this device for navigation). Overall, one out of three participants preferred the Moving Map (Table 4).

Table 4. User experience questionnaire (Schrepp, Hinderks, and Thomaschewski 2014) results and preferred device.

								Preferred device
	Attractiveness	Perspicuity	Dependibility	Efficiency	Novelty	Stimulation	Total	Young	Older
Paper map	−0.06	0.85	1.08	0.32	−1.35	0.13	0.15	0%	0%
Moving map	1.51	1.94	1.79	1.48	0.48	1.34	1.43	38%	29%
Auditory	1.24	1.57	1.21	1.13	0.44	0.97	1.10	13%	0%
Dedicated LEDs	1.74	1.97	1.24	1.43	2.15	1.65	1.70	50%	71%
Statistics	F(3,69) = 16.57, p < 0.001	F(3,69) = 5.38, p = 0.002	F(3,69) = 1.99, NS	F(3,69) = 5.48, p = 0.002	F(3,69) = 49.05, p < 0.001	F(3,69) = 14.92, p < 0.001	F(3,69) = 16.39, p < 0.001

Notes: For the user experience scales average values per scale are shown. All scales have a range from −3 to +3.

4.8. Correlations

Spatial ability as reflected by the score on the Paper Folding Test, was correlated with cycling speed and number of mistakes made. Only in conditions where these skills were required, i.e. the paper map and Moving Map conditions (Table 5), a significant relation was found. Participants with higher spatial abilities cycled faster and made less mistakes. No relation between spatial abilities and either errors or speed was found in the auditory and Dedicated light conditions.

Table 5. Correlation between score on the paper folding test and two performance measures; number of mistakes and speed, in each of the conditions.

Pearson’s r	r (No. mistakes)	p	r (Speed)	p
Paper map	–0.47	0.008	0.47	0.011
Moving map	–0.44	0.015	0.57	0.001
Auditory	0.01	NS	0.16	NS
Dedicated LEDs	0.22	NS	0.20	NS

5. Discussion

In the present study the effects of and opinion on, different bicycle navigation devices were studied. It was further investigated whether performance was linked to spatial ability skills. Results showed that performance on a spatial abilities test correlated with cycling speed and with the number of errors made, but only when navigating with navigation systems that presented a map, not with systems that provided either auditory or visual instructional directions (turn-by-turn information). Not all devices are optimal for all users, and user preferences also differ. Auditory information or devices such as the Dedicated lights device can serve the needs of cyclists who have less spatial skills in particular.

5.1. Age

The effect of age on navigation performance was studied. Although spatial abilities decline with age, age itself was not found to be a factor that had a major influence on performance. However, it should be noted that the older participants were mainly from the younger-older group, a fit group of people who still cycle. On the other hand, that is also the group that is most likely to use navigation devices such as the ones that were evaluated.

5.2. New technology versus a paper map

There are a few important differences when comparing the ‘old fashioned’ paper map to the evaluated modern devices; i.e. Moving Map, auditory guidance and the Dedicated lights. The paper map lacks the property of indicating (or at least ‘knowing’) the ego position and the ability to adapt route information accordingly. Related to this is adapting what is shown or instructed on the basis of the direction of travel, including a ‘track-up’ display of the map. There are also differences between the new devices. A remarkable difference between the Dedicated lights and Moving Map condition is that in the Moving Map condition the users do not only see a map that indicates where they are, but they also receive a preview and information about the surroundings. The track-up function of the map adjusts the cyclist’s position on the map whenever the cyclist moves. The cyclist constantly has to compare the two-dimensional map on the screen to the three-dimensional environment while cycling. On the map street, names are given and the user can find confirmation about whether a turn was indeed correct. The Dedicated lights device does not have either of these functions. Left and right turn indications are given without showing a map. Hence, awareness of one’s position in relation to the environment is not required, instructions can be ‘simply followed’ and this is likely to result in low survey knowledge. Active processing is important, as was shown by Huang, Schmidt, and Gartner (2012) who performed an experiment where pedestrians were guided by advanced systems. Participants did not have to actively process route guidance information in any of the conditions and spatial knowledge acquisition was found to be poor. The Dedicated lights only showed directions by flashing LEDs, there is no hierarchical order of information visible. In the moving Map condition additional information is given in the top of the screen, as can be seen in Figure 1d. Distance to the next turn and street names are provided at the top, and below that the distance to the subsequent turn is shown. However, according to the Human Factors Visual Display Guidelines (Green et al. 1994), this information is not displayed large enough for cycling purposes. The contrast of white letters on a green background is also not optimal. In terms of the legibility criterion (Table 1) the Dedicated lights device performs better than the Moving Map. The auditory device is in many ways similar to the Dedicated lights device, with as major difference the modality of information presentation, and the duration of the availability of information. In sum, the devices differ on different aspects, and they were evaluated mainly as provided, and not systematically by isolating these aspects. Such an evaluation would require a different set-up, a set-up that would probably be easier and better to realise in a laboratory environment.

5.3. Looking behaviour

The initial plan was to assess lateral position on the road. Cyclists who operate their phone have been found to increase safety margins to the curb (e.g. De Waard, Westerhuis, and Lewis-Evans 2015). However, the present study was performed in a quiet residential area on the main road, not on cycle paths. There were many parked cars in this area, making it very difficult to assess lateral position to the curb. As a result, it was decided to point the camera towards the participant’s face and assess looking behaviour, which is also relevant to safety. What is remarkable is that there were no differences in the total time spent looking at the devices that display information on a screen, on a dedicated light device, or on a paper map. However, the duration per glance was highest for the paper map condition. This could be related to the fact that these are allocentric-topographic maps where ego position was not displayed, thus participants first had to figure out where they were. They also had to mentally rotate the map as it was North-up only. Remarkable is that a large amount of time was spent looking at all the devices, overall 25% of the time fixating was spent looking at the maps or at the Dedicated lights, similar percentages as found in car navigation (Burnett and Joyner 1996). However, compared with the study by Ahlstrom et al. (2016) where participants had to look up information on the internet or send a text message while cycling, this percentage is not high, as they found on average 60% of the time was spent looking at the telephone in the self-paced conditions. Ahlstrom et al. (2016) also found that fixations at the phone came at the expense of fixations at irrelevant objects, and therefore conclude that ‘spare capacity’ was used. In this study looking at the navigation device was also self-paced, and a lower total percentage of time fixating was found, as well as a lower average fixation duration of maximum 1.6 s in the paper map condition (and 1.2 s in the Moving Map and Dedicated lights conditions). Ahlstrom et al. (2016) found much higher average fixation durations of 3.35 s when sending text messages or looking up information on the internet. It is however still too early to conclude that navigating with these devices is safe, as the way fixations were assessed differs between studies. In this study only fixations at the device could be assessed, not where people were looking when they did not look at the navigation device. Whether fixations were also performed ‘at the expense’ of looking at irrelevant objects is not certain, but also not unlikely.

5.4. Auditory information and preferences

Less than 5% of time was spent looking at the device in the auditory condition where no information was displayed; participants hardly glanced at the phone, simply because there was not much to see on the device in that condition. The auditory condition was not standardised in the sense that participants could choose to use either earbuds or the loudspeaker. The advantage of using earbuds is that the instructions can only be heard by the person for whom these instructions are intended. However, older cyclists were often not used to using earbuds, and the use of earbuds could also interfere with their hearing aids. Hearing performance declines with age, and therefore providing only auditory information is not optimal for them. Ambient noise may also mask instructions if the loudspeaker is used, and the information has to be processed when presented, while visual information can be consulted when the traffic situation allows (see also Burnett and Joyner 1996). This is all visible from the preference ratings; none of the older cyclists preferred the auditory condition, while 13% of the young cyclists did. The other turn-by-turn information device, the Dedicated lights, was very much appreciated and preferred by all participants, but in particular in the older group. This is remarkable as some have said that older people are hesitant to new technology (Hancock and Parasuraman 1992), although others found that once they have received instructions and have had the time to get used to new technology, it is very much embraced (Yang and Couglin 2014; Yannis et al. 2010). There may also have been a Halo effect, the Dedicated lights are likely to be aesthetically attractive, and as Sonderegger and Sauer (2010) found, ‘what is beautiful, is good’. Apart from aesthetics, the number of mistakes made in navigation was not different from the other conditions, and what is displayed on the device is close to the user’s needs, as information that is not needed is omitted.

5.5. Spatial abilities and reliability of technology

As said, participants with higher spatial skills perform better with a map, while those with lower skills perform as good as those with higher skills with turn-by-turn information. Cyclists with lower spatial skills may not benefit from a preview. They are simply taken by the hand and told where to turn, instead of having or being able to plan a route, or at least checking position on the route (egocentric indication). It would be useful to explore this issue further in a more controlled laboratory environment. Performance with turn-by-turn information can be good, but only if the provided information is accurate. As often with new technology, sometimes there were issues related to newness. In this experiment the Hammerhead (Dedicated lights condition) refused to work with three participants, either due to losing the Bluetooth connection or because an acute software update was needed. If one is cycling and the connection gets lost and the device goes blank, the rider only has information about the next turn, not about what follows beyond that. Low survey knowledge may then lead to problems and one can easily get lost, in particular if the route that has to be followed is not obvious (as in this experiment). With regard to auditory information there were also the occasional issues, such as that information was sometimes inaudible or was issued too late. These are crucial issues, but these are technology/software issues, i.e. growing pains that will most likely decrease in time. Therefore, these conditions were removed from the data analyses in the present experiment, leaving only optimal system performance, enabling the evaluation of behavioural effects. It is also interesting to see that there are many new devices for cyclist navigation being developed, some have entered the marked, such as the Dedicated lights in this study, some are experimental. For example Dancu and colleagues have evaluated a projection system that displays maps on the street in front of the bicycle for night-time cycling (Dancu, Zlatko, and Fjeld 2014), as well as a bike-mounted head-up display (Dancu et al. 2015). In both cases the vicinity of the cyclist is used as display where navigation information is projected enabling integration with a view of the environment. Innovative systems will be developed at rapid pace, and some of these will enter the market. In this study however, the focus was on systems that are already available to the public.

6. Conclusion

Overall, in conclusion, the reference condition, the paper map, was the least favoured of all four conditions. The paper map condition also stands out in terms of requiring high mental effort investment and in this condition the longest fixations were found per glance. When looking at speed and errors, performance was similar in all conditions. The relation between performance and spatial ability is however noteworthy: if spatial ability is high, the extra information on maps, with surrounding information and route previews, coincides with less errors made and a higher cycling speed. If spatial abilities are lower, performance with visual maps is worse, but performance is not deteriorated when turn-by-turn information is presented by lights on a separate device or in auditory format. New navigation devices can make use of this knowledge by always providing turn-by-turn information and by making visual maps available for those who like to have this preview available, i.e. cyclists with higher spatial abilities.

Disclosure statement

No potential conflict of interest was reported by the authors.

Acknowledgements

The authors of this paper would like to acknowledge the support of Daryl Kryst, Maxime Ley and Mengxi Huang during the data collection phase, while Peter van Hunen gave advice with regard to statistical tests. Two anonymous reviewers are thanked for their helpful comments during the publication process.