Evaluating PPGIS Usability in a Multi-National Field Study Combining Qualitative Surveys and Eye-Tracking

Figures & data

Table 1. Objectives and non-objectives of this paper.

Objectives	Non-objectives
Analyse the usability of a globally used PPGIS that is not designed for a very specific use case. Understand how user characteristics are related to usability issues. Based on this, examine an adaptation gradient.	Analyse the usability of a locally used PPGIS that is designed for a very specific use case. Understand the usability issues of all PPGIS. This paper explores the usability of one PPGIS that might or might not represent other PPGIS. Not comparing user performances and user quality, especially not between study areas.

Figure 1. GeoCitizen application.

Table 2. Survey design.

Part 1: Application interactions tasks	Evaluation of error rate, task completion duration and participant’s satisfaction with the following tasks: Enter the application and navigate to the map. Change the basemap. Find a POI and review its description. Leave a comment on a POI. Add a new POI. Find the profile configuration and change the age.
Part 2: Interface design preferences	Choice of preferred interface design of the participants: GeoCitizen interface. Simple interface with a chat wizard. Simple interface without a wizard. Simple map interface with basic functionalities.

Figure 2. Interface alternatives.

Table 3. User profile and number of participants.

Metric	Response	Colombia (n = 30)	Uganda (n = 23)	Austria (n = 20)
Age	18–29 years	16	10	10
	30–65 years	14	13	10
Gender	Female	12	9	11
	Male	18	14	9
Educational level	Secondary	17	8	3
	Undergraduate	9	10	7
	Postgraduate	4	5	10
Self-rated computer skills	I don’t use the computer	–	3	–
	Basic	9	9	4
	Intermediate	15	11	6
	Advanced	6	–	10
Ownership of a smartphone	Yes	30	22	19
	No	–	1	1
Self-rated smartphone skills	I need help	1	3	–
	I can carry out basic tasks	8	11	3
	I am sufficiently able to use it	21	9	17
Frequency of smartphone use	Sometimes	–	3	1
	Every day	3	1	–
	Several times a day	27	19	19
Activities with smartphone	Messages and social networks	30	22	19
	News	20	13	16
	Other applications	24	15	19

Table 4. User groups and number of participants.

Age/IT-literacy	Colombia	Uganda	Austria
18–29 years/medium-high	8	5	5
30–65 years/medium-high	7	5	5
18–29 years/low-medium	8	5	5
30–65 years/low-medium	7	8	5

Figure 3. Error rate (effectiveness).

Figure 4. Interface preference by user group and country.

Table 5. Category of remark and participants' quotes.

Remark:	Participants' Quotes:
78% of all participants (57 out of 73) stated the button meanings not being intuitive enough.	‘I don’t like how the buttons are distributed; I can’t find them right away.’ ‘I don’t understand the meanings of the buttons. I need more time to go and use each button to understand how to use them.’ ‘The button icons don’t represent what the button actually does.’
59% of all participants (43 out of 73) were criticizing the design or some aspects of it.	‘I am having trouble finding the right information. I need a bit more time to locate were to go next.’ ‘I don’t like how the map is designed. It’s difficult to zoom or move on the map.’ ‘There is a lot of information to process.’
32% of all participants (23 out of 73) stated overall problems with the application; 30% of all participants (22 out of 73) commented on needing more guidance or time to learn how to use the application	‘I don’t understand how to use the app right now, but it would be nice to learn how to use it.’ ‘I don’t know at all how to use the app and need somebody to help me.’ ‘It would be good if you could provide more information on how to use the app.’ ‘I need a little bit more guidance to understand the functionalities and what to do with them.’
38% of all participants (28 out of 73) said that they find the application and its information useful.	‘I really like what you can do with the app.’ ‘This could be very useful for my neighbourhood.’
34% of all participants (25 out of 73) liked the design or some aspects of it.	‘I like the colours and the font of the app. It looks nice.’ ‘I like the structure of the app.’ ‘The elements and buttons are placed nicely.’
88% of all participants (64 out of 73) found a map element useful.	‘I think a map element is very useful and I can see right away where other users uploaded their observations.’ ‘I like maps because the information is easy to read.’

Figure 5. Time to first glance and duration of task completion.

Figure 6. Number of fixations.

Figure 7. Mean fixation duration.

Figure 8. Adaptation gradient.