RAMES – Framework supporting user centred evaluation in research and practice

ABSTRACT

One way to ensure good user experience of IT systems is to conduct user centred evaluation, aimed to provide feedback to IT professionals on their IT systems from the user perspective. The objective of this paper is to explore a conceptual framework, named RAMES that supports evaluators when planning, comparing and documenting user centred evaluations in a structured way. The framework structures the elements of an evaluation into five categories: Roles, Activities, Materials, Environments, and System. The framework was developed based on a theoretical analysis. Two explorative studies with 92 participants in total show that two thirds of the participants found it easy or rather easy to use the framework, and 33% find it likely that they would use the framework again. The conclusion of the studies is that frameworks such as RAMES enhance the implementation of user centred evaluations.

KEYWORDS:

• Usability testing
• heuristic evaluation
• walkthrough evaluations
• laboratory experiments
• user studies
• user centred design

Acknowledgements

The authors would like to thank Dr Silvia Abrahão at the University of Valencia, Dr Gilbert Cockton at Northumbria University, Dr Effie Law at the University of Leicester and Dr Kristina Höök at the Royal Institute of Technology, for giving valuable comments to the paper in different stages. We also would like to thank the COST IC0904 action named Twintide for supporting short term scientific missions for working on this paper. Additionally, we would like to thank all the students that took part in the explorative study, particularly Axel Máni Gíslason, Gadidah Margrét Ögmundsdóttir and Haraldur Andri Stefánsson that provided us with the cases of using the RAMES framework.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Jan Gulliksen http://orcid.org/0000-0002-2411-6417