A critical approach to evaluation

Abstract

Within information systems, the question of evaluation remains as a major contentious issue, and perhaps even more so in the field of health informatics where the traditions of medicine meet and mingle with the information systems field. A review of the literature in these areas indicates that there is little agreement on the essential role of evaluation, a ‘best way’ to evaluate, on what and how to evaluate, whom to involve and within what paradigm to proceed. Reflecting on discussion within both traditions, this paper develops an approach to evaluation broadly based on critical theory, and argues that such an approach, while not offering a solution to all the problems of evaluation, does bring into focus fundamental questions relating to evaluation process and content. To illustrate this, the paper reflects on the strengths and weaknesses of an evaluation of a primary care intranet, and suggests how a critical approach might lead to a more meaningful evaluation and provide more significant and useful findings.

Keywords:

• Evaluation
• health information systems
• critical theory

Notes

1 In the UK family doctors, known as general practitioners (GPs), provide primary care. Usually, they form partnerships of between 4 and 10 doctors, and operate as doctors practices from local premises in the high street. A small and declining number of practices are operated by a single person. GPs are the first point of contact for people's health care needs, and they act as gatekeepers to the health care system, making references onward to specialist clinics and hospitals as needed.

2 More recently, since the study reported here PCGs have evolved into bigger and more powerful administrative groups – primary care trusts – PCT.

Additional information

Notes on contributors

Ela Klecun

About the authors

Ela Klecun is Lecturer in Information Systems at the London School of Economics. She holds a Ph.D. in telehealth from the London School of Economics. Her research interests include health information systems and telehealth, evaluation of information systems (IS), social exclusion in information society, sociotechnical approaches and application of critical theory in the field of IS. Currently, she is involved in two funded projects: evaluation of electronic prescribing (with Tony Cornford) and research into how people encounter e-illiteracy and how they can take action to overcome it.

Tony Cornford

Tony Cornford is Senior Lecturer in Information Systems at the London School of Economics. His research interests focus around the implementation of information systems within organisational contexts and the working out of socio-technical accommodations. Among his current research activity is work on open source software process, underlying models of e-government, an electronic prescribing systems in hospital settings. The second edition of Project Research in Information Systems: A Student's Guide (with Steve Smithson) will be published by Palgrave later this year.