inPractice: a practical nursing package for clinical decisions

Abstract

This paper examines the recent development of a computer-assisted learning program—in Practice—at the School of Health Science, in the University of Wales Swansea. The project, which began in 2001, was developed in close collaboration with The Meningitis Trust, the aim being to produce a software package to increase nursing students' knowledge of meningitis-related illnesses, and to enhance their decision-making and problem-solving skills by using lifelike scenarios. It incorporates two multimedia meningitis modules incorporating the use of text, film, and sound, in which students are presented with information about the illness (symptoms, treatment etc.), and are required to use their knowledge to make decisions at various key points. A general discussion of decision-making theories and CAL design principles is presented, which has provided a foundation for the main design aspects of the package. This is followed by an outline of how the program was created to promote students' application of knowledge and their decision-making and problem-solving skills. Results from an evaluation questionnaire are presented. Consideration is also given as to how the program can be extended.

Introduction

The ability to reach sound, effective clinical decisions is paramount in nursing practice. This is reflected in the considerable research literature on issues such as the factors that contribute to effective clinical decision-making, and how it can be improved and evaluated (see literature reviews by Taylor [2000] and Simpson and Courtney [2002]). Emphasis in this context has been placed on the use of CAL materials for which there are many of the usual reported advantages (learning flexibility, immediate feedback, active student participation, stimulation of motivation, etc.), and, in the particular case of medicine, the ability to practise making potentially life-critical decisions for patients in a safe environment (Hooper & Toye, 1975; Barker, 1989; Whiting, 1989; Aitken et al., 1997; Brain et al., 1999).

CAL materials in medicine range from relatively simple electronic textbooks to multimedia applications using text, still images, film, and sound. inPractice, a custom-developed CAL package funded by The Meningitis Trust and developed at the School of Health Science, University of Wales Swansea, is an example of the latter. It has been in development since 2001, and incorporates two meningitis-related scenarios that are regularly used by nursing students to complement their traditional training. The design, construction, and information for the scenarios were achieved with the help of experienced clinicians, nursing lecturers, and others with a relevant background. The principal aim of the package is to promote effective decision-making with respect to meningitis by placing student nurses in a simulated but highly realistic environment. This paper outlines the development of inPractice, and some of the lessons learnt since the project began.

Background

There are two principal bodies of research work associated with clinical decision-making. One tends towards a theoretical approach to analysing the cognitive steps associated with how and why a decision is made. The other leans towards evaluating the quality of decisions made (Taylor, 2000; Denig et al., 2002; Dowding & Thompson, 2003). In developing a CAL package that would help to promote effective decision-making and problem-solving skills in nursing education, consideration was given to both these approaches, but particularly the former during the design stage.

Decision-making strategies and evaluation

Tennyson et al. (1987) draw a distinction between three main categories of thinking strategy—recall, problem-solving, and creativity. These range from low-order, automatic recall of existing knowledge to a more complex higher-order creation of knowledge. Similarly, Bonk and Smith (1998) and Girot (2000) outline the importance of developing critical thinking (incorporating higher-order and decision-making skills) and problem-solving skills to enhance logical reasoning abilities and to promote effective decision-making in nursing. In an extensive review of the literature concerning how individuals make decisions in a nursing environment, Taylor (2000) suggests that the process of clinical decision-making involves three key stages: a decision with respect to which observations should be made about a patient in a given situation; derivation of meaning from the observed data; and selection of a course of action to be taken for the optimal benefit of the patient. Likewise, Schacter et al. (1999) comment on how good learners and good problem-solvers spend a considerable amount of time digesting a problem and associated information before breaking it down. Indeed, it is generally acknowledged that the ability to be consistent and effective in decision-making and problem-solving involves an ability to digest information, and to make a detailed breakdown of the problem at hand, before coming to an informed logical decision on the best course of action. Furthermore, evidence from studies examining the use of problem-based learning initiatives points to numerous advantages including enhanced teamwork, a greater sense of learning enjoyment, and a better understanding of the subject (Biley & Smith, 1998; Antepohl & Herzig, 1999; Chang, 2001). Consequently, it is plain that a CAL package aimed at improving decision-making and problem-solving skills needs to do more than simply prescribe information. Students should be required to use their subject knowledge to make effective decisions during a given problem-based scenario, and this is, of course, vital in a nursing environment.

As regards evaluation of the decisions made, there has been a considerable amount of research relating to various analytical methods. These include decision trees, Bayes' Theorem, Utility Theory, Brunswik's Lens model, and inter-judge comparisons (Lamond & Farnell, 1998; Taylor, 2000; Denig et al., 2002; Dowding & Thompson, 2003). Such approaches have attracted some criticism given the fact that nursing decisions cannot be easily quantified and evaluated since they rarely involve straightforward either/or choices (Taylor, 2000), but it is clear that evaluation techniques need to be adopted in order to provide some indication as to the effectiveness of training (Berner, 1999, p. 132). It was nevertheless thought that the application of techniques such as these would be best suited for a post hoc assessment of whether the use of inPractice had improved decision-making and problem-solving skills (see later).

The design team had to examine how all the aforementioned factors could be translated into an appropriate CAL package. Crucially, in order effectively to complement factual material given in lectures, the content would allow students to study the procedural aspects of meningitis diagnosis and treatment, such as how the illness is detected, various treatment options and the types of decision that would have to be made by those involved. The program would be used as a facilitator for existing lectures at the School, and as a tool with which to prescribe the necessary information and to manage and monitor student learning outside the classroom environment. To achieve this, different general CAL and problem-based learning design principles were considered. As outlined by Tennyson et al. (1987), there are three main design considerations for a package of the inPractice type: (1) to provide the necessary (basic) knowledge and concepts of the subject; (2) to present a problem scenario where students' proposed solutions are evaluated before the next iteration is presented; and (3) to provide an appropriate learning environment and the promotion of group work where possible. Similar guidelines can be found in Tennyson (1988), Barker (1989), Rendas et al. (1999), and Maudsley and Strivens (2000). Berner (1999) and Grabinger et al. (1997) also stress the importance of presenting problems that are meaningful, and giving students a reason to want to use computer technology as a part of their education.

Taking such factors into account, it was decided to develop inPractice as a tutorial package, but one that would incorporate elements of assessment and decision-making. The program would also provide students with feedback on the quality of their decisions. Provision for delivering the appropriate learning environment (e.g. encouraging group work) and for developing relevant and interesting scenarios was also considered. These issues are discussed in more detail in the following section.

The design of inPractice

General design aspects of inPractice

As noted by Laurillard (2002, p. 134), tutorial-based CAL packages contain an explicit teaching strategy. For inPractice, the primary objectives have been to enhance students' knowledge of meningitis, and to incorporate an element of decision-making and problem-solving. In order to develop a package appropriate to these needs, considerable emphasis was placed on design aspects such as providing a clear outline of the learning objectives, obtaining regular input from students to gauge their progress, and the delivery of multimedia material.

Program implementation

After the completion of beta testing to elicit usability and content issues (see later), second-year and third-year nursing students at the school were assigned one-hour practical classes where inPractice was (and still is) used. Since, from their first-year study, these students already possessed a degree of knowledge with respect to the basic principles of nursing practice as well as the relevant IT skills, it was believed that they would easily acclimatise to the content of inPractice. The aim of the practical classes was to provide students with an introduction to the program, and to give assistance on any technical issues that might arise. Each scenario requires around one hour to complete, and the students were encouraged to use the program in their own time in addition to the timetabled classes.

Scenario construction

There are two meningitis scenarios in inPractice. One deals with meningococcal disease, and the other with pneumonia and secondary meningitis. The eLearning team at the school has been continuously developing such scenarios since 2001. They deal with various aspects of how cases of meningitis can develop and how they can be treated. Each scenario integrates combinations of textual information and video clips (including interleaved sound) of around two to three minutes, and each shows the development of a patient's condition. The dialogue requires a response from the student before the next set of information is presented. In collaboration with The Meningitis Trust, the scenarios were constructed with the help of experienced health professionals who worked closely with the design team and nursing lecturers in order to ensure that the material was both accurate and relevant for student nurses.

Scenario presentation

Each scenario begins by presenting a preliminary set of basic information about the case in point (e.g. the age of the patient and relevant history, and the symptoms) and about the proposed learning outcomes. The program also provides general introductory information about the case (see Figure 1), and a video clip illustrates the current condition of the patient. Lohman and Finkelstein (2002) have shown that students are better able to solve problems when presented with scenarios segmented into comparatively small sections, as compared with delivering large amounts of information all at once. The decision was therefore made to split each scenario into separate sections. Each scenario contains around 16 pages covering key aspects of the case such as diagnosis, treatment, and the recovery process.

Figure 1 A typical introductory screen in an inPractice scenario

Presentation of alternatives

Once a set of information has been presented, the student is given either a summary of patient information (requiring limited or no feedback from the user), a question based on the current situation (see Figure 2 for an example of a typical question screen), or a decision point (Figure 3) before the next iteration of the scenario is displayed. It is well known that there are numerous problems related to methods of retrieving input from students in order to measure their learning progress (see Laurillard, 2002, pp. 134–138). The best method for inPractice was considered by both content and program designers to be the use of multiple-choice questions to measure students' basic knowledge of key meningitis facts. The main reasons for this decision were that it would allow the program to provide an immediate response as to correct and incorrect answers without the complexities involved with open-ended questions, and that some evidence exists to suggest that students prefer multiple choice over questions requiring a discursive response (Reid & Arends, 1998). The questions, and the correct and incorrect answers, were written by the relevant lecturers and health professionals. Feedback and explanation of correct and incorrect answers are given as soon as the student has submitted a choice.

Figure 2 A typical question screen in an inPractice scenario

Figure 3 A typical decision screen in an inPractice scenario

Each scenario contains two critical decision points. For example, in the first scenario, the student is required to decide on the course of action for each observation interval of the patient where various pieces of information are given (such as blood pressure, heart rate, and body temperature). Two options are provided, namely to continue with the observation (in this case, the correct answer) or to inform medical staff of what the student perceives as significant changes in the observations. If the correct answer is given at each interval, an explanation of the choice is provided—in this instance giving reasons why it is necessary to continue observations or inform medical staff. If an incorrect choice is made, the scenario develops accordingly—in this case, the patient suffers if the student fails to inform medical staff at the appropriate interval. The student is then given an explanation of the outcome, and prompted to revisit the decision point of the scenario to correct the original decision.

Individual study/group work

Once the students have been presented with the question or decision point, they are encouraged to use their knowledge of the subject to come to an informed and educated response. They are helped by the program's built-in glossary and definitions of various clinical conditions and terminology, and can call on lecture notes, textbooks, as well as learning via group work with other students. The work of Tennyson et al. (1987), Tennyson (1988), and Maudsley and Strivens (2000), who argue that promoting group work is beneficial in terms of enhancing clinical judgement and effective decision-making, was taken into account in this context, and students are strongly encouraged to work in groups.

Consequence of the decision/end of scenario

As already mentioned, feedback is given to students throughout each stage of the scenario, with correct or incorrect answers highlighted with explanations. Key decision points allow the student to make the wrong decision, and see the consequences, before being given the opportunity to correct the decision. At the end of each scenario, a summary of the learning outcomes is presented, and students are urged to consider further reading material and information (descriptions of various diseases, disorders, terminologies, etc.) built into the program. Figure 4 shows the general structure of a scenario.

Figure 4 Structure of inPractice (diamond, information given by the program; square, student activity; circle, application of knowledge)

Implementation study

The actual implementation of inPractice in 2002 was relatively trouble-free (apart from the all too common difficulty of the scheduling and organisation of practical classes). General usability issues such as interface and navigation were also unproblematic—the vast majority of students had sufficient IT knowledge to use the program competently (see the feedback analysis later).

As for content, the most significant design aspect of inPractice is that of enabling students to apply their existing knowledge to make decisions in various clinical situations. As opposed to CAL packages that prescribe information, scenarios in inPractice present information in the form of details of a situation, just as a nurse or other health professional would have in real life. While students are given the option to refer to terms and definitions, the design of the program is centred on stimulating active participation, which requires self-study in order to answer questions and to make the correct decisions at key points. Students are clearly shown the consequences of making an incorrect decision, and how it affects the patient, a feature that has already received positive feedback from both students and lecturers. Indeed, initial indications from students, lecturers, and health professionals generally show very positive attitudes towards the program. Encouragingly, lecturers have observed repeated and frequent use of it by students in their spare time (some have even enquired about obtaining their own version for use at home, and even at the ward during placement), something that, as highlighted by Finlay (1989, pp. 149–155), is often a factor that represents a meaningful measure of success.

Before inPractice was incorporated into the course, a beta test was conducted to gather feedback in a practical session where the package was used by a group of 35 students for one hour in order to provide a basis from where content and usability issues, if any, could be raised. This gave sufficient time for students to study and complete at least one of the two scenarios. None of the students had used inPractice before, but all had experience with using computers and existing CAL packages at the school. A lecturer and IT supervisor were present to provide content and technical assistance. Students were given a brief introduction, and asked to work through the program either individually or in small groups.

After the session, all users completed a brief questionnaire designed to obtain preliminary feedback of general usability and content issues. All but one respondent were female, and between the ages of 18 and 35 years. The majority (80%) were enrolled on the Nursing Diploma course and most had achieved at least GCSE and A-level qualifications. The main questions were constructed using five-point Likert scales. The respondents were then asked to participate in a short focus-group session to discuss the salient points in greater detail. To make the numbers more manageable for discussion, students were split into two groups, and interviews were conducted in separate rooms by an interviewer. The following is a discussion of the key findings obtained in the questionnaire and focus groups.

Usability

Users were first asked how well they rated the package overall. Promisingly, more than 88% of respondents (31 out of 35) rated it as ‘very good’ or ‘excellent’. Only three considered the program to be average, and on inspection of their written comments it appeared that possible reasons might include the construction of the text and video clips, which some users found a little too long. Four further questions were asked in order to examine usability issues in greater detail. Two of these questions related to navigation and ease of use. There were positive results for both, with the majority of respondents either agreeing or strongly agreeing with the statement that it was easy to navigate around the program (30 students; 88%). All respondents either agreed or strongly agreed that the program was easy to use, and comments supported this result.

Content

The first two questions in this section examined the decision-making aspects of the program. Students were asked whether they found the decision points realistic, and if they appreciated having to consider different options. The responses were generally positive about the use and design of decision points within the program; 76% indicating that the decision points seemed realistic, and 94% stating that they liked being given the opportunity to make decisions at critical points in the scenario. Of significance were comments given during discussions in the focus group, which included:

The program really made me think. I enjoyed having the opportunity to make decisions I would be expected to make when I qualify.

Please can we have more lessons like this where you have to think, and are easy to relate to practice!

A very good way of learning—it's good to make decisions and to be given the rationale as to whether they were wrong and why.

I think that this program is extremely good for student nurses allowing them to make their own decisions.

One respondent, however, suggested:

Perhaps some more interactive points/decision-making demands, e.g. what do you think should happen next?

Twelve further questions were asked to get an indication of whether students felt that their knowledge of meningitis issues had been enhanced by the program. The questions ranged from recognising the symptoms of meningitis, and the importance of keeping a close observation of the patient, to the pathophysiological processes associated with pneumococcal meningitis and the ongoing management and care of a patient recovering from the illness. An analysis of the results showed that the vast majority of users felt their knowledge of these various aspects of meningitis had improved since using the program, and even some respondents with practical meningitis-related nursing experience said they had learnt something new about the subject. While this result must be considered tentative as regards the true effectiveness of inPractice in terms of knowledge enhancement, the initial signs are indeed promising. Feedback from a control group will be gathered in the next research phase in order to establish the validity of the results so far produced.

Comments from the focus groups gave some useful indication as to how they felt the program compares with traditional lectures, and confirm some of the initial advantages expected from the use of CAL:

Learned more from the program than from lectures.

You could learn at your own pace.

You could use the program to read around the subject and use the reference list provided.

Conclusions and further work

The results are reassuring. While there is room for improvement in some aspects of the program's design (such as interface and decision points—suggestions that will be taken into account during further development), it appears that its initial implementation and general aspects can be considered successful. More importantly, students felt positive about the content, and in particular about the decision points, which helped them carefully to consider and apply knowledge in relevant scenarios. A more detailed and longer-term evaluation of the package is currently being undertaken in order to determine its effectiveness in terms of knowledge enhancement as compared with traditional lectures, and in particular whether students feel more confident about making clinical and ethical decisions with the help of inPractice and its problem-solving and decision-making approach.

There is, in any event, certainly scope for further scenarios to be written to cover a wider range of clinical situations, and since the multimedia design aspects of inPractice play a crucial role, research is continuously being undertaken to investigate the suitability of various ways of delivering information (e.g. by examining the effectiveness of different filming perspectives for video clips). However, since each scenario demands considerable resources (information from experts, filming, scenario design and construction, and ultimately financing), there is a limit as to how many of these can be developed in a reasonable time-frame and within a restricted budget.

At present, each scenario offers only two key decision points. Nevertheless, inPractice shows that there is significant potential for the development of an extended CAL package incorporating a much greater number of decision points. The structure of such a package is likely to resemble that of modern-day educational and video games, as discussed by Leary (1995), and briefly by Laurillard (2002, p. 143) where the number of decisions to be made is enormous, and their consequences immediate to the user. Given that the use of game-based approaches in the context of teaching has so far been relatively scarce, there appears to be some potential for development in this area.

Furthermore, while inPractice seems to help promote decision-making and problem-solving skills, the design team feels that it could also be usefully modified and extended into a more rigorous evaluation tool. A possible idea is that it would be used as a general introduction to typical clinical scenarios to provide students with a feel for the type of clinical decision-making situations they are likely to encounter in real life. The structure of the program, which includes the use of multiple-choice questions and decision points, is well suited for beginners (such as first-year students) and those with a limited nursing background for them to develop knowledge-application and decision skills. For more advanced users (such as final-year degree students), a more demanding version of the program could be developed to test for empirical evidence as to whether or not users have improved in their decision-making and problem-solving skills. A student could, for instance, be presented with symptoms of an illness, from where he/she would be required to ask a hypothetical patient an initial question (‘where does it hurt’, or ‘how long has it been hurting for?’), before the system branches into a decision point at each subsequent stage. Developed with formal application of decision-evaluation techniques in mind, a more complex evaluation package could probably incorporate such analyses to good effect. Applied in conjunction with inPractice, such a program would provide a useful evaluation tool to deliver a holistic delivery and assessment of nursing education and training.