Can the APO method be used for measuring soft data?: A pilot study

Abstract

Objectives. The aims were to develop auditing according to the APO (Audit Project Odense) method for measuring soft data, exemplified by a holistic view, and to test the instrument. Design. A descriptive study of the development of an APO chart and a test registration. Setting. Primary health care, Blekinge County, Sweden. Subjects. Ten general practitioners (GPs) were invited to transform categories of the concept of a holistic view obtained in an earlier study, into 30 variables on an APO registration chart. The participants chose to study different kinds of knowledge as aspects of holistic care. Main outcome measure. An APO registration chart and test of the instrument. Results. After three meetings the group had drawn up an APO registration chart supplemented with Likert scales. A pilot audit was performed. Eight doctors registered 255 consultations. In assessment of the patients’ problems, factual medical knowledge was important in 83% of the cases, familiarity in 53%, and a capacity for judgement in 36%. In decision-making factual medical knowledge was used in 88% and capacity for judgement in 58%. A holistic view was necessary for the outcome in 43% and valuable in 25%. The GPs used the Likert scales in a majority of the cases. Conclusions. In this first step in developing an instrument, the results indicate that the APO method could be an alternative for studying what happens in the consultation, and the occurrence of an abstract phenomenon such as the use of different kinds of knowledge as part of a holistic view.

Key Words::

• APO method
• consultation
• development of measurement instruments
• general practice
• quality development

The APO process is a pronounced bottom-up method for quality development and for influencing medical decision-making in general practice.

• It was possible to use the APO method for measuring soft data, using knowledge to exemplify one aspect of a holistic view.

• Our results indicate that the APO method could be an alternative to time-consuming methods for studying the consultation.

Introduction

The National Board of Health and Welfare was commissioned by the Swedish government in 2002–2005 to follow up the national action plan for the development of medical care in Sweden. This included studying the watchwords of primary care: continuity, accessibility, cooperation, and a holistic view. A group of researchers was commissioned to perform the study. A qualitative focus-group study of the meaning of the concept of a holistic view was performed with general practitioners (GPs) and district nurses [1,2]. According to the participants the term holistic view could be divided into three main categories: attitude, knowledge, and circumstances (Table I).

Table I. Categories and subcategories from the content analysis of the concept of a “holistic view” [2].

Category	Subcategory
Attitude	Professional attitude
	Political/administrative attitude
Knowledge	Factual knowledge
	Tacit knowledge
Circumstances	Motivating factor
	Organization
	Sphere of activity
	Tool

The National Board also raised the question of whether it was possible to measure the prevalence of consultations in which a holistic view was used in the decision-making (Table II). We wanted to test the APO (Audit Project Odense) method, a well-established tool for quality development among Nordic GPs [3–7]. An APO audit is a simple and feasible method for common medical problems, which to our knowledge has never been used for soft data. We chose this opportunity to examine the possibilities and limitations of the APO tool. This participatory bottom-up process is based on a registration of consultations, where each physician registers a considerable amount of consultations during a limited period on a simple registration chart [3]. The participants decide on the variables, test them in a pilot audit, and adjust them for the real registration. Next, each participant receives his/her personal result, for comparison with colleagues in follow-up discussions between professionals.

Table II. A holistic view in primary care – two linked projects.

Earlier study
Strandberg EL, Ovhed I, Borgquist L, Wilhelmsson S. The perceived meaning of a (w)holistic view among general practitioners and district nurses in Swedish primary care: a qualitative study.
BMC Fam Pract 2007;8:8
Period
January 2002	Invitation and start of planning the study
November 2002–March 2003	Seven focus-group interviews with 22 GPs and 20 nurses
August–December 2003	Analysis
August 2007	Publication
Current study
Spring 2006	Planning of the current study
September 2006–February 2007	Initial APO audit brainstorming process
2 weeks in March 2007	Audit registration

The aims of this pilot study were to develop an audit instrument to measure soft data exemplified by the concept of a holistic view, and to test the instrument.

Material and methods

Previously, 22 GPs defined criteria for a holistic view. In this pilot study, all 10 members of a Continuing Medical Education (CME) group were invited to develop an APO audit. Eight of them participated in the first study [1,2]. Their task was to transform concepts based on the earlier results into 30 measurable variables, and to test the registration chart.

All participants were specialists in family medicine with long experience, four women and six men. Eight doctors worked in villages, two in urban settings. All of them worked in clinics with several GPs, each with their own patient list, and they helped each other with patients.

Participation in an APO audit is voluntary and participant-centred. Every process starts with the participants defining central concepts of the topic in order to operationalize them on a registration chart [3]. This process started with a summary of the meaning of the categories from the previous study to reach common ground. The group met three times, led by a person with long experience of constructing APO registration charts.

The pilot audit was conducted in March 2007. The participants were asked to register all patients. The results were compiled in a report presenting the results of the entire group. The report and the final registration chart can be ordered from the corresponding author.

As described in the Results, Likert scales were introduced. The Likert scale assessments were analysed by dividing the 10-grade scale into three main groups (low value 1–3, neutral value 4–7, and high value 8–10).

Results

The registration chart takes form

The first meeting began with a run-through of the findings from the interview study concerning the concept of a holistic view. The participants were encouraged to discuss the concept and how it could be measured. Each person stated his/her opinion about the different categories and how their presence in the consultation could be captured, in accordance with the APO method. The group chose to focus on one aspect of holistic view, the concept of knowledge, and on the presence and significance of knowledge for medical decision-making.

The participants discussed the need for detailed information on the patient. At first, a traditional APO audit was regarded as too blunt to capture the presence of abstract phenomena in the decision-making. Therefore they decided to add three 10-step Likert scales to capture more detailed information: Familiarity with the patient, Symptomatology, and Agreement/Discrepancy concerning the patient's stated reason for the consultation (Figure 1).

Figure 1. The Likert scales.

The scales were regarded as important cornerstones in the practical application of the concept of a holistic view. The third scale was envisaged as a way for doctors to capture the very core of holistic thinking, i.e. to reveal a hidden agenda (if any); in other words, what is initially concealed from both the doctor and the patient.

At the second meeting further definitions of the concept of “knowledge” were added to the concepts in the previous study. The group introduced a subordinate level concerning familiarity. The group discussed the importance of knowledge regarding patients’ networks, knowledge from the primary care team, written and oral knowledge, and the doctor's own “capacity for judgement” inspired by Pörn [8], concerning both well-founded factual knowledge and what the GP has learned from practice related to the particular individual in the actual consultation. The capacity for judgement is thus not transferable from one GP to another. Regarding factual knowledge, the group stressed that in medical contexts this is an essential type of knowledge. Sometimes no factual knowledge is available “here and now”, and the group considered it important to reveal such situations, and situations where factual knowledge had not been used.

At the third meeting the variables were adjusted and instructions for registration were completed (Table IIIa, IIIb).

Table IIIa. Main groups, groups, and variables in the APO registration chart concerning a holistic view in the general practitioner's medical decision-making.

	Groups	Variables	No.
Patient characteristics	Patient	Own patient	1
		Other doctor's patient	2
		No one's/everyone's patient	3
	Problem	Well-defined problem	4
		Difficult to define	5
Assessment	Important type of knowl.	Purely factual knowledge	6
		Familiarity	7
		Knowledge from the patient's network	8
		The capacity for judgement	9
Considered for medical decision	Factual knowl.	ConsideredNot consideredNot obtainable	101112
	Team knowl.	Considered	13
		Not considered	14
		Lacking	15
	District knowl.	Considered	16
		Not considered	17
		Not obtainable	18
	Oral knowl.	Considered	19
		Not considered	20
		Not obtainable	21
	Written knowl.	Considered	22
		Not considered	23
		Not obtainable	24
	Capacity for judgement	ConsideredNot consideredNot obtainable	252627
Evaluation	Significance of holistic view	NecessaryNot necessaryValuableNegative/added nothing	28293031

Table IIIb. Definitions: The participants’ explanations of the meaning of the different types of knowledge assessed in the pilot audit.

• Purely factual knowledge:The patient's problem requires factual medical knowledge.

• Familiarity:The overview and interpretation of a situation is not always fully underpinned by medical knowledge. The patient's problem requires familiarity (sometimes expressed as tacit and implicit knowledge), which is acquired through extensive experience.

• Knowledge from the patient's network:The patient's problem requires knowledge about and from the patient's network, family, and social surroundings (may have reached the doctor in different ways, but) is (regarded as) essential for helping the patient.

• The capacity for judgement:The patient's problem requires that you use your personal judgement, sometimes in contradiction to EBM, accepted practice, or your own former experience, since you believe that this is best for your patient.

The pilot audit

Eight of 10 doctors handed in completed charts after two reminders. The GPs had registered a total of 255 consultations with an average of 32 per doctor. The distribution was between 25 and 52 (median = 30) consultations per doctor. Of the 255 consultations, 60% were with a patient from the doctor's list.

In the assessment of the patient's problem (criteria 6–9), the doctors regarded special factual medical knowledge as important in 83% of the cases, familiarity with the patient in 53%, and the capacity for judgement in 36%.

In the decision-making (criteria 10–27), factual knowledge was considered in 88% of the cases and the capacity for judgement in 58%. In contrast, knowledge obtained from the primary care team, orally from colleagues or others, and from written sources had been considered in the decision-making only to a limited extent, from 11% to 28%.

Evaluating the patient's problem, the doctors assessed a holistic view (criteria 28–31) as necessary for the outcome in 43% of the consultations and valuable in 25%. In 31% of the cases a holistic view was not regarded as necessary. A holistic view was never considered negative.

In 88% of the consultations the doctors had filled in the Likert scale concerning familiarity with the patient, in 85% the scale concerning symptomatology, and in 85% the scale concerning the stated and the ultimate reason for the visit.

Of all the patients, 47% were well known to the doctors. Some 15% had a complex symptomatology, and in 3% of the consultations the GP changed opinion regarding the main problem during the consultation. In a subset of the consultations when GPs met patients on their own lists only and completed the Likert scales, 72% of the patients were well known, 18% had a complex symptomatology, and in 2% of the consultations the doctor changed view on the reason for the visit.

Discussion

This pilot study concerns the possibilities and limitations of the APO method. In this first step in developing an instrument, we wanted to examine whether it is possible to measure abstract phenomena in GPs’ clinical work with an APO audit. The collegial brainstorming that starts every APO process also worked in this context [3]. The registration chart was supplemented with three Likert scales, used in a majority of the consultations, giving more detailed information about the consultation. It was possible to construct an APO chart with almost exclusively “soft” variables. It was also possible to measure the presence of one part of a holistic view if one shares these doctors’ idea of what a holistic view is.

A full APO audit includes construction of the registration chart, a pilot registration, a meeting for adjustments of variables and registration chart, and then the intended registration, followed by a meeting to compare results. In this study the process stopped after the pilot registration.

Significance of findings

An APO audit reflects everyday reality. The accuracy depends on how well the variables reflect important actions. It is a snapshot, which is used for discussions among colleagues concerning important values [3,4,9,10]. This pilot study shows alternative uses for the APO method, and introducing Likert scales led to development of the instrument. Although this is a crude way to study a GP's everyday work, the advantage is the speed of data collection and processing/analysis compared with recordings, interviews, etc. An APO audit captures subjective experiences which describe and influence reality [9,11–14].

The formation of variables is strongly participant-centred. We lack an evaluation of the participants’ views on the audit. In a full-scale audit concerning the use of different kinds of knowledge and a holistic view in GPs’ decision-making, these experiences should be utilized in the design of a final registration chart.

An APO audit, with its quick prospective and consecutive collection of data, is a method to reflect clinical work and decision-making. The significance of factual knowledge was particularly discussed during the construction process of the registration chart as the basic form of knowledge for a GP, as stressed by Pendleton and Wulff & Götzsche [15,16].

Learning and development do not follow a set pattern and are not the same for everyone. In the personal learning process it is therefore essential to proceed from how “I” act and think in given situations, in order to improve. An APO audit, if honestly completed, exposes defects and demolishes myths. Behaviour is influenced best if discussions with colleagues are combined with feedback of personal data [10,17,18].

Pilot audit

The pilot audit shows that it is possible to use the APO method to measure soft data in decision-making.

The Likert scales gave more detailed knowledge regarding the situations in which the doctors use their capacity for judgement, alone and in different combinations. Pendleton and Malterud stress the importance of the right questioning technique to arrive at difficult-to-obtain knowledge about patients and their symptoms [16,19–21]. The APO instrument can serve to reflect the extent to which the doctor uses such models to discover what patients keep concealed.

The Likert scale concerning familiarity has a direct link to continuity, which is one of the watchwords of primary care also defined by WONCA Europe [15,22–27]. Continuity and the relationship to the patient are also an important part of treatment [28]. The concept of cooperation is reflected through the variables concerning consultation with colleagues and others, as well as the Likert scales concerning symptomatology and reason for visit.

Conclusions

We have not tried to answer the question of whether the APO method could be used to measure a holistic view in full. We have focused only on the aspect of what kind of knowledge is used in the decision-making. We know that an APO audit influences doctors’ medical decision-making in well-demarcated and commonly occurring cases such as respiratory tract infections [5,9,11,13]. We have tested the instrument in a new context and with a partly new methodology. To gain some idea regarding the prevalence of other aspects of a holistic view in the everyday work of GPs, other methods could be useful.

Soft data, such as a holistic view, can be studied with the APO method, provided that the definition of knowledge is accepted as an aspect of the holistic view. The use of Likert scales made it possible to reach a deeper understanding of the consultation. The results indicate that the APO method could be an alternative to more time-consuming methods for studying the consultation.

Acknowledgements

The authors would like to thank the participating GPs for their important contribution. This study was supported by Blekinge County Council.

Competing interests

The authors declare that they have no competing interests.