Development and psychometric testing of a trans-professional evidence-based practice profile questionnaire

Figures & data

Table

Practice points

The EBP^2 questionnaire is a self report instrument which includes the major domains of Relevance, Terminology, Confidence, Practice and Sympathy. The EBP^2 questionnaire was developed and tested across a range of professions with acceptable psychometric properties (reliability and validity) and confidently discriminates for three factors (Relevance, Terminology and Confidence) between respondents with varying degrees of prior exposure to EBP education. The EBP^2 questionnaire shows promise as an instrument to monitor changes in EBP learning outcomes following instruction or cumulative learning throughout a degree.

Table 1.  Domains addressed in 10 most cited self-report allied health EBP questionnaires

		Attitude/ perceptions	Beliefs	Knowledge/ terminology	Practice/ behaviour	Confidence	Database/ research use	Factor analysis	Expert panel	Pilot	Reliability	Validity
Jette et al. (2003)	PT	x	x	x	x					x	x
Humphris et al. (2000)	OT	x			x
Metcalfe et al. (2001)	PT OT SP	x	x							x
Bennett et al. (2003)	OT	x			x	x	x			x
Dysart and Tomlin (2002)	OT	x			x	x
Curtin and Jaramazovic (2001)	OT	x			x					x
Taylor et al. (2001)	HLTH	x		x		x	x			x		x
Pollock et al. (2000)	PT OT N SP	x				x
O’Donnell (2004)	PH	x	x	x			x
McCluskey and Lovarini (2005)	OT	x		x		x	x			x

Note: PT, physiotherapy; OT, occupational therapy; SP, speech therapy; N, nursing; PH, pharmacy; and HLTH, health professions.

Figure 1. Scree plot illustrating components extracted from the data. The number of components retained is five as indicated by the change in shape of the plot after the fifth component.

Figure 2. Sample of questionnaire items for each of the five factors.

Table 2.  Test–retest reliability of the questionnaire

Factor	Range of weighted kappas for items in each factor	Range of ICCs for items in each factor	Factor ICCs	Mean difference	Bland Altman 95% limits of agreement (bias) for factors	Maximum possible score
Relevance	0.47–0.72	0.60–0.86	0.92	0.52	−7.68 to 8.72	70
Terminology	0.51–0.73	0.67–0.86	0.94	0.46	−10.26 to 11.18	85
Practice	0.48–0.66	0.64–0.80	0.83	−1.07	−8.70 to 6.56	45
Confidence	0.46–0.62	0.66–0.76	0.83	−1.07	−11.16 to 5.48	55
Sympathy	0.40–0.53	0.53–0.64	0.77	−0.23	−5.57 to 5.11	35

Note: Shown are the weighted kappa values and ICCs for questionnaire items, and ICCs, means difference, limits of agreement and maximum possible scores for each of the factors.

Table 3.  Factorial ANOVA, for the three groups of exposure to EBP training for relevance, terminology, confidence, practice and sympathy

	No course n = 48	≤20 h course n = 28	>20 h course n = 29
	Percentage of maximum (SD)	Percentage of maximum (SD)	Percentage of maximum (SD)
Relevance*	67.7 (21.5)^ab	79.4 (12.4)^a	81.7 (11.1)^b
Terminology*	38.6 (18.8)^ab	57.2 (23.9)^a	54.8 (17.8)^b
Confidence**	50.3 (17.9)^a	65.3 (20.9)^a	57.4 (16.6)
Practice	34.1 (17.7)	44.0 (21.9)	40.1 (16.7)
Sympathy	53.1 (15.7)	60.3 (12.4)	54.1 (13.7)

Notes: Exposure groups with the same superscript are significantly different to each other within the same domain. Mean factor scores as a percentage of maximum (SD).

*p ≤ 0.001; **p = 0.004.

Table 4.  Mean factor scores as a percentage of maximum (SD) for the academics, practitioners and students for the three groups of exposure to EBP training

	Academic (n = 10)			Practitioner (n = 69)			Student (n = 26)
	No training percentage of maximum	<20 h percentage of maximum	>20 h percentage of maximum	No training percentage of maximum	<20 h percentage of maximum	>20 h percentage of maximum	No training percentage of maximum	<20 h percentage of maximum	>2 h percentage of maximum
	n = 1	n = 7	n = 2	n = 31	n = 17	n = 21	n = 16	n = 4	n = 6
Relevance	100.0^a	90.1 (5.4)	80.4 (10.1)	66.0 (22.6)	77.2 (12.6)	81.7 (11.7)	69.0 (18.7)	70.1 (7.8)	81.8 (11.1)
Terminology	50.0^a	73.9 (16.8)	71.3 (7.3)	38.7 (20.9)	53.3 (23.5)	54.5 (18.3)	37.6 (14.7)	44.5 (26.0)	50.5 (17.2)
Practice	75.0^a	50.0 (22.2)	50.0 (15.7)	35.7 (15.8)	43.8 (22.5)	41.1 (15.3)	28.6 (18.2)	34.7 (20.6)	33.3 (21.9)
Confidence	61.4^a	74.4 (19.4)	70.5 (6.4)	50.4 (19.5)	64.0 (20.4)	56.9 (16.8)	49.4 (15.5)	54.5 (24.7)	54.9 (18.0)
Sympathy	60.7^a	62.8 (12.5)	48.2 (2.5)	54.1 (18.2)	61.8 (12.8)	55.8 (12.2)	50.4 (10.1)	50.0 (5.1)	50.0 (20.2)

Note: ^aSample of n = 1 no SD included.

Jette DU, Bacon K, Batty C, Carlson M, Ferland A, Hemingway RD, Hill JC, Ogilivie L, Volk D. Evidence-based practice: Beliefs, attitudes, knowledge, and behaviours of physical therapists. Phys Ther 2003; 83(9)786–805

 PubMed Web of Science ®Google Scholar

Humphris D, Littlejohn P, Voctor C, O’Halloran P, Peacock J. Implementing evidence-based practice: Factors that influence the use of research evidence by occupational therapists. Br J Occup Ther 2000; 63(110)516–522

 Google Scholar

Metcalfe C, Lewin R, Wisher S, Perry S, Bannigan K, Klaber MJ. Barriers to implementing the evidence base in four NHS therapies dieticians, occupational therapists, physiotherapists, speech and language therapists. Physiotherapy 2001; 87(8)433–441

 Google Scholar

Bennett S, Tooth L, McKenna K, Rodger S, Strong J, Zivani J, Mickan S, Gibson L. Perceptions of evidence-based practice: A survey of Australian Occupational Therapists. AOTJ 2003; 50: 13–22

 Google Scholar

Dysart AM, Tomlin GS. Factors related to evidence-based practice among US occupational therapy clinicians. Am J Occup Ther 2002; 56: 275–284

 PubMed Web of Science ®Google Scholar

Curtin M, Jaramazovic E. Occupational therapists’ views and perceptions of evidence based practice. Br J Occup Ther 2001; 64(5)215–222

 Google Scholar

Taylor R, Reeves B, Mears R, Keast J, Binns S, Ewings P, Khan KS. Development and validation of a questionnaire to evaluate the effectiveness of evidence-based practice teaching. Med Educ 2001; 35: 544–547

 PubMed Web of Science ®Google Scholar

Pollock AS, Legg L, Langhorne P, Sellars C. Barriers to achieving evidence-based stroke rehabilitation. Clin Rehabil 2000; 14: 611–617

 PubMed Web of Science ®Google Scholar

O’Donnell C. Attitudes and knowledge of primary care professionals toward evidence-based practice: A postal survey. J Eval Clin Pract 2004; 10(2)201–210

 Web of Science ®Google Scholar

McCluskey L, Lovarini M. Providing education on evidence-based practice improved knowledge but did not change behaviour: A before and after study. BMC Med Educ 2005; 5: 40

 PubMedGoogle Scholar