A simple heuristic for Internet-based evidence search in primary care: a randomized controlled trial

Figures & data

Figure 1 The three-step heuristic of decision making.

Figure 2 Workshop table of contents.

Abbreviations: ARR, absolute risk reduction; RRR, relative risk reduction; NNT, number needed to treat.

Figure 3 Flowchart of general practitioners’ participation, and dropout.

Abbreviation: WS, workshop.

Table 1 Preconditions of study participants

Skills and preconditions	External comparison: general practitioners				Internal comparison: complete dataa
Group characteristics	Gb	Hc			IG	CG	P*
	n=32,477	n=2,860	n=84	n=51	n=20	n=31
Personal preconditions
Age (years, SD),	52b	52 (7.5)	49 (7.4)	49 (9.8)	50 (7.3)	48 (11.2)	0.56
Sex (female, mean %)	38d	36	30	26	30	23	0.79
Professional preconditions
Years in practice, (median, SD)	–	14 (9.3)	13 (8.5)	14 (8.2)	15 (9.1)	13 (7.6)	0.29
Working in group practice (mean %)	32b	42	58	61	53	68	0.34
Internet preconditions
Internet access: “Yes”	–	–	95%	96%	95%	97%	0.44
Broadband account: “Yes”	–	–	61%	61%	65%	58%	0.58
DocCheck™ password:e “Yes”, %	–	–	44	42	55	32	0.08
Yes: Internet searchf, %	–	–	58	61	75	52	0.1
Skillsg (good or very good, %)
Internet	–	–	45	42	47	42	0.12
Computer	–	–	41	36	47	39	0.2
English language	–	–	31	38	37	39	0.9

Notes:

a Complete data: 1: participation and evaluation of both workshops; 2: documentation and evaluation of seven literature searches; 3: answering 39 questions twice.

b G = published statistics of the Federal Chamber of Physicians in Germany;³⁷

c H = details of the Regional Chamber of Physicians in Hessen (Landesärztekammer Hessen);³⁸

d representative sample of the Commonwealth Fund;³⁹

e the DocCheck™ password allows licensee to access drug formulary and labeling information;

F participants have conducted Internet search for medical topics in the previous year;

g skills as judged by participants themselves. Scale: 1 (low) to 5 (high).

* P-values for between-group comparison.

Abbreviations: IG, intervention group; CG, control group; SD, standard deviation.

Figure 4 Participants’ workshop evaluation.

Notes: (N=51, 1= very good, 6= deficient). Please note that GPs in the control group are included in this process evaluation.
Abbreviations: IG, intervention group; GPs, general practitioners; CG, control group.

Figure 5 Commitment to change at the end of the workshop (n=48).

Note: Please note that GPs in the control group are included in this process evaluation.
Abbreviation: GPs, general practitioners.

Table 2 Literature search experience

Group allocation
Intensitiy and satisfiction of search	IG (n=20)	CG++ (n=31)	Difference#	P*
Number of sources used per topic
Median (IQR)	1.9 (0.6)	1.4 (0.5)	36%	0.01
Duration of search per topic
Median min (IQR)	20.9 (9.6)	17.8 (11.5)	17%	0.34
“Yes, I am satisfied with […] […] quality of information”	65%	54%	11%	0.08
“[…] time spent”	80%	65%	15%	0.35
“[…] amount of information”	53%	47%	6%	0.22
Surprised by result	26%	35%	9%	0.28
Trustworthiness of sources
Median (score: 1= low to 5= high) (IQR)	3.7 (0.6)	3.6 (0.7)	0.1	0.47
Results relevant for daily practice
Median (score: 1= low to 5= high) (IQR)	3.1 (0.9)	3.1 (0.8)	0.0	0.88

Notes:

* P-value for between-group comparison: Mann-Whitney U test.

++ Please note that physicians in the control group provided these evaluations before they were offered workshop participation.

# Difference in % between IG and CG.

Abbreviations: IG, intervention group; CG, control group; IQR, interquartile range.

Table 3 Sources of information used for literature review

Intervention group (IG)/control group (CG)	IG (n=20)	CG++ (n=31)	Difference# (IG-CG)	OR
Sources of information (n=570)
Number of sources per topic	1.9	1.4	0.5	1.4
1. Internet sources (n=382, =67%)	80.7%	59.4%	21.3%	2.9
(of which n= 119 search engines: Google, Yahoo, etc)	10.2%	26.2%	−16.0%	0.3
(of which n=263 targeted site selection)	70.5%	33.2%	37.3%	4.8
2. Print media (magazines, textbooks)	15.7%	30.5%	−14.8%	0.3
3. Personal interview	3.6%	10.1%	−6.5%	0.4
Internet sources (n=263, without search engines)
1. Pharma-critical drug bulletins	17.1%	8.0%	9.1%	2.4
2. Guidelines (national and international)	38.7%	23.0%	15.7%	2.1
3. Commercial homepages (pharmaceutical, publishing houses, clinics)	1.8%	31.0%	−29.2%	0.2
Adherence to workshop recommendations	67%	15%	52%	9.6

Notes:

++ Please note that physicians in the control group provided these evaluations before they were offered workshop participation.

# Difference in % between IG and CG.

Abbreviation: OR, odds ratio.

Table 4 Effectiveness measures of search and appraisal competence

Correctness and confidence	Pre test	Post test	Difference (95% CI)	P^*
Correctnessa (SAC)b
Pre–post trial
IG (n=20)	39.6%	54.5%	14.9% (7.85–21.90)	0.00
CG++ (n=31)	41.9%	52.6%	10.7% (5.98–15.36)	0.00
Randomized controlled trial
IG/CG (n=51)	−2.32%	1.88%	4.2% (−4.26–1.01)	0.29
Confidencec
Pre–post trial
IG (n=20)	50.1%	81.8%	31.7% (20.9–42.5)	0.00
CG++ (n=31)	53.6%	79.4%	25.8% (20.1–31.5)	0.00
Randomized controlled trial
IG/CG (n=51)	−3.5%	2.4%	5.9% (−17.7–5.9)	0.28

Notes:

a Correctness: additional correct answers: before vs after search in pre–post test.

b SAC corresponds with score of correctness.

c Confidence: perceived safety when responding (0 to 100%: 0= completely unsafe/100%= absolutely certain that the answer is correct. P^* indicates Student’s t-test used, for paired samples, twotailed significance.

++ Please note that physicians in the control group provided these evaluations before they were offered workshop participation.

Abbreviations: CI, confidence interval; SAC, search and appraisal competence; IG, intervention group; CG, control group.

German Medical AssociationPublished statistics of the Federal Chamber of Physicians in Germany Available from: www.bundesaerztekammer.de/page.asp?his=0.3.1667.6371Accessed May 6, 2016

 Google Scholar

Landesarztekämmer HessenDetails of the Regional Chamber of Physicians in Hessen Available from: www.laekh.de/aerzte/mitgliedschaft/mitgliedschaft-mitgliederstatistikenAccessed May 6, 2016

 Google Scholar

Deutscher Ärzteverlag GmbH37.Representative sample: Koch K Primary Care in Germany – an International Comparison Available from: www.aerzteblatt.de/pdf/DI/104/38/a2584e.pdfAccessed May 6, 2016

 Google Scholar