Evidence and practice in spine registries

Abstract

Background and purpose — We performed a systematic review and a survey in order to (1) evaluate the evidence for the impact of spine registries on the quality of spine care, and with that, on patient-related outcomes, and (2) evaluate the methodology used to organize, analyze, and report the “quality of spine care” from spine registries.

Methods — To study the impact, the literature on all spinal disorders was searched. To study methodology, the search was restricted to degenerative spinal disorders. The risk of bias in the studies included was assessed with the Newcastle-Ottawa scale. Additionally, a survey among registry representatives was performed to acquire information about the methodology and practice of existing registries.

Results — 4,273 unique references up to May 2014 were identified, and 1,210 were eligible for screening and assessment. No studies on impact were identified, but 34 studies were identified to study the methodology. Half of these studies (17 of the 34) were judged to have a high risk of bias. The survey identified 25 spine registries, representing 14 countries. The organization of these registries, methods used, analytical approaches, and dissemination of results are presented.

Interpretation — We found a lack of evidence that registries have had an impact on the quality of spine care, regardless of whether intervention was non-surgical and/or surgical. To improve the quality of evidence published with registry data, we present several recommendations. Application of these recommendations could lead to registries showing trends, monitoring the quality of spine care given, and ultimately improving the value of the care given to patients with degenerative spinal disorders.

Lumbar spine disorders are a heterogeneous group of conditions with a lack of diagnostic clarity. Thus, in both surgical and non-surgical spinal interventions there are large variations in practice. The increasing frequency of spine-related interventions with increasing costs has led to a shift towards the delivery of value-based spine care (Hoy et al. 2014). Here, value is expressed as patient-centered outcomes (safety and effectiveness; quality) divided by the related costs of care or per unit cost (Porter 2010). While randomized clinical trials (RCTs) are considered to be the gold standard for assessing the efficacy of interventions, the difficulties with RCTs—specifically for assessment of surgical procedures in spinal disorders—are acknowledged (Jacobs et al. 2012a). Some barriers are surgeon preferences, patient selection, patients’ reluctance regarding randomization, difficulties in blinding, high cost, the need for long-term follow-up, the often high proportion of loss to follow-up, and the problem with crossover. Well-designed observational cohort studies, reflecting daily clinical practice, have been reported to produce as trustworthy and externally valid results as RCTs (Benson and Hartz 2000, Concato et al. 2000, Weinstein et al. 2009, Concato et al. 2010, Colditz 2010, Jacobs et al. 2012b, Phillips et al. 2013).

An outcome registry is an organized system that uses observational study methods (Gliklich and Dreyer 2010) based on STROBE recommendations (von Elm et al. 2007). Registries could therefore be used to describe care patterns, including appropriateness of care and disparities in the delivery of care. Registry data could also be used to understand variations in treatment and outcomes, and to identify and select subgroups in the heterogeneous chronic low back pain population with a probability of successful or poor outcome. The ultimate goal of health service registries is to increase the value of care delivered (i.e. outcome per unit cost) (Kuenen et al. 2011). Moreover, it has been suggested that measuring with continuous feedback (audit cycles) of outcomes captured in registries raises awareness and improves the quality of care (Larsson et al. 2010, van Leersum et al. 2011). However, as yet, little is known about the effect of spine registries on the quality of spine care and the methods for registering and feedback.

This study had 2 aims: (1) to evaluate the available evidence for the effect and possible impact of introducing and using spine registries on the quality of spine care after any intervention and on patient-related outcomes; (2) to evaluate the methodology used to organize, analyze, and report the “quality of spine care” from spine registries.

Methods

We performed a systematic review according to the PRISMA statement for reporting in systematic reviews and meta-analyses (PRISMA 2014). In addition, for the second aim (regarding methodology) a survey among spine registry representatives was performed to acquire information about the current status of existing spine registries. The complete protocol for this study was presented at the preconference meeting of the International Society for the Study of the Lumbar Spine 2014 (ISSLS) (Jacobs et al. 2014).

Selection of studies and appraisal of their quality was performed independently by MH and WJ. Discrepancies were discussed during consensus meetings, with mediation by a third author (PW) where disagreements persisted.

Search

A comprehensive search was conducted because terminology in the field of chronic low back pain is not yet standardized, and because we aimed to include both randomized and non-randomized studies. The search was performed by one of the authors (WJ) on May 12, 2014, using the most common databases: the CBRG trials register (up to the search date), MEDLINE (from 1966 to the search date), EMBASE (from 1980 to the search date), and ISI Web of Science (up to the search date). The search string for MEDLINE is given in Appendix 1 (see Supplementary data) and was adapted for the other databases. No language or date restrictions were made. References and citations of selected articles were tracked and included in the search.

Impact of registries on the quality of spine care

Articles were included if they met the following criteria.

Types of studies

Studies on spine registries based on results from prospectively acquired data were included. To comply with the definition of patient or outcomes registry, we used the following registry characteristics: inclusion principle, mergeable data, standardized dataset for all consecutively included patients, rules for data collection (i.e. systematically and prospectively collected, including pre-intervention data), knowledge about patient-related outcomes, and observations collected over time (i.e. follow-up assessments) (Drolet and Johnson 2008). Studies were included if published between 2000 and May 2014 and written in English.

Types of spine disorders

We included patients of all ages with spine disorders who underwent any elective or acute, non-surgical or surgical spinal intervention. The following specific disorders were defined: degenerative disc disease (DDD; “non-specific” and/or chronic low back pain, segmental pain), spinal stenosis, disc herniation, spondylolisthesis/-lysis (isthmic/degenerative), failed back surgery syndrome, spinal deformities (degenerative deformity: de novo, osteoporotic; idiopathic, neuromuscular, congenital), spinal oncology, spine trauma, and infections of the spine.

Types of interventions

Interventions were included that provide a system to register the quality of spine care, i.e. outcomes, including a system for feedback: quality improvement strategies (Shoania et al. 2006, Tricco et al. 2012, Munce et al. 2013). These strategies include those targeted: health systems (e.g. team changes), healthcare professionals (e.g. reminders), and patients (e.g. reminders). Specific improvement strategies included: (clinical) audit and feedback, (electronic) patient registries, case management, clinician education, (promotion of) self-management, patient reminder systems, and continuous quality improvement.

Types of outcome measures

Quality of care is a multidimensional concept and is defined in many ways, e.g. “doing the right thing, at the right time, in the right way, for the right person, and having the best possible results” (AHRQ 2014). Following this definition, the outcomes of spine interventions are a proxy for quality of care. As measures for quality of spine care, we included patient-related outcome indicators: patient-reported outcome measures (PROMs) and clinical outcome measures. PROMs (McCormick et al. 2013) are functional status (e.g. the Roland and Morris disability questionnaire (RMDQ), the Oswestry disability index (ODI), Scoliosis Research Society-22 (SRS22)), pain intensity—back and leg (e.g. visual analog scale (VAS), numeric pain rating scale (NPRS)), and health-related quality of life (e.g. Short Form-36 (SF36), EuroQol 5 dimensions (EQ5D)). Clinical outcomes were regarded as reintervention (i.e. reoperation), complications, and failed back surgery syndrome (FBSS).

The search did not reveal any studies related to the first aim of this study (impact) concerning the effect and possible impact of introducing and using spine registries on the quality of spine care and on patient-related outcomes (Figure 1).

Flow chart of studies through the different phases of the systematic review.

Methodology used in existing spine registries

Selection criteria

The same criteria were used as for the first aim, but they were restricted to include studies with patients with degenerative lumbar spine disorders: degenerative disc disease (DDD, “non-specific” and/or chronic low back pain, segmental pain), spinal stenosis, disc herniation, spondylolisthesis/-lysis (isthmic/degenerative), and spinal deformities (degenerative deformity: de novo, osteoporotic).

Risk of bias assessment

The studies included were assessed for methodological quality, to get an impression of the quality of published scientific studies based on registries. Quality was assessed with the Newcastle-Ottawa scale (NOS; Appendix 2, see Supplementary data) for cohort studies (Wells et al. 2008). Studies were considered to be of high quality if the total score was 6 or more (75% of the maximum score). The clinical relevance of study results was assessed with 3 questions: (1) “Are the patients described in detail so that you can decide whether they are comparable to those that you see in your practice?”; (2) “Are the interventions and treatment settings described well enough so that you can provide the same for your patients?”; 3) Were all clinically relevant outcomes measured and reported?”.

Data extraction and management

Using forms already developed, the following data were extracted: authors (affiliation, sponsoring), name and type of registry (i.e. based on exposure: health service, disease/condition, and medical devices (Gliklich and Dreyer 2010)), setting (nationwide, multicenter), diagnosis, methods (purpose, study design, outcomes, covariates, statistical analysis, patient numbers recruited and included), follow-up response, non-responder analysis, conclusion. MH extracted the data and WJ checked the data; inconsistencies were discussed and PW was consulted if necessary.

Survey

A web-based survey, built using the Harvard Business Online Qualitrics Survey Software and provided by International Consortium for Health Outcomes Measurement (ICHOM), was performed among spine registry representatives. The survey consisted of 21 questions regarding: (1) organizational structure and financing, (2) methodology used and outcome assessment, (3) procedures concerning response rates and missing data, and (4) approaches for analysis and reporting.

The sample included participants of the ICHOM Low Back Pain Working Group, representatives of spine registries identified through spine registry websites, and corresponding authors of publications on spine registries as identified in this systematic review. All recipients were contacted by e-mail and asked to participate in an online survey. Subjects who did not respond were sent a reminder after 2 weeks.

Analyses

Results from the studies included in the review were not pooled; instead, we compared and reported on the methods used in these studies. The data from the survey are described and support the results found in the review. The PROMs used were checked with the ICHOM-LBP PROMs criteria (ICHOM 2014): (1) functional status (ODI [0–100] version 2.1a); (2) pain intensity (NPRS [0-10] back and leg; average pain during the last 7 days); (3) health-related quality of life (EQ5D-3L and EQ-VAS); (4) timeline assessments included were baseline, 3 and 6 months, and 1, 2, and 5 years after the “index event” (3-month and 5-year assessments were optional). The index event was defined as the first intervention episode reported.

Results

Impact of registries on quality of spine care

Studies included

4,273 unique references were identified, 1,210 of which were eligible for screening and assessment (Figure). No studies on the effect of spine registries on quality of care were identified.

Methodology used in existing spine registries

Studies included

34 studies were identified for study of the methodology used to organize, analyze, and report the quality of spine care in degenerative lumbar spine disorders (Appendix 3, see Supplementary data; Table 1). The 34 studies were based on 11 separate registries, representing 7 countries. Indications included were disc herniation (3), spinal stenosis (9), chronic low back pain (5), adult deformity (5), and spinal disorders (7). In 3 studies, a mixture of indications (non-specific subacute and chronic neck, back, and low back pain) was included, but these allowed us to extract data concerning the methodology used for separate indications.

Table 1. Risk of bias assessment according to Newcastle-Ottawa scale (NOS)

Study	Spine registry	Selection				Comparability		Outcome			Total *score ^b	Clinical relevance
		1	2	3	4	5a	5b	6	7	8		1	2	3
Nerland et al. 2014	NORspine	a *	a *	a *	a *	yes *	yes *	c	a *	n.a.	7 (7) ^a	n.a.	n.a.	n.a.
Solberg et al. 2013	NORspine	a *	n.a.	d	b	no	no	c	a *	c	2 (7)	no	no	yes
Corcoll et al. 2006	NRT en el SNS	a *	n.a.	b *	a *	no	no	c	b	c	3 (7)	yes	yes	yes
Kovacs et al. 2012	NRT en el SNS	b *	n.a.	b *	a *	yes *	yes *	c	b	b *	6 (7) ^a	no	yes	yes
Kovacs et al. 2007	NRT en el SNS	b *	n.a.	d	a *	yes *	yes *	c	b	c	4 (7)	yes	yes	yes
Royuela et al. 2014	NRT en el SNS	a *	n.a.	d	b	yes *	yes *	c	b	c	3 (7)	yes	yes	yes
Fritzell et al. 2014	SweSpine	a *	n.a.	c	a *	no	yes *	c	a *	c	4 (7)	no	no	yes
Forsth et al. 2013	SweSpine	a *	a *	a *	a *	yes *	yes *	c	a *	c	7 (8) ^a	no	no	yes
Jansson et al. 2005	SweSpine	a *	n.a.	d	a *	yes *	yes *	c	a *	c	5 (7) ^a	no	no	yes
Jansson et al. 2009	SweSpine	a *	n.a.	d	a *	yes *	yes *	c	a *	c	5 (7) ^a	no	no	yes
Knutsson et al. 2013	SweSpine	a *	n.a.	d	a *	yes *	yes *	c	a *	c	5 (7) ^a	no	no	yes
Robinson et al. 2013	SweSpine	a *	a *	d	a *	yes *	yes *	c	a *	c	6 (8) ^a	no	no	yes
Sanden et al. 2011	SweSpine	a *	a *	d	a *	yes *	yes *	c	a *	c	6 8) ^a	no	no	yes
Sigmundsson et al. 2012	SweSpine	a *	n.a.	a *	a *	yes *	yes *	c	a *	d	6 (7) ^a	yes	yes	yes
Sigmundsson et al. 2013	SweSpine	a *	n.a.	d	a *	no	no	c	b	d	2 (7)	no	no	yes
Sigmundsson et al. 2014	SweSpine	a *	a *	d	a *	yes *	yes *	c	a *	c	6 (8) ^a	no	no	yes
Stromqvist et al. 2012	SweSpine	a *	n.a.	d	a *	no	no	c	a *	c	3 (7)	no	no	yes
Berg et al. 2010	SSE Spine Tango	b *	a *	a *	a *	no	no	c	a *	b *	6 (8) ^a	yes	yes	yes
Grob and Mannion 2009	SSE Spine Tango	c	n.a.	d	b	no	no	c	a *	b *	2 (7)	no	no	yes
Porchet et al. 2009	SSE Spine Tango	c	a *	d	a *	no	no	c	a *	d	3 (8)	no	no	yes
Aghayev et al. 2012	SWISSspine	a *	b	d	a *	yes *	yes *	c	a *	d	5 (8)	no	no	yes
Aghayev et al. 2010	SWISSspine	b *	a *	d	a *	yes *	yes *	c	a *	c	6 (8) ^a	no	no	yes
Schluessmann et al. 2009	SWISSspine	a *	n.a.	d	a *	yes *	yes *	c	a *	c	5 (7) ^a	no	yes	yes
Zweig et al. 2011	SWISSspine	a *	n.a.	d	a *	yes *	yes *	c	a *	d	5 (7) ^a	yes	yes	yes
McGirt et al. 2013	N^2QOD	a *	a *	d	a *	n.a.	n.a.	c	a *	n.a.	4 (5) ^a	n.a.	n.a.	n.a.
Deer et al. 2004	Nat Outc Reg ^c	b *	n.a.	c	a *	no	no	c	a *	c	3 (7)	no	yes	yes
Taylor et al. 2000	Com outc m study ^d	b *	n.a.	d	b	yes *	yes *	c	a *	c	4 (7)	no	no	no
Bridwell et al. 2007	A D O Database ^e	b *	n.a.	d	a *	no	yes *	c	a *	d	4 (7)	yes	no	yes
Glassman et al. 2009	A D O Database ^e	b *	n.a.	d	a *	no	yes *	c	a *	d	4 (7)	no	no	yes
Glassman et al. 2007	A D O Database ^e	b *	n.a.	d	a *	yes *	yes *	c	a *	d	5 (7) ^a	yes	no	yes
Kasliwal et al. 2012	A D O Database ^e	b *	a *	d	a *	yes *	yes *	c	a *	d	6 (8) ^a	yes	no	yes
Schwab et al. 2008	A D O Database ^e	a *	n.a.	d	b	yes *	yes *	c	a *	c	4 (8)	yes	no	yes
Adogwa et al. 2014	Multicenter reg ^f	a *	a *	d	a *	no	yes *	c	a *	d	5 (8)	no	no	yes
Seng et al. 2013	Singapore GH Reg ^g	b *	a *	c	a *	no	no	c	a *	a *	5 (8)	no	yes	yes
Percentage with *, ^aor yes		94	92	17	86	59	71	0	85	13	50	31	31	97

n.a.: not applicable.

* score NOS.

^a high quality.

^b n items, considering n.a.

^c National Outcomes Registry

^d Community outcomes management study

^e Adult Deformities Outcomes Database

^f Multicenter registry for lumbar spine surgery

^g Singapore General Hospital Spine Outcomes Registry

Explanation of NOS, including description of items a–d and method of scoring, is given in Appendix 2:

Selection: 1 representativeness exposed cohort; 2 selection non-exposed cohort; 3 ascertainment exposure; 4 outcome not present at start study.

Comparability: 5a and 5b comparability cohorts on the basis of design/analysis.

Outcome: 6 outcome assessment; 7 follow-up long enough; 8 adequacy follow-up.

Explanation of clinical relevance (yes/no):

1: Are the patients described in detail so that you can decide whether they are comparable to those that you see in your practice?

2: Are the interventions and treatment settings described well enough so that you can provide the same for your patients?

3: Were all clinically relevant outcomes measured and reported?

Risk of bias

Half of the studies were classified as having a high methodological quality (17 of 34; Table 1). Although selection of the non-exposed cohort (item 2) scored high quality (i.e. low risk of bias), 21 studies were rated “n.a.” as these studies did not include a control group. In 20 of the 34 studies, a low risk of bias in comparability (item 5a) was seen, meaning studies controlled for the most relevant case-mix variables (i.e. diagnosis and baseline outcome score). In all studies, the assessment of outcome (item 6) was rated “self-report” (c), and with that scoring low quality. The follow-up was long enough for outcomes to occur (item 7; 29 of 34). However, low quality was seen in the adequacy of follow-up of cohorts (item 8; 18 of 32). Although in almost all the studies the clinically relevant outcomes were measured and reported (31 of 32; clinical relevance item 3), in less than one-third of the studies (10/32) the description of patients and intervention (clinical relevance items 1 and 2) was sufficiently detailed.

Survey

We identified 25 spine registries, representing 14 countries, within the ICHOM Low Back Pain Working Group (ICHOM-LBP WG; 10) through the literature (10; 4 overlap with ICHOM-LBP WG) and through internet searches (9). We were unable to make contact with representatives of 7 of the multicenter registries; the remaining 18 were invited to participate in the survey. 16 of them responded, representing 12 countries and 2 including “diverse” countries (Spine Tango and the European Spine Study Group database (ESSG)). The non-responders were representatives of Russian and Indian registries.

An overview of existing spine registries is presented in Table 2: survey responders (16) and searches though internet and literature (6). 3 multicenter registries in the USA were found through internet searches, but we were unable to obtain relevant information (NASS Spine Registry, SMISS Prospective Data Registry, and Scolisoft Scoliosis Database (see references for websites)); these registries were not included in Table 2.

Table 2. Registry characteristics

Registry name ^1	Location	Setting ^2	Since	Location spine ^3	Outcomes lumbar spine			PROMs assessments ^7	ICHOM-LBP PROMS criteria	Benchmark	Scientific publication ^8	NOS risk of bias score ^9
					Functional Status ^4	Pain ^5	Quality of life ^6
Survey respondents
SweSpine	Sweden	N	1998	L; C; D; T; I; M	ODI	VAS B&L	SF36; EQ5D	B; P; 12; 24; O, 5y	Yes	average & individual centers	Yes; 12	5.3 (2–7) ^10
NORspine	Norway	N	2006	L; D	ODI	NPRS B&L	EQ5D	B; P; 3; 12	No	average	Yes; 2	4.5 (2–7)
DaneSpine	Denmark	N	2009	L; C; D; T; I; M	ODI	VAS B&L	EQ5D; SF36 occasion.	B; P; 3; 6; 12; 24; O, 5y; 10y	No	average	No
Dutch Spine Surgery Registry	The Nether-lands	N	2014	L & D(mainly)C; T; I; M	ODI	NPRS B&L	SF36; EQ5D	B; P; 2/6; 12; 24*	Yes	average & individual centers(planned)	No
British Spine Registry	UK	N	2012	L; C; D; T; I; M	ODI	VAS B&L	EQ5D	B; 3; 12; 24; O, 5y	No	average & individual centers(planned)	No
NRT en el SNS	Spain	N	2002	L; C	RMDQ	NPRS B&L	not meas.	B; P; 3; O, each 3 mo #	No	only own data	Yes; 4	4.0 (2–6)
SWISSspine	Switzerland	N	2005	L; C; D; T; I; M	NASS; COMI	NPRS B&L	EQ5D	B; P; 3; 6; 12; 24; O, 5y;10y	No	average	Yes; 4	5.3 (5–6) ^10
SSE Spine Tango	Diverse	M	2002	L; C; D; T; I; M	ODI; COMI	COMI: NPRS B&L	EQ5D occasion.	B; P; 6w; 3; 12; 24 $	Yes	average	Yes; 3	3.3 (2–5)
Canadian ^a	Canada	N/M	2012	L; C; T	ODI	VAS B&L	SF12; EQ5D	B; P; 3; 12; 24	No	average	No
Singapore ^b	Singapore	N/I	2001	L; C; D; T; I; M	ODI; NASS	NPRS B&L	SF36	B; 1; 3; 6; 24	No	only own data	Yes; 1	5
Newro Foundation	Australia	I	2010	L; C	ODI; RMDQ	NRS B&L	SF12, future EQ5D	B; 6w; 3; 6; 12; 24	No	only own data	No
Texas Back Institute	USA	I	New	L; C; D; T; I; M	ODI	VAS B&L	SF12, future EQ5D	B; 3 %	No	only own data	No
Kaiser Permanente	USA	M	2009	O, instru-meneted procedures	n.a.	n.a.	n.a.	P	No	only own data	Yes
N²QOD ^d	USA	M	2012	L; C; D	ODI	NPRS B&L	EQ5D	B; 3; 12	No	average	Yes; 1	4 ^10
Schön-Clinics Spine Registry	Germany	M/I	2010	L; C; D; T; I; M	ODI	VAS B&L	EQ5D	B; P; 3; 12; 24	No	average & individual centers	Yes
European Spine Study Group	Diverse	M	2010	D	ODI; SRS22r; COMI	NRS B&L	SF36	B; P; 6w; 6; 12; 24	n.a.	n.a.	Yes
Other sources
Russian Spine Registry 11;12	Russia	M	2012	L	ODI	VAS	SF36	B; P; not reported	No	Not found	Yes
Indian Spine (surgery) Registry ^12	India	M; future N	Plan.	L	Not found	Not found	Not found	Not found	No	Not found	Not found
National Spine Network ^e	USA	M	1995	L; C	ODI	Not found	SF36 or SF12 or Sf8	B; 12 optional: 3; 6; 24	Not found	Not found	Yes; 1	3
Multicenter registry ^f	USA & Canada	M	2003	L	ODI	VAS B&L	Not found	B; P; 12; 24	No	Not found	Yes; 1	5
ADO Database ^g	USA ^h	M	2002	D	SRS-22; ODI	SRS22(pain)	SF12	B; 6; 12; 24; 5y; 10y; 15y; 20y; 25y	n.a.	Not found	Yes; 5	4.6 (4–6)
ATSD Database ^i	USA ^h	M	2010	D	SRS-22 ODI	SRS22(pain)	SF36	B; 12; 24	n.a.	Not found	Yes

n.a.: not applicable.

¹ Registry names:

^a Canadian Spine Outcomes and Research Network

^b Singapore General Hospital Spine Outcomes Registry

^c Kaiser Permanente Spine Implant Registry

^d National Neurosurgery Quality and Outcomes Database

^e National Spine Network Spine Outcomes Registry (SpineChart) ¹²

^f Multicenter registry for lumbar spine surgery ¹³

^g Adult Deformity Outcomes Database ¹⁴

^h USA Spine Deformity Study Group

ⁱ Adult Thoracolumbar Spinal Deformity Database ¹⁵

² Settings: N National, M Multicenter, I Institutional

³ Locations: L lumbar spine; C cervical spine; D spinal deformity; T spine trauma; I spinal infections; M spinal metastases; O other.

⁴ Functional status: ODI Oswestry disability index; RMDQ Roland and Morris disability questionnaire; NASS North American Spine Society lumbar spine outcome scale; COMI core outcome measures Index; SRS22 Scoliosis Research Society 22 questions.

⁵ Pain: B&L back and leg; VAS visual analog scale; NPRS numeric pain rating scale.

⁶ Quality of life: SF8, SF12, SF36 Short Form 8 or 12 or 36 questions; EQ5D EuroQol 5 dimensions (including EuroQol VAS).

⁷ PROMS at: B baseline; P perioperative; 6w 6 weeks; 1 1 month; 3 3 months; 6 6 months; 12 12 months; 24 24 months; O other, …; * 2 months in hernia/stenosis; # until discharge; $ at least 1 follow-up; % variabel: when patient returns to clinic.

⁸ n; according to Appendix 2; Table 1

⁹ NOS Risk of bias score Newcastle-Ottawa scale – total score; median (range) according to Table 1

¹⁰ high quality.

¹¹ Shevelev et al. 2013;

¹² See references for websites.

¹³ Adogwa et al. 2013;

¹⁴ e.g. Kasliwal et al. 2012 (see Table 1);

¹⁵ Scheer et al. 2013.

Organization and methods used in spine registries

Organization (Table 2)

9 of 22 registries are organized on a nationwide basis. Most spine registries started within the last decade. All registries are health services registries—except for Kaiser Permanente, which is a device or implant registry, and SWISSspine, a device or implant registry for health technology assessment purposes.

Methods (Table 2)

All registries incorporated the main patient-reported outcome domains. ODI, NPRS back and leg, and EQ5D are mainly used as PROMs. In the majority of the registries, clinical outcomes (e.g. complications and reoperations) are also registered. All registries have baseline and 12- and 24-month follow-up assessments, except for NORspine and N²QOD (with only a 12-month follow-up). Although 15 registries report on lumbar spine disorders, only 3 fulfill all the ICHOM-LBP criteria for PROMs. The Adult Deformity Outcomes Database registry has the longest follow-up (25 years). To improve the response rate, all registries use postal, e-mail, or telephone reminders.

Analysis and reporting

In the 34 scientific publications, various analytical approaches were used (Appendix 3, see Supplementary data, Table 4), varying from descriptive statistics, all studies, to multivariate techniques such as mixed linear modeling (Nerland et al. 2014, Robinson et al. 2013) and propensity modeling (Kasliwal et al. 2012, Adogwa et al. 2014). To evaluate the study purpose, all studies used 1 or more PROMs as an outcome measure. In 8 studies, secondary clinical outcomes were defined: complications (Schwab et al. 2008, Kasliwal et al. 2012, Adogwa et al. 2014, Nerland et al. 2014), reoperation (Bridwell et al. 2007), BMI (Knutsson et al. 2013), adverse events (Deer et al. 2004), and Bridwell classification for fusion rates (Seng et al. 2013). In 5 studies, no adjustment for covariates was performed to explain variation in outcomes (Corcoll et al. 2006, Grob and Mannion 2009, Murray et al. 2012, Seng et al. 2013, Sigmundsson et al. 2013). In the remaining 29 studies, patient characteristics were used as covariates, varying from 17 predefined covariates (Royuela et al. 2014) to adjustment for baseline PROMs only (Solberg et al. 2013). Adjustment for baseline PROMs was not performed in 8 studies (Deer et al. 2004, Bridwell et al. 2007,Glassman et al. 2009, Zweig et al. 2011, Kovacs et al. 2012, Sigmundsson et al. 2012, McGirt et al. 2013, Adogwa et al. 2014). In 8 studies, a dropout analysis was performed to compare baseline characteristics (missing data on assessment) with the remaining cases. To handle missing data, multiple imputation techniques were used in 2 studies (Kovacs et al. 2012, Royuela et al. 2014). In the remaining studies, complete case analysis was performed.

All 16 registry representatives reported in the survey, to describe the population and outcomes using descriptive statistics and to provide feedback reports on a regular basis to all participating institutions and spine societies. Benchmarking is performed against the average value of participating institutions in 10 registries (Table 2). The 12-month follow-up responses on PROMs vary from 20% (British Spine Registry) to 88% (the Neuroreflexotherapy Registry of the Spanish National Health Service (NRT en el SNS))

Discussion

We found a lack of evidence to support or refute the effect that spine registries may have on the quality of spine care and on patient-related outcomes. Nonetheless, the publications that have resulted from the spine registries have yielded relevant evidence on interventions or predictive factors for spinal disorders. We have therefore described the methodology used to organize, analyze, and report the “quality of spine care” from spine registries. To improve the quality of results published from registry data and to study the effects of spine registries in future, we have formulated and included several recommendations, which are summarized in Table 3. First of all, the registries should be methodologically well constructed, and we need to learn from existing registries so that a more standardized approach to registering and analysis can be achieved to allow international collaboration, to allow national and international benchmarking, and to make sure that in future spine care is value-based (Table 3, recommendation (rec.) 1).

Table 3. Recommendations to improve the quality of study results published from registry data

Organization and method

1. Use a standardized approach to registering in design, methodology, and analysis to allow international collaboration, to achieve benchmarking, and to make sure that in future spine care is value-based.

2. Study and incorporate strategies to improve quality of care, e.g. continuous feedback and audit cycles of results collected in spine registries, on spine care delivered.

3. To increase the quality of registry studies, the population needs to be well-defined in terms of diagnosis and indications for surgery. Both at the developmental stage of a registry and when reporting on registry data, follow the STROBE guidelines.

4. Include a minimum follow-up period of 1 year for surgically treated patients.

5. To meet the definition of a patient registry, all characteristics of a registry should be present (Drolet and Johnson 2008). This means an inclusion principle, mergeable data, standardized dataset for all consecutively included patients, rules for data collection (i.e. systematically and prospectively collected, including pre-intervention data), knowledge about patient-related outcomes, and observations collected over time (i.e. follow-up assessments).

Patient-related outcomes

6. Patient-reported outcome measures for degenerative lumbar spine disorders are PROMs with good measurement properties, and as recommended by ICHOM. Although often defined as patient-related clinical outcomes (i.e. reoperation, complications, and failed back surgery syndrome), these indicators are in fact process measures for a complicated course.

Analysis and reporting

7. To explain differences in outcomes with advanced multivariate analytical techniques, include a reliability adjustment and an adjustment for covariates. For degenerative lumbar spine disorders, the recommended factors in ICHOM could be used as covariates.

8. To reduce bias in results a 60-80% 12-months follow-up response is recommended.

9. To increase PROM response at follow-up, reminders could be sent by text messaging or e-mail.

10. To understand potential sources of bias, a non-responder analysis on baseline characteristics should be provided, including a quantitative sensitivity analysis in order to evaluate the extent to which the results are affected by bias.

11. Multiple imputation techniques are recommended for sensitivity analysis when missing data are randomly divided.

Practical issues^a

12. Linkage between electronic medical records and registry data to avoid double data entry and to enhance routine in daily practice.

13. Participating departments should have direct access to their own data and should have real-time comparisons with other departments and, if available, with the national mean.

14. After approval, analyzed results corrected for case mix should be presented for public access on open web pages in order to increase credibility and to allow adequate and relevant comparisons.

^a not discussed in this study.

Quality improvement in spine care

Although we did not find any scientific evidence for an effect of introducing and using spine registries on the quality of spine care, in 16 registries feedback reports are compiled and disseminated on a regular basis to the participating institutions and the spine societies in order to improve the quality of spine care delivered. In general, improvement strategies include (clinical) audit and feedback, (electronic) patient registries, case management, clinician education, (promotion of) self-management, patient reminder systems, and continuous quality improvement (Shojania et al. 2006, Tricco et al. 2012, Munce et al., 2013) (Table 3, rec. 2).

That registries can have an important effect on the quality of healthcare has, however, been reported in other fields. For example, the Swedish Hip Register has shown that prospectively and systematically collected data have reduced revision rates by describing trends in outcomes adjusted for case-mix factors and early problems (Karrholm 2010). Antibiotic treatment for patients with hard-to-heal ulcers was reduced from 71% before registration to 29% after registration and feedback (Oien and Forssell 2013). A collaborative cohort study of 5 ICUs in the USA showed that an evidence-based intervention resulted in a large sustained reduction (up to 66%) in the rate of catheter-related bloodstream infection, which was maintained throughout the 18-month study period (Pronovost et al. 2006). A study of 13 patient registries in 5 countries demonstrated that these systems have great potential to improve health outcomes and lower healthcare costs (Kuenen et al. 2011).

Recently, reports from Sweden have indicated that spine registries have a positive effect on healthcare, i.e. on patient-related outcomes (complying with the recommendations in Table 3). For example, today the national mean length of stay (LoS) for surgery for lumbar disc herniation is 2 days, with a range of 0–4 days. After introduction of new routines, the LoS in 1 university hospital was reduced from 4 days to 2.5 days, giving the same patient-related outcomes at lower costs (Stromqvist et al. 2014). Another example is a change in surgical procedures in elderly patients with lumbar spinal stenosis (LSS). 2-year registry data on 8,785 elderly patients showed that surgery can be limited to an invasive procedure of decompression alone, in order to avoid unnecessary complications associated with fusion procedures (Forsth et al. 2013). These registry findings have recently been confirmed with a multicenter RCT involving 229 elderly patients with 1- or 2-level LSS. After 2 years, no benefit from adding fusion to decompression surgery was found, which means that in this population a less invasive procedure of decompression can reduce the number of complications and costs to society (Forsth et al. 2014).

Quality of studies based on registry data

In the present study, only half of the publications could be regarded as having a low risk of bias, as assessed on the Newcastle-Ottawa scale (NOS). The main weaknesses of the studies included were the inaccurate descriptions of the patients and of the interventions studied, and the lack of long-term assessment of the outcome. We therefore recommend use of the STROBE guidelines for reporting of observational studies (von Elm et al. 2007) (Table 3, rec. 3). Moreover, as it is known that changes in patient-related outcomes are seen during the first year after spine surgery (Weinstein et al. 2009, Desai et al. 2013, Mannion et al. 2013, Adogwa et al. 2014); a minimum follow-up period of 1 year is recommended (Table 3, rec. 4). As stability of patient-reported outcome measures (PROMs) results is seen between 2 and 4 years (Weinstein et al. 2009, Adogwa et al. 2014), longer follow-up periods are desirable. Another weakness causing high risk of bias according to NOS is that the assessment of outcomes in all studies was performed with PROMs. As PROMs are self-reported measures, these measures are assessed as being of low quality by NOS (Appendix 2, see Supplementary data). Although of lower quality methodologically, PROMs are the recommended outcome measures in spine surgery (McCormick et al. 2013), as there is no valid biomedical measure currently available to evaluate recovery after a spinal intervention. Although they are usually defined as patient-related clinical outcomes, reoperation and complications are in fact process measures for a complicated course to an endpoint defined by PROMs.

Methodology of studies based on registries

Although all 34 studies met the 6 inclusion criteria for registry characteristics (Drolet and Johnson 2008) (Table 3, rec. 5), we found various analytical approaches. Thus, we cannot give the best approach to use when comparing institutions in the search to identify best practices. In measurement of patient-based outcomes, the PROMs used should ideally fulfill the criteria for good measurement properties (Terwee et al. 2007). Recently, consensus was reached within the ICHOM collaboration on which PROMs should be recommended for the evaluation of outcomes of interventions for degenerative lumbar spinal disorders (Oswestry disability index (ODI), numeric pain rating scale (NPRS)—back and leg, and EuroQol 5 dimensions (EQ5D) (ICHOM, 2014) (Table 3, rec. 6). Although as yet only 3 lumbar spine patient registries (SweSpine, Dutch Spine Surgery Registry, and Spine Tango) use the specific ICHOM PROMs, every other registry studied has already evaluated functional status, pain intensity, and quality of life with PROMs, but the assessment used other tools.

To prevent selection bias (Wood et al. 2004) and to explain real differences in outcomes between institutions, multivariate approaches with adjustment for covariates (corrections for differences in characteristics of patients treated in hospitals; “case-mix adjustments”) and correction for chance variation (reliability adjustments) are needed (Iezzoni 1995, Wouters et al. 2008, Dimick et al. 2010, Desai et al. 2013) (Table 3, rec. 7). A shortcoming of these techniques is that they only account for known covariates. In this systematic review, we found that a large variety and number of covariates are included in different registries. A recently published study showed that a large number of patient characteristics (biomedical, psychosocial, and health-related indicators) could influence the outcome of interventions for lumbar spinal disorders or maintain the complaints (van Hooff et al. 2014). Within the ICHOM collaboration, consensus was reached to use a minimum set of factors: age, sex, education level, work status, duration of sick leave, smoking status, comorbidities, BMI, duration of back/leg pain, morbidity state, diagnosis and indication surgery, need for continuous analgesic use, prior intervention, baseline patient-reported disability, back and leg pain baseline, and health-related quality of life (ICHOM 2014) (Table 3, rec. 7). Currently, none of the registries collect data on all of these recommended factors. In the countries influenced by the SweSpine registry format (Sweden, Norway, Denmark, and the Netherlands), consensus has been reached to implement all these factors in the registry, to allow them to be used as covariates in future benchmark analyses. When benchmarking across centers, it has been suggested recently that together with these patient-related covariates, center-specific characteristics might also influence the outcome of surgery in degenerative lumbar spine disorders (Desai et al. 2013).

In the registries, the 12-month follow-up on PROMs varied from 20% (British Spine Registry) to 88% (NRT en el SNS). A suggested rule of thumb is that a loss to follow-up of more than 20% would probably lead to bias in the results, whereas a rate of less than 5% would not (Sackett et al. 2000, Schulz and Grimes 2002). A recently performed study on Norwegian registry 2-year follow-up data on 633 patients showed that a loss to follow-up of 22% would not alter conclusions about the outcome of interventions (Solberg et al. 2011). Efforts should be made to increase 12- and 24-month follow-up responses to 60–80% (Table 3, rec. 8). Although patients are 3 times more likely to respond when invited for follow-up visits (Solberg et al. 2011), it is too demanding for all parties involved to arrange long-term follow-up visits in large patient registries (Fritzell et al. 2006). Solberg et al. (2011) found that forgetfulness is the most important reason for not responding, which could possibly be prevented by modern communication techniques such as text messages and e-mail (Table 3, rec. 9). To handle missing data in most registries, analyses are only performed on complete cases. To understand potential sources of bias, a non-responder analysis on baseline characteristics should be provided (Table 3, rec. 10). Statistical sensitivity techniques are available to test whether there is bias present. When data are missing at random, indicating that the missing data are related to other observed or documented patient data but not to unobserved outcomes, we recommend multiple imputation techniques (Table 3, rec. 11). The major advantage of this method over single imputation techniques or “complete cases only” is that it does not underestimate variability (Twisk and de Vet 2002, Donders et al. 2006).

Strengths and limitations

To evaluate the risk of bias in the studies included, as a quality assessment, we defined a cutoff value (total score ≥ 6 (75%)) on the Newcastle-Ottawa scale (NOS) for cohort studies. However, research is needed to identify whether this is the correct tool for assessment of the risk of bias in patient registries. Another limitation is that selection bias might be present in the spine registries identified. We identified 2 types of registries: national and institutional. The national registries are in most cases part of an obligatory, government- or insurer-driven need for quality control and/or audit. The multicenter institutional registries carry the risk of selection bias (e.g. many institutional registries include premier spine institutes with selected patients) and even more when they are sponsored by industry (ESSG, SWISSspine, Canadian Spine Outcomes, and the Research Network, Indian Spine Registry) or by membership (SSE Spine Tango, British Spine Registry, National Spine Network Spine Outcomes Registry). Although we performed a thorough search and gained an overview of 25 large registries for degenerative spinal disorders, we cannot rule out that more spine registries exist.

The main strength of this study was that we adopted a systematic approach, including a systematic search and an appraisal of quality. In addirion, we conducted a survey among representatives of all the known registries to add information to that found in the literature. To increase the response, we contacted (successfully) all the representatives of the spine registries to complete our data.

Conclusions

Currently, despite there being some evidence in other fields of healthcare, there is a lack of evidence to either support or refute the impact that spine registries may have on the quality of spine care and, with that, on patient-related outcomes. To improve the quality of results published from registry data, we have formulated several recommendations. With the first indications of the effects of the SweSpine registry already known (e.g. improved outcomes after feedback on length of stay and no patient-related benefit from adding fusion to decompression surgery), we believe that application of these recommendations could lead to spine registries demonstrating trends and outcomes, monitoring the quality of spine care delivered, resolving controversies in the management of degenerative spinal disorders, and ultimately improving the value of the care given to our patients.

Supplementary data

Appendices 1–3 are available at Acta’s website (www.actaorthop.org), identification number 8170.

MH, WJ, and PF had the idea for the study. All authors contributed to the design of the study. RO provided methodological input, and MK and PF provided clinical input during the protocol phase. MH and WJ designed the search strategy for the literature search and developed the survey. They performed the study selection, appraised the quality of the literature, and performed the data extraction. They also analyzed the data both for the systematic review and for the survey. All the authors contributed to interpretation of the results. MH, WJ, and PF drafted the manuscript. MH and WJ wrote the appendices. All the authors critically revised the manuscript and read and approved the final version.

We thank the members of the ICHOM Low Back Pain Working Group and all the participants who completed the spine registry survey. We especially thank Caleb Stowell (ICHOM) for his support in building and managing the survey in the Harvard Business Qualitrics online web-application and Serge Stommels for his support in translating Russian texts in relation to the Russian Spine Registry.

No competing interests declared.