External validity of a population-based study on osteoporosis and fracture

Abstract

Background and purpose — Little is known about the characteristics of non-participants in epidemiological studies. We evaluated external validity by comparing fracture and mortality rate in participants and non-participants in a longitudinal study on risk factors for fracture.

Methods — 1,604 randomly selected women, 75 years of age, were invited to attend a study on osteoporosis and fracture. 1,044 women attended the study (participants) and 560 women did not participate (non-participants). Fracture data for all were obtained prospectively from radiographic records. Mortality data were obtained through the population register. Mean follow-up was 13 (11–15) years. Cumulative survival was compared with the log-rank test. Fracture incidence rates per 1,000 person-years were compared with Mann-Whitney U-tests. In addition, fracture comparisons were made with the cumulative incidence function and Gray’s test.

Results — 454 participants (44%) died during the follow-up, as compared to 372 of the non-participants (66%) (p < 0.001). The fracture incidence rate for any type of fracture was 43 for participants and 47 for non-participants (p = 1.0). The fracture incidence rate for typical osteoporotic fracture was 36 for participants and 39 for non-participants (p = 0.6). The corresponding values for distal forearm fracture were 11 and 7 (p = 0.002), they were 8 and 9 for proximal humerus fracture (p = 0.9), 13 and 10 for vertebral fracture (p = 0.007), 15 and 18 for hip fracture (p = 0.8), and they were 6 and 5 for pelvic fracture (p = 0.3). The cumulative incidence function confirmed the results.

Interpretation — Our findings suggest that participants had a lower mortality rate than non-participants. Distal forearm and vertebral fractures were more frequent in participants. However, the external validity for fractures in general appeared to be satisfactory.

In epidemiological research, there is very little prospective information on disease events and other characteristics of non-participants compared to what is available on voluntary participants. Understanding the nature of non-participation is of importance; systematic differences between non-participants and participants may cause loss of external validity of study outcomes. To obtain a wider understanding of possible non-response bias, the group of non-participants must be characterized. Furthermore, it may enhance recruitment rates (Hartge 2006, Galea and Tracy 2007).

Osteoporosis and associated fractures have far-reaching consequences, and incidence rates are projected to rise, which underscores the importance of identifying individuals who are at high risk (Cummings and Melton 2002, Melton 2003, Burge et al. 2007). Most studies describing risk factors for osteoporosis and fracture rely on knowledge acquired through studies based on active, voluntary participation. On some occasions, conclusions based on outcomes from such studies can be generalized to other population groups and conditions. The external validity of study outcomes is dependent on how much the study population is a representative sample of the background population. However, in studies relying on active participation, there are inevitably those who decline to participate. The reasons for non-participation may vary and be more or less known. External validity is not solely dependent on the response proportion; high response rate constitutes an enhancing factor, although it is not necessarily associated with an unbiased estimation of study outcome. If the composition of the group of non-participants differs substantially from that of the group of participants, it may reduce the external validity (Hartge 2006, Galea and Tracy 2007).

Studies from other disease areas have suggested that there may be poorer health status, lower socioeconomic status, increased cognitive impairment, higher mortality rates, and higher cancer rates in non-participants (Bisgard et al. 1994, Norton et al. 1994, Hoeymans et al. 1998, Riedel-Heller et al. 2000, Manjer et al. 2001, Hara et al. 2002, Kauppi et al. 2005, Galea and Tracy 2007, Suominen et al. 2012). However, efforts to provide characteristics of non-participants in osteoporotic research have seldom been made. To our knowledge, prospective mortality and fracture data from a complete population-based cohort including non-responders have never been described.

Differences in fracture patterns before the baseline investigation between participants and non-participants in the OPRA cohort have been described previously (Gerdhem and Akesson 2007). The results suggested that studies on osteoporosis and fracture might attract individuals who have previously sustained fractures. The fracture rate before inclusion was greater in women who attended the full investigation than in those who did not.

The objective of the present study was to prospectively compare mortality and fracture rates between participants and non-participants in the OPRA cohort. Our aim was to determine whether the differences in fracture rate between participants and non-participants in the same study population (Gerdhem and Akesson 2007) remained during more than 10 years of follow-up, and to depict any differences in mortality rates, thus defining the external validity of this cohort in terms of mortality and fracture rate.

Material and methods

Study population

The Osteoporosis Prospective Risk Assessment (OPRA) study was started in the city of Malmö, Sweden (Gerdhem et al. 2003). The aim was to study risk factors for osteoporosis and fractures in elderly women. From the population files, 75-year-old women—all residents of Malmö—were randomly recruited between the years 1995 and 1999. Letters of invitation were sent out to 1,604 women 1 week after their seventy-fifth birthday. No exclusion criteria were applied. For those who did not respond, reminders were sent by mail. If no response was obtained, additional attempts were made by telephone. Details of the study have been published elsewhere (Gerdhem et al. 2003, Gerdhem and Akesson 2007).

Of the 1,604 women invited to this cohort, 1,044 women responded to the study invitation by participating in the extensive investigations at the research facility. As earlier reported, 94% of the non-participants gave reasons for not participating: illness 27% or unwillingness 67% (Gerdhem and Akesson 2007). The last data retrieval for this specific study was in July, 2010. Mean follow-up was 13 (11–15) years from baseline.

The study was approved by the ethics committee of Lund University (LU 363-02).

Mortality

All Swedish residents have a unique 10-digit personal identification number based on their date of birth. This number is always used when they are in contact with public services such as healthcare. For all 1,604 women, participants and non-participants, mortality data were obtained through the national population registry. Population mortality data are considered to be complete in Sweden.

Fractures

To avoid inconsistency in fracture registration between the participants and non-participants, fracture data were obtained solely through the Department of Radiology at Skåne University Hospital in Malmö, using the personal identification number and radiology files. Thus, no active participation was needed in this study to complete registration of fractures. We acquired fracture data from several time points throughout the follow-up. These data were retrieved without knowing which group the individual belonged to.

There is only one hospital with a radiology department in Malmö. During the study period, there were at times 2 additional private radiological clinics, but with no capacity to do emergency fracture management. Thus, most patients who needed treatment were referred to the hospital. In terms of identifying fractures in the study population, the radiological records can be regarded as being almost complete. However, fractures treated elsewhere, and not requiring radiological checks at Skåne University Hospital in Malmö, may not have been registered. The proportion of fracture cases missed from only using the hospital fracture registry has been estimated to be less than 3% (Jonsson 1993). For the few cases where the information from radiology files was unclear, radiological images were requested and reviewed for diagnosis and classification. Regarding vertebral fractures, only symptomatic fractures were registered. Vertebral fractures where there was no evidence of symptoms were not regarded as symptomatic (those noted on chest radiographs, for example).

Statistics

Cumulative survival was calculated with the Kaplan-Meier analysis and log-rank test.

Comparisons were made for women with any type of fracture, osteoporotic fracture, and non-osteoporotic fracture. Distal forearm, proximal humerus, hip, pelvic, and clinical vertebral fractures were considered to be typical osteoporotic fractures and were also analyzed separately. Only the first observation of a specific fracture was accounted for, regardless of the occurrence of previous fractures. Each observation was considered to be independent in the statistical analysis. Multiple fractures were defined as sustaining 2 or more fractures of any type.

Fracture incidence rates were calculated per 1,000 person-years and group comparisons were made using the Mann-Whitney U-test. To account for mortality as a competing risk, fracture rates were compared between groups with the cumulative incidence function and Gray’s test (Gray 1988, Fine and Gray 1999). Descriptive statistics and graphical methods were used to characterize the data.

Statistical analyses were carried out using the SPSS software package version 19. The graphs of the cumulative incidence function were made in SPSS by the use of a syntax, modified from Porta et al. (2008). Gray’s test was performed using the %CIF macro in the SAS software package (Lin et al 2012). We used the 5% level of significance.

Results

Mortality

In the entire cohort, 778 (49%) of 1,604 women were alive and 826 (52%) had died by the end of follow-up (mean 13 years). Mortality was lower in the participants; 454 (44%) of the 1,044 participants and 372 (66%) of the 560 non-participants had died at the end of follow-up (p < 0.001) (Figure 1).

Figure 1. Kaplan-Meier curves showing cumulative survival in participating and non-participating women in the Osteoporosis Prospective Risk Assessment study with up to 14 years of follow-up. Inclusion in this study was at the age of 75. Over time, mortality was higher in non-participating women than in participating women.

Fractures

699 women (44%) sustained at least 1 fracture, with 591 (37%) sustaining a typical osteoporotic fracture. The incidence rates for distal forearm fracture and clinical vertebral fracture were statistically significantly higher in participants than in non-participants (Table). The incidence rates for proximal humerus, hip, and pelvic fracture were similar between participants and non-participants (Table), and when fractures were categorized as “any fracture”, “osteoporotic fracture” or “non-osteoporotic fracture”. The incidence rate for multiple fractures was higher in the participants (Table).

Incidence rates (IR) calculated per 1,000 person-years during the mean follow-up of 13 years. Only the first observation of a fracture was accounted for, with the exception of “multiple fractures”, which was defined as sustaining 2 or more fractures of any type

Fracture type	Participants (n =1044) IR (n)	Non-participants (n = 559) IR (n)^a	p-value ^b
Any fracture	43 (468)	47 (231)	1.0
Osteoporotic fracture	36 (398)	39 (193)	0.6
Non-osteoporotic fracture	14 (155)	14 (69)	0.3
Multiple fractures	20 (222)	14 (70)	<0.001
Osteoporotic fracture ^c
Distal forearm	11 (115)	7 (34)	0.002
Proximal humerus	8 (84)	9 (43)	0.9
Clinical vertebral	13 (145)	10 (50)	0.007
Hip	15 (165)	18 (88)	0.8
Pelvis	6 (61)	5 (26)	0.3

^a Of the 560 non-participants, 1 died before receiving the letter of invitation and was excluded.

^b Mann-Whitney U test

^c Distal forearm fractures, proximal humerus fractures, clinical vertebral fractures, pelvic fractures, and hip fractures were regarded as typical osteoporotic fractures and analyzed separately and as a group: “osteoporotic fracture”.

The cumulative incidence function and Gray’s test for distal forearm fracture and clinical vertebral fracture showed a higher fracture rate in participants (Figure 2). The rates of proximal humerus fracture and hip fracture were similar in participants and non-participants.

Figure 2. Cumulative incidence function curves for fracture in participants and non-participants, from inclusion at the age of 75. Distal forearm fracture and clinical vertebral fracture were more frequent in participants than in non-participants. There was no significant difference in the occurrence of proximal humerus fracture or hip fracture between participants and non-participants. The p-value refers to Gray’s test for equality of cumulative incidence functions. a. Distal forearm fracture. b. Proximal humerus fracture. c. Clinical vertebral fracture. d. Hip fracture.

Discussion

Our findings suggest that mortality during follow-up was lower in the voluntary study participants than in the non-participants. This is in line with a previous study in the field of osteoporosis (Hasserius et al. 2002), and with studies in other disease areas where non-participation has been associated with higher mortality and inferior health status (Bisgard et al. 1994, Norton et al. 1994, Hoeymans et al. 1998, Riedel-Heller et al. 2000, Manjer et al. 2001, Hara et al. 2002, Kauppi et al. 2005, Galea and Tracy 2007, Suominen et al. 2012). Although not all studies agree (Heilbrun et al. 1991, Buist et al. 2004), it is reasonable to believe that study participants are a generally healthier group than non-participants.

In this prospective study, which followed the same women over at least 11 years from the age of 75, distal forearm and vertebral fracture rate were statistically significantly higher in women who participated than in women who did not participate. To our knowledge, similar findings regarding upper extremity fragility fractures in participants and non-participants have not been described previously. A previous cross-sectional study suggested that bias due to non-response has a minor effect on fracture rates in studies on vertebral fractures (Oneill et al. 1995). However, our findings suggest that vertebral fracture rate may be slightly higher in participants.

Hip fracture rates and proximal humerus fracture rates were similar in participants and non-participants. An earlier report has shown a lower rate of hip fractures in participants than in a subset of non-participants, although the study setting was different and the non-participants were only followed for 2 years (Buist et al. 2004).

Multiple fractures during follow-up were more common in participants than in non-participants. The higher rate of multiple fractures may have been the result of an increased distal forearm and vertebral fracture rate in the participants. Another plausible explanation would be that this study on risk factors for osteoporosis and fracture attracted women who were more prone to incur fractures. A history of previous fracture increases the risk of subsequent fracture (Klotzbuecher et al. 2000, Kanis et al. 2004). The participants may have been more likely to have had a previous fracture, as indicated by a previous study on this cohort (Gerdhem and Akesson 2007), and in line with former studies (Oneill et al. 1995, Hasserius et al. 2002).

A previous report on this cohort did not show any differences in specific fracture rates (distal forearm, proximal humerus, hip, vertebrae) prior to inclusion at age 75 years. However, the rates of combined osteoporotic fractures and multiple fractures were more frequent in participants before inclusion (Gerdhem and Akesson 2007).

The study design had several advantages. It was based on a large well-defined prospective cohort. Mortality data from the population registry can be regarded as complete. For all subjects, the fracture data were obtained solely from the radiological records, so they did not require any self-reporting, which may be a source of bias (Ismail et al. 2000, Gerdhem and Akesson 2007).

One limitation of the study was the inevitable lack of additional information from the non-participants, apart from fracture and mortality. It is therefore difficult to draw additional conclusions regarding activity level, health status, or other possible factors that might explain the differences in fracture rate and mortality patterns. Furthermore, we did not register asymptomatic vertebral fractures. However, we do not believe that there were systematic errors with differences in fracture ascertainment between the groups.

A longer lifespan in participating women means a longer time to be at risk of fracture. Also, fracture risk increases with increasing age. Participating women are therefore at risk of sustaining more fractures, which might explain part of the differences seen. We used different methods in an effort to compensate for the discrepancies in observation time caused by mortality, cumulative incidence function, and incidence rates.

Our findings suggest that distal forearm fracture rate and clinical vertebral fracture rate may be slightly overestimated in studies with similar settings, due to non-response bias. However, the rates of any type of fracture, of any osteoporotic fracture, of hip fracture, and of proximal humerus fracture were similar between participants and non-participants. Whether these findings can be generalized and are applicable to women of other ages, to men, to different geographical or ethnic settings, or to studies with lower participation rates cannot be determined from this study. However, caution is required when interpreting data from population-based studies relying on active participation. Care should be taken in the recruitment process, and non-response issues should be discussed in order to further enhance the external validity of study outcomes.

In summary, in this population-based study involving osteoporosis and fracture risk in elderly women, when followed for over a decade, the non-participants had higher mortality than the participants. Distal forearm fractures and vertebral fractures were more frequent in participants than in non-participants. However, the external validity was satisfactory for fractures in general, and for humerus fractures and hip fractures.

AW: acquisition of data; design, analysis, and interpretation of data; and drafting and revision of the manuscript. KÅ: acquisition of data and revision of manuscript. PG: conception and design; acquisition of data; and revision of manuscript. Statistical analysis was done in collaboration with Per Näsman, KTH Royal Institute of Technology, Stockholm, Sweden.

Funding was obtained from the regional agreement on medical training and clinical research (ALF) between Stockholm County Council and Karolinska Institutet, and from Karolinska Institutet Research Funds, the Swedish Research Council (K2009-53X-14691-07-3, K2010-77PK-21362-01-2), FAS (Grant 2007-2125), the Greta and Johan Kock Foundation, the A. Påhlsson Foundation, A. the Osterlund Foundation, the Knut and Alice Wallenberg Foundation, the H Järnhardt Foundation, the King Gustav V and Queen Victoria Foundation, the Malmö University Hospital Research Foundation, and the Research and Development Council of Region Skåne, Sweden.

No competing interests declared.