Abstract
Background: Pretesting a questionnaire is an important activity before the start of the main epidemiological study. Aim: The study tested the reliability (internal consistency) of the modified ATS-DLD-78 adult respiratory diseases questionnaire developed by the British Medical Research Council. Results: The questionnaire was found to be reliable and admissible with an overall Cronbach’s alpha coefficient of α = 0.74. Conclusion: The questionnaire is a reliable instrument for data collection and can be a useful tool for collecting data in a developing country. The questionnaire will be used to study the association between respiratory and cardiovascular diseases among older people and community's proximity to mine dumps
Introduction
Pretesting a questionnaire in a relevant population is an important activity before the start of the main epidemiological study. This is true for all questionnaires including those that have been used many times as well as internationally. A pilot study can be used for pretesting a particular research instrument in preparation for a major study. This will ensure that the questionnaire is comprehensible, appropriate, and that the questions are well-defined and presented in a consistent manner.Citation1 Pilot studies can be very informative for both the researchers conducting them, as well as others doing similar work in that they can avoid making the same errors in their studies.Citation2 One of the benefits of the pilot study is that it might give advanced warning about whether the proposed methods or instruments are unsuitable, understandable or too complicated.Citation3 Research studies in South Africa have used self-administered modified standard questionnaires to study respiratory and cardiovascular diseases,Citation4,5 but barely mention whether the questionnaire was tested for reliability or not.
The British Medical Research Council approved the American Thoracic Society and the Division of Lung Diseases (ATS-DLD-78) questionnaire as a standard to conduct epidemiological surveys for chronic respiratory diseases.Citation6 In this study, the questionnaire was modified by the addition of two questions about arrhythmias and myocardial infarction. To date, no studies in South Africa have tested the reliability of the ATS-DLD-78 questionnaire. This study is part of a larger project initiated by the Mine Health Safety Council of South Africa (MHSC) on communities located near mine dumps in the Gauteng and North West provinces. The aim of the study was to the test the reliability (internal consistency) of the ATS-DLD-78 respiratory symptoms and diseases questionnaire among older people (55 years and above).
Method
Between January and February 2013, a pilot study was conducted to test the reliability (internal consistency) of the modified ATS-DLD-78 adult respiratory symptoms and diseases questionnaire developed by the British Medical Research Council among older people (55 years and above) in Pennyville, Soweto, Gauteng, South Africa. People that reside in this township are predominantly black and of lower socio-economic status with an estimated population size of 2757.Citation7 A unique aspect of the township is that it is located near a mine dump. The questionnaire was self-administered to measure the presence of emphysema, chronic bronchitis, pneumonia, hay fever, asthma, wheezing, cough, family history, chest illness, past illnesses, arrhythmia and myocardial infarction. Each respiratory symptom and disease and cardiovascular disease had a subset of questions.
A knock-on-the-door approach was used to recruit study participants. The questions were mainly in English. Streets within the community were randomly selected. Four to five houses were then randomly selected in each street. There were 48 participants in this study. The participants that took part in this study were:
| • | Male or female | ||||
| • | Fifty-five (55) years and above | ||||
| • | Living in Pennyville | ||||
If a selected household had no older people or no one at home during the visit, or were unwilling to participate, the researcher proceeded to the next household. This study was approved by the ethics committee of the University of Pretoria (235/2011).
The collected questionnaire data was double entered into a database set up in Epi-Info version 3.5.3. Data were analysed using Stata version 12. Data collected during the study was divided into the following categories:
Demographics: Population group, sex, marital status, date of birth, level of education and number of years living in the community being researched.
Respiratory diseases and symptoms: Cough, phlegm, wheezing, cold, asthma, pneumonia, breathlessness, past illnesses, hay fever, bronchitis, emphysema and chest injuries.
Cardiovascular diseases and symptoms: Arrhythmia, heart trouble, high blood pressure and myocardial infarction/heart attack.
Family History: Respondents have to report whether their parents had suffered from respiratory and/or cardiovascular diseases.
Home heating and fuel: Coal, wood, liquefied petroleum (LP) gas, electricity, fuel oil, and paraffin.
Occupational History: Respondents had to report whether they have worked in a dusty area, and/or exposed to gas/chemical fumes in their work areas.
In this study, only questions pertaining to the outcome variables, respiratory and cardiovascular diseases were tested for reliability during analysis, and are shown in the supplementary tables. Questions on predictor variables and outcome variables, such as family history, home heating, fuel and occupational history, were excluded.
Descriptive statistics were used to explain data, means, frequencies, standard deviations, and Cronbach’s alpha statistics to measure internal consistency of the group questions of outcome variables in the questionnaire. A Cronbach’s alpha coefficient of 0.7 ≤ α < 0.8 is considered acceptable.Citation8 In this study, an alpha coefficient of 0.7 and above indicated that the set of questions was reliable. Factor analysis was then further carried out to remove questions from the questionnaire that were correlated. The factor or subset of questions with an Eigenvalue greater than 1.0 was retained.Citation9
Results
A total of 48 predominantly black respondents participated in the questionnaire interview; 67 % were females. The majority of the respondents had primary education as the highest form of education level. Respondents’ ages ranged from 55 to 100. Table summarizes the demographic characteristics of the respondents. Table gives a synopsis of the Cronbach’s alpha coefficients for the questions on the outcome variables. All questions were considered acceptable and reliable, except that of past illness. Question on cough, breathlessness and wheezing had Cronbach’s alpha coefficients of 0.96, 0.88, and 0.78, respectively. The average number of years that the respondents had each condition was 5±5.3, 13±16.0 and 4±10.5, respectively. Questions of two additional outcome variables, namely myocardial infarction/heart attack and arrhythmia, were found to be acceptable and reliable with reliability coefficients of 1.00 and 0.98, respectively. Questions regarding past illnesses had a reliability coefficient of 0.53, with each question having an Eigenvalue of 1.08, 0.05 and -0.26, as shown in Supplementary Tables 1 and 2.
Table 1: Demographics of the participants
Table 2: Reliability coefficients of the outcome variables
Supplementary Table 3 shows the uniqueness of each question that is not shared with other questions in the questionnaire. The question “Did you have any lung trouble before the age of 16?” had 0.88 uniqueness with an Eigenvalue of 1.08, “Have you had a bronchitis attack?” had a uniqueness of 0.52 with an Eigenvalue of 0.05, and the question “Was the bronchitis attack confirmed by the doctor?” had a uniqueness of 0.45 with an Eigenvalue of -0.26.
Discussion
The aim of the pilot study was to the test the reliability of the ATS-DLD-78 respiratory symptoms and diseases questionnaire among older people (55 years and above) in Pennyville, a community located near a mine dump in Soweto, Gauteng, South Africa. The results of this study confirm that the ATS-DLD-78 questionnaire, with the additional questions, is still a reliable instrument that can be used to collect data, and that the respondents could comprehend the questions contained despite their level of education. All the subset questions for outcome variables such as cough, phlegm, wheeze, breathlessness, chest cold and chest illness, pneumonia, asthma, chronic bronchitis, emphysema, myocardial infarction/heart attack and arrhythmia were found to be reliable, except those relating to past illness.
The findings of this study are in agreement with other worldwide studies that have piloted the ATS-DLD-78 respiratory symptoms and diseases questionnaire.Citation10,11 In all these studies, the ATS-DLD-78 respiratory symptoms and diseases questionnaire was deemed reliable.
The question: “Did you have any lung trouble before the age of 16?” had 0.88 uniqueness with an Eigenvalue of 1.08 and was retained in the questionnaire since it has an Eigenvalue of greater than 1.0. A unique variance of 0.88 indicates that 88% of the variance was not shared with any other questions in the questionnaire. The questions: “Have you had a bronchitis attack?” and “Was the bronchitis attack confirmed by the doctor” will be dropped from the questionnaire because they had lower Eigenvalues for the main study. The ATS-DLD-78 respiratory questionnaire is a reliable instrument for data collection, and it will be a useful tool for collecting data in a developing country. Additional questions concerning cardiovascular diseases, i.e. heart attack and arrhythmias, were also admissible. The questionnaire will be used to study the association between respiratory diseases among older people and their proximity to mine dumps.
Competing interests
The authors declare no competing interest.
Supplementary material
Supplementary material for this article can be accessed here http://dx.doi.org/10.1080/23120053.2016.1156318.
OJID_A_1156318_Sup.pdf
Download PDF (114.2 KB)Acknowledgements
We are grateful to the Pennyville residents that took part in this study. This project was funded by the Mine Health Safety Council of South Africa (SIM 100801).
References
- Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract. 2004;10(2):307–12.10.1111/jep.2004.10.issue-2
- Thabane L, Ma J, Chu R, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol. 2010;10(1):1–15.10.1186/1471-2288-10-1
- Van Teijlingen E, Hundley V. The importance of pilot studies. Nurs Stand. 1998;16(40):747–55.
- Nriagu J, Robins T, Gary L, et al. Prevalence of asthma and respiratory symptoms in south-central Durban, South Africa. Eur J Epidemiol. 1999;15(8):747–55.10.1023/A:1007653709188
- Phaswana-Mafuya N, Peltzer K, Chirinda W, et al. Self-reported prevalence of chronic non-communicable diseases and associated factors among older adults in South Africa. Glob Health Action. 2013;6(20936):1–7.
- Recommended respiratory disease questionnaires for use with adults and children in epidemiological research. 1978 [cited 2015 Sep 22]. Available from: www.thoracic.org.
- Statistics South Africa: statistical release (Revised) Census 2011. 2012 [cited 2015 Sep 23]. Available from: www.statssa.gov.za.
- DeVellis RF. Scale development: theory and applications. Appl Soc Res Meth Series. Sage C, Chapel Hill, USA: University of Carolina; 1991. p. 115205.
- Torres-Reyna O. Getting started in factor analysis (using Stata 10). Princeton, NJ: Data and Statistical Services; 2012 [cited 2015 Jul 21]. Available from: http://dss.princeton.edu/training/Factor.pdf.
- Shaikh S, Nafees AA, Khetpal V, et al. Respiratory symptoms and illnesses among brick kiln workers: a cross sectional study from rural districts of Pakistan. BMC Pub Health. 2012;12(999):1–9.
- Comstock GW, Tockman MS, Helsing KJ, et al. Standardized respiratory questionnaires: comparison of the old with the new. Am Rev Respir Dis. 1979;119(1):45–53.