Evaluation of Approaches to Analyzing Continuous Correlated Eye Data When Sample Size Is Small

ABSTRACT

Purpose: To evaluate the performance of commonly used statistical methods for analyzing continuous correlated eye data when sample size is small.

Methods: We simulated correlated continuous data from two designs: (1) two eyes of a subject in two comparison groups; (2) two eyes of a subject in the same comparison group, under various sample size (5–50), inter-eye correlation (0–0.75) and effect size (0–0.8). Simulated data were analyzed using paired t-test, two sample t-test, Wald test and score test using the generalized estimating equations (GEE) and F-test using linear mixed effects model (LMM). We compared type I error rates and statistical powers, and demonstrated analysis approaches through analyzing two real datasets.

Results: In design 1, paired t-test and LMM perform better than GEE, with nominal type 1 error rate and higher statistical power. In design 2, no test performs uniformly well: two sample t-test (average of two eyes or a random eye) achieves better control of type I error but yields lower statistical power. In both designs, the GEE Wald test inflates type I error rate and GEE score test has lower power.

Conclusion: When sample size is small, some commonly used statistical methods do not perform well. Paired t-test and LMM perform best when two eyes of a subject are in two different comparison groups, and t-test using the average of two eyes performs best when the two eyes are in the same comparison group. When selecting the appropriate analysis approach the study design should be considered.

KEYWORDS:

• Correlated eye data
• generalized estimating equations
• linear mixed effects model
• paired t-test
• two sample t-test
• small sample size

Acknowledgements

The results of our study were partially presented at the Joint Statistical Meeting (JSM), August 3-8, 2013, Montreal, Quebec, Canada.

Declaration of interest

The authors report no conflict of interest. The authors alone are responsible for the writing and content of this article.

Funding

Our study was supported by vision core grant P30 EY01583-26 from the National Eye Institute, National Institutes of Health, Department of Health and Human Services, an unrestricted grant from Research to Prevent Blindness, and the Mackall Trust Funds to the University of Pennsylvania.

Additional information

Funding

Our study was supported by vision core grant P30 EY01583-26 from the National Eye Institute, National Institutes of Health, Department of Health and Human Services, an unrestricted grant from Research to Prevent Blindness, and the Mackall Trust Funds to the University of Pennsylvania.