Assessing Agreement Between Two Methods of Quantitative Measurements: Exact Test Procedure and Sample Size Calculation

Abstract

Assessment of agreement between measurement methods is an important problem in medical practice and research. Several two one-sided tests (TOST) procedures have been proposed to evaluate whether two methods have sufficient agreement. These TOST techniques of limits of agreement or normal percentiles are natural generalizations of the favorable TOST procedure of mean differences. But their fundamental behavior of Type I error control has not been examined. This article describes an improved and exact approach for declaring agreement between two measurement methods when the range of central proportion of paired differences is within the designated threshold interval around zero. Theoretical examinations and numerical investigations are provided to justify the advantages of the proposed approach and to illustrate the shortcomings of the existing TOST procedures for assessing agreement between two methods. To facilitate the suggested agreement appraisal in design planning, the corresponding power and sample size formulas are also derived. Both SAS and R algorithms are presented for conducting agreement test, power calculation, and sample size determination in method comparison studies.

Keywords:

• Equivalence trials
• Limits of agreement
• Method comparison
• Percentile
• Tolerance interval

Supplementary Materials

SAS and R computer algorithms for performing the recommended test procedure, power computation, and sample size determination.

Acknowledgments

The author would like to thank the editor, the associate editor, and two referees for their constructive comments and helpful suggestions that led to an improved article.

Additional information

Funding

This work was supported by the Ministry of Science and Technology, Taiwan grant MOST 107-2410-H-009-024-MY3.