Determination of Limits for a Linear Regression or Calibration Curve

Abstract

Trace analysis has become increasingly more important because of its connection with recent environmental, chemical, and biological concerns; therefore, measurement techniques often must be evaluated particularly for their ability to measure low concentrations with acceptable accuracy and precision. A methodology is presented here for determining objectively the lower (or upper) limit associated with a linear regression; that is, the point below (or above) which a regression model fails. Methods are also given for determining, provided the data include multiple observations at some x values, whether problems observed beyond the limit are due to increased variability or to breakdown of the linear relationship. The methodology is applied to calibration curve data from four chlorine measurement techniques, to estimate lower limits and to show that the principal problem found below the lower limit is dramatically increased variability.

• Accuracy
• Calibration
• Heteroscedasticity
• Linear Regression
• Nonlinearity
• Precision

Additional information

Notes on contributors

William H. Swallow

Dr. Swallow is an Associate Professor in the Department of Statistics.

J. Richard Trout

Dr. Trout is an Associate Professor in the Department of Statistics of Cook College. He is a member of ASQC.