Randomized comparison of in vivo performance of two point-of-care tear film osmometers

Abstract

Purpose

To compare the in vivo precision of two commercially available point-of-care osmometers among normal subjects with no dry eye disease.

Methods

Twenty healthy adults with healthy ocular surfaces were evaluated by licensed eye care practitioners. All subjects had low Ocular Surface Disease Index score (<5), normal tear breakup time (>10 seconds), and no evidence of corneal fluorescein staining. Five consecutive measurements of tear osmolarity were measured on each eye using each of the two osmometers: the TearLab Osmolarity System (TearLab) and the I-Med i-Pen (i-Pen), for a total of 200 measurements per device. Performance of the osmometers was determined by specificity, estimated by the percentage of osmolarity data at or below the clinical cutoff (308 mOsm/L) and precision, and represented by the standard deviation per subject. In addition, to assess analytical performance, on each day of patient testing, standardized osmolarity quality control solutions (338 mOsm/L) were tested on the TearLab per manufacturer instructions. i-Pen manufacturer instructions do not neither provide for, nor recommend quality control procedures.

Results

The mean age of the 20 subjects was 27±8 years (range: 19–48 years, 16 females, four males). Over 2 months of testing, the TearLab reported analytical performance on quality control solutions of 335.8±4.2 mOsm/L with a coefficient of variation of 1.3%. In the subject cohort, 90.9% of TearLab measurements were in the normal range ≤308 mOsm/L. The i-Pen reported 37.5% of all measurements in the normal range. The average intra-subject osmolarity of the TearLab was 295.4±8.6 mOsm/L, which was significantly lower and less variable than the i-Pen, which reported an average of 319.4±20.3 mOsm/L (P<0.001). When the measurements were grouped by subject, the TearLab accurately identified 100% of subjects as normal while the i-Pen accurately identified only 15% of subjects as normal.

Conclusion

In this randomized comparative study of two point-of-care osmometers among normal, healthy non-dry eye subjects, the TearLab Osmolarity System demonstrated accuracy, precision, and agreement with clinical interpretation in line with the manufacturer claims. The i-Pen lacked sufficient performance to delineate subjects with and without dry eye disease.

Keywords:

• tear osmolarity
• TearLab osmolarity system
• i-Pen osmolarity
• precision

Introduction

Tear osmolarity is a physiological biomarker of ocular surface health and serves as a quantitative diagnostic for dry eye disease and its management.¹ Excess tear evaporation and/or decreased aqueous production results in abnormal tear osmolarity leading to tear instability and stress on the ocular surface. This cascade of events includes inflammation, ocular surface damage, and symptoms of discomfort.^1–6 Tear osmolarity is maintained through homeostatic, compensatory mechanisms in healthy subjects.^7,8 In dry eye disease, this homeostasis is lost, resulting in increased osmolarity and tear instability.^9–12

Measurement of tear osmolarity has historically required a large volume tear sample collection from the human eye, and thus early laboratory osmometers proved impractical for use at the point-of-care, in a clinical setting. Recent advances in technology have resulted in devices (TearLab and i-Pen) that both measure tear osmolarity at the point-of-care by measuring the electrical impedance of nanoliter samples of tear fluid. The TearLab Osmolarity System (TearLab) was broadly commercialized in 2012,¹³ with numerous studies having been published on the utility of TearLab osmolarity measurements in the diagnosis, grading of severity, and therapy tracking in dry eye disease as of the time of writing.^9,14–16

More recently, the I-Med i-Pen (i-Pen) has become available in Canada.¹⁷ The i-Pen measures tear osmolarity via electrical impedance of the ocular tissues on the palpebral conjunctival membrane.¹⁸ Its utility has been questioned in the literature, as in vitro data suggest that the i-Pen provides insufficient performance to accurately and precisely delineate osmolarity levels in the physiological range.¹⁹ However, no data on the precision or performance of the device when testing tear fluid of human subjects, as per device labeling, is available in the peer-reviewed literature, and it may be that the technical implementation of the previous study did not accurately reflect the operative mechanism of the i-Pen device.

It is well known that tear osmolarity varies with high temporal frequency in patients with dry eye disease, with the variability increasing with increasing severity of disease.^12,20 This is contrasted by very stable readings in normal subjects, both diurnally and over successive days.^12,21 As the i-Pen instructions for use do not provide for a method to perform quality control testing on laboratory standards, the only available method to evaluate device precision consistent with manufacturer instructions is to reproduce the expected range of osmolarity in a normal population. This is known as a biological control, which is an accepted approach to establish performance under Clinical and Laboratory Standards Institute guidelines.²²

The objective of this study was to compare the performance of the two osmometers in a healthy, normal human population, according to the manufacturers’ intended use statements. Non-dry eye subjects are likely to have a low, stable tear osmolarity, as previous reports suggest that there is minimal change in tear osmolarity over time in normals, given intact homeostatic mechanisms.^8,12 In this cohort, it is expected that variance between the two osmometers and deviation from the expected normal range (≤308 mOsm/L) would be attributed to analytical measurement errors, and not biological variations.²³

Methods

This prospective, device randomized, cross-sectional, single visit study included 20 asymptomatic subjects with no clinically evident dry eye disease or active ocular surface infection, allergy, or disease. Subjects were aged ≥18 years, were not using any ocular medications, and were non-contact lens wearers. To be included in the study, each subject was required to have an Ocular Surface Disease Index (OSDI) symptom score of <5, fluorescein tear breakup time >10 seconds, and no fluorescein corneal staining.^1,24 This study was conducted under Chesapeake Research Review LLC IRB approval, adhered to the tenets of the Declaration of Helsinki, and subjects provided written informed consent before testing.

If the subject passed the inclusion criteria tests, five consecutive bilateral osmolarity measurements were made with each of the two osmometers by a trained technician according to the manufacturers’ instructions. The order of testing was randomized to the osmometers and to the eye initially tested for each subject. All testing was completed with a single osmometer type before switching to the alternative osmometer. A new disposable Test Card (TearLab) or Single Use Sensor or (SUS; i-Pen) was used for every measurement.

The TearLab consists of two collection pens, one for right and other for left eye tear collection, a reader into which the collection pens are docked and single-use disposable Test Cards. The TearLab uses a temperature compensated impedance to measure tear osmolarity on a 50 nL tear fluid sample collected from the lower tear meniscus of the human eye.²⁵ The i-Pen consists of an SUS inserted into a handheld pen that analyzes tear osmolarity by pressing the tip of the SUS onto the lower, retracted conjunctiva. The manufacturer states that it measures impedance of the extracellular fluid contained in the eyelid tissue.¹⁸ On each day of study testing, the analytical performance of the TearLab was measured using National Institute of Standards and Technology traceable sodium chloride osmolarity quality control solution with a target of 338 mOsm/L, per the manufacturer’s Quality Control (QC) Guidelines.²⁵ No quality control or calibration procedures are neither provided for nor recommended as per the i-Pen’s labeling, and thus QC testing on the i-Pen was not performed.

Data were tested for normality prior to comparison using the Kolmogorov–Smirnov test. Accordingly, descriptive statistics, including mean, standard deviation, and distributions, were calculated across devices as well as per subject in Microsoft Excel. As this was a non-dry eye cohort, by broadly accepted clinical standards, the ability of each device to correctly identify the absence of dry eye disease (specificity; defined by osmolarity ≤308 mOsm/L) was compared as a percentage of all tests for each of the two devices and on a per subject basis.

We calculated a sample size of 20 subjects which provides 90% power with alpha error of 0.05 to detect a difference of 10 mOsm/L between the two osmometers with a standard deviation of 9 mOsm/L. The standard deviation of 9 mOsm/L was chosen based on the report in the literature on standard deviation of osmolarity among a population of non-dry eye subjects.¹⁰

Results

Analytical data from testing quality control materials on the TearLab were measured on 11 clinic days that spanned a little over 2 months (October 4 to December 14, 2016). The mean of these 24 data points was 335.8±4.2 mOsm/L, corresponding to an accuracy of 99.4% and a coefficient of variation (CV) of 1.3%, which was consistent with the device labeling of 1.5% CV.¹³ The average deviation from the target osmolarity of 338 mOsm/L was 2.2±4.2 mOsm/L with a range from −7 to +8 mOsm/L. About 100% of TearLab measurements were within 10 mOsm/L of the target osmolarity. The TearLab showed a normal distribution when measuring quality control material (P>0.15). No quality control testing was performed on the i-Pen per the manufacturer’s instructions for use.

The mean age of the 20 subjects was 27±8 years (range: 19–48 years) with 16 females and four males. Unlike the analytical data, the human osmolarity data from both devices were not normally distributed (P<0.01). The distribution of osmolarity measurements using TearLab was heteroscedastic with a long hyperosmolar tail, and a median value of 294 mOsm/L (Figure 1). About 90.9% of all TearLab measurements were in the normal range of ≤308 mOsm/L. One of 200 (0.51%) TearLab measurements was >340 mOsm/L, indicating moderate dry eye disease.

Figure 1 Distribution of osmolarity in non-dry eye subjects with TearLab Osmolarity System.

The i-Pen osmolarity data were more uniformly distributed across the entire range of normal and abnormal human osmolarity (Figure 2), and thus were unable to differentiate this cohort of normal subjects from data frequently observed in moderate-to-severe dry eye patients.²⁴ The i-Pen reported a median osmolarity value of 321 mOsm/L for the normal subjects. About 37.5% of all i-Pen measurements were in the normal range ≤308 mOsm/L, whereas 21% of the i-Pen values were >340 mOsm/L. When the measurements were grouped by subject, TearLab accurately identified 100% of subjects with no dry eye disease versus i-Pen, which accurately identified only 15% of subjects with no dry eye disease. An example of data grouped by subject is shown in Table 1. When using other published osmolarity cut-off values of >312²⁶ and >316 mOsm/L,²⁷ the i-Pen reported only 35% and 40% of these subjects to have no dry eye disease (Table 2).

Figure 2 Distribution of osmolarity in non-dry eye subjects with i-Pen Osmometer.

Table 1 Representative raw data (mOsm/L) from the TearLab Osmolarity System and i-Pen Osmometers on normal healthy patients

Test	Subject 1				Subject 2
	TearLab		i-Pen		TearLab		i-Pen
	OD	OS	OD	OS	OD	OS	OD	OS
1	302	299	339	360	291	303	324	292
2	294	296	320	342	292	294	335	297
3	290	299	335	297	297	295	310	323
4	292	295	312	311	293	298	315	345
5	297	298	330	333	292	295	291	301
Average		296.2		327.9		295.0		313.3
Standard deviation		3.6		18.2		3.6		18.5

Table 2 Diagnosis of dry eye disease using TearLab Osmolarity System and i-Pen Osmometers based on diagnostic cutoff values in peer review literature

Osmolarity cutoff value (mOsm/L)	TearLab, n=20 (%)	i-Pen, n=20 (%)
>300	4 (20)	19 (95)
>308	0	17 (85)
>312	0	13 (65)
>316	0	12 (60)

As an estimate of precision, a box and whiskers plot (Figure 3) shows the range of TearLab measurements from the first quartile (Q1=289 mOsm/L) to third quartile (Q3=299 mOsm/L) was 10 mOsm/L. The range of the i-Pen first quartile (Q1=298 mOsm/L) to third quartile (Q3=338 mOsm/L) was 40 mOsm/L, suggesting that the imprecision of i-Pen was roughly four times greater than that of the TearLab. To assess precision independent of baseline differences, the data were also analyzed per subject (Figures 4 and 5). The TearLab reported a significantly lower mean and average intra-subject variation (295.3±8.6 mOsm/L) when compared with the i-Pen (319.4±20.3 mOsm/L, P<0.001). No signifi-cant difference was measured between the average left and right eye measurements for both devices (P>0.05).

Figure 3 Box and whiskers plot comparing the TearLab Osmolarity System to the i-Pen Osmometer.

Figure 4 Tear osmolarity reported per subject as measured by the TearLab Osmolarity System (mean ± 1 SD).

Note: The dashed line represents a diagnostic cutoff to delineate mild from normal dry eye subjects.¹⁰
Abbreviation: SD, standard deviation.

Figure 5 Tear osmolarity reported per subject as measured by the i-Pen Osmometer (mean ± 1 SD).

Note: The dashed line represents a diagnostic cutoff to delineate mild from normal dry eye subjects.¹⁰
Abbreviation: SD, standard deviation.

Discussion

Our study results demonstrate a significant difference in two point-of-care osmometers with the TearLab providing accurate identification of normal tear films while the i-Pen did not. The TearLab results are congruent with previous literature that reported specificities of 90.0%–92.0% for TearLab.^10,15 Historical meta-analyses of tear osmolarity using laboratory-based freezing point osmometers report specificities of upward of 94% for the marker as a whole.¹⁵ In normal subjects with a healthy lacrimal system, repeated testing has been shown to have little to no effect on the measured tear osmolarity.^12,28 Insofar that the recruited subjects are truly devoid of dry eye disease, an accurate osmometer is expected to report the majority of test results below the established cutoff for dry eye disease. As such, the TearLab was able to reproduce the expected distribution of normal patients, with a few rare “outliers,” a phenomenon that has also been noted in other studies on tear osmolarity.²⁹

“Outliers” tend to appear in subjects with undetected mild or early stage dry eye. A case in point is subject #5, who displayed an outlier >340 mOsm/L on the TearLab. This subject stands out in Figure 4 as being elevated and unstable when compared with the rest of the cohort. In the first four pairs of osmolarity tests on subject #5 with the TearLab, each pair had ≥10 mOsm/L inter-eye difference, indicating a loss of homeostasis of the tear film and an early sign of dry eye,¹⁰ as well as a reading of 337 mOsm/L in the second pair of tests. Taken together, these data are strongly indicative of early stage dry eye that was otherwise undetectable with standard clinical tests such as fluorescein staining and symptom assessment. Szczesna-Iskander et al²⁹ and Bunya et al,³⁰ two studies which also report “outliers” using the TearLab, recruited cohorts that included mild dry eye subjects. Nine of 19 subjects in the study by Szczesna-Iskander et al reported an OSDI ≥7.5, with a range as high as 54.2, whereas Bunya noted that,

It is also possible that some of our control participants had undiagnosed asymptomatic dry eye … some control participants had minimal positive examination findings.³⁰

This, of course, is one of the benefits of measuring tear osmolarity, in that it is frequently more sensitive than clinical observation and other available tests of ocular surface inflammation.^10,26 As there was no evidence of analytical outliers from the TearLab in this study, we can only surmise that the elevated values measured on the TearLab both in this study and others are an essential aspect of tear osmolarity in early stage or mild dry eye. In other words, rather than being outliers, these data points are important signals that indicate an unstable ocular surface before traditional clinical signs have revealed the disease. This also means that, even if a perfectly accurate and precise osmometer existed, the specificity of tear osmolarity would be limited to the 90% range that is reported elsewhere due to the inherent nature of the marker.^10,29,30

By contrast, the i-Pen was unable to reproduce the expected range of normal values, reporting random values across the clinically relevant range of tear osmolarity. As evidenced in Figure 2, the i-Pen reported an equivalent chance of measuring 345 mOsm/L as it did 295 mOsm/L in normal eyes. In this context, the specificity of the i-Pen would not be consistent with historical meta-analyses of osmolarity as a biomarker of dry eye, as determined by other non-impedance-based methods of osmometry.¹⁵ The i-Pen identified the majority of the healthy, non-dry eye cohort in this study as having moderate-to-severe dry eye disease.

The results of this study reinforce a recent laboratory analysis of the TearLab and i-Pen.¹⁹ The data published by Rocha et al reported an average analytical CV of 1.6% for TearLab, which is essentially equivalent to the 1.3% CV for TearLab found in this study. Similarly, Rocha et al reported an average 6.3% CV for i-Pen when measuring contrived tears; a 3.94-fold increase in variation over TearLab. Our results mimic this, showing a fourfold increase of clinical interquartile range of the i-Pen compared to the TearLab. The data herein validate the methods of Rocha et al, and the findings support their conclusion that i-Pen has insufficient precision to delineate osmolarity levels in the physiological range, or to meaningfully distinguish normal from dry eye patients. Rocha et al also found that the i-Pen produced random values across the full range of human tear osmolarity on laboratory samples, similar to the results of this study.

In attempting to explain the imprecision of the i-Pen, we identified several potential sources of error in the design of the device: the unaccounted influence of temperature variation on impedance while the i-Pen is placed atop the lid, and the user-dependent introduction of motion artifacts in the polarizable electrodes. Although one may assume that the palpebral conjunctiva is at a constant temperature between individuals, studies on temperature variations in the conjunctiva report ranges in the order of 31°C–37°C.^31,32 Impedance measurements are strongly affected by the temperature of the sample.^33,34 For every degree temperature change, the measurement of impedance changes ~2%,³⁴ which suggests that normal temperature variation of the lid can contribute as much as a 36 mOsm/L error in osmolarity measurement in the i-Pen if not properly compensated. Second, it is well known in the biomedical engineering literature that movement artifacts are potent noise sources in polarizable electrodes at a wet tissue interface.³⁵ Motion artifacts are one reason electrocardiogram electrodes require an adhesive film saturated with electrolyte to function properly. As users of the i-Pen are actively holding the device to the lid during measurement, small variations over time from blinks or shaking of the hand will almost certainly contribute to the imprecision of the i-Pen observed in this study. Although this may be overcome with significant skill of the operator, it does call into question whether one could expect acceptable performance from such a device without extensive training if this was indeed a major source of variation.

In this randomized comparative study of two point-of-care tear fluid osmometers among otherwise healthy, normal subjects without dry eye disease, the TearLab Osmolarity System demonstrated accuracy, precision, and agreement in line with the manufacturer claims and prior literature. The I-Med i-Pen produced random values across the physiological range of tear osmolarity, and lacked sufficient performance to delineate subjects with and without dry eye disease in the clinical setting.

Acknowledgments

The study was sponsored by TearLab Corporation.

Disclosure

Dr Nolfi is a consultant to TearLab. Dr Caffery has no competing financial interest in the study material.