An evaluation of two novel capillaroscopy techniques in suspected scleroderma-spectrum disorders: A single-centre cross-sectional study

Abstract

Objectives: The primary aim was to evaluate the diagnostic performance of digital photographs taken with a smartphone camera using both a lens attachment and, separately, a dermatoscope. The secondary aims were to assess the influence of prior capillaroscopy experience and familiarity with the novel techniques on diagnostic accuracy.

Methods: All patients referred for capillaroscopy between May 2016 and January 2017 were eligible for inclusion. Nailfolds were classified by widefield microscopy before proceeding double-blinded to nailfold photography using both novel techniques. Randomised photographs were assessed by three independent investigators and results were compared to widefield microscopy. Sensitivity, specificity, inter- and intra-observer variability were calculated.

Results: Sixty-five participants contributed over 1000 digital photographs for assessment. The ‘smartphone-lens’ technique performed with moderate sensitivity (65%; 58–72) and high specificity (90%; 84–96). The ‘smartphone-dermatoscope’ technique performed with higher sensitivity (74%; 66–82) and excellent specificity (95%; 88–100) and was used more accurately by a novice. Prior assessor experience with nailfold capillaroscopy in general and prior experience with the novel techniques positively modulated the diagnostic accuracy.

Conclusion: New technologies, in this case utilising a smartphone camera, could help to improve accessibility to nailfold capillaroscopy, an important diagnostic tool and putative biomarker in scleroderma-spectrum disorders, whilst retaining accurate results.

Keywords:

• Microscopic angiography
• nailfold capillaroscopy
• photography
• smartphone
• systemic sclerosis

Introduction

Nailfold capillaroscopy is a well-established, clinically valuable technique for visualising the microcirculation in patients with suspected scleroderma-spectrum disorders. First described in this clinical context in 1926, great progress was made in the 1970s and 1980s helping to further characterise the range of abnormalities present and establishing the technique of widefield microscopy, paving the way for an ever burgeoning field [1–3]. Recently, there has been particular focus on accurately defining nailfold abnormalities, on developing scoring and classification systems to aid diagnostic accuracy and reproducibility, and on establishing the implications for prognosis and risk of organ specific complications [4–6].

The two techniques currently most employed and validated for nailfold capillaroscopy are widefield microscopy and nailfold videocapillaroscopy. Each requires relatively expensive (typically well in excess of $5000USD) non-portable equipment and, consequently, access to them is largely restricted to centres with a special interest in the area. This current state of affairs sits somewhat at odds with the inclusion of nailfold capillaroscopy in the most recent consensus diagnostic guidelines for systemic sclerosis that recognise its utility in identifying early systemic sclerosis in particular which is likely to become increasingly important as therapeutics intended to slow disease progression are investigated [7–9]. It is important to consider options to maximise access to the wider rheumatology community to this highly specific, non-invasive investigation.

A recent estimate is that 68% of adults in countries with advanced economies and 37% in developing economies own a ‘smartphone’, and this figure is rising steadily over time [10]. As smartphones typically incorporate a digital camera and this technology is readily available to clinicians, even at the bedside, we sought to evaluate the performance of a novel combination of smartphone digital camera with each of two relatively inexpensive and portable devices. There are a number of commercially produced camera lens attachments for the iPhone6™ (Apple Inc., Cupertino, CA) and we identified the Macro Pro Lens™ (Olloclip^®, San Francisco, CA) as an attractive option to evaluate further. Dermatoscopes have been used in clinical practice for many years to visualise nailfold capillaries and recently have been shown to perform well when compared to alternative capillaroscopy techniques [11]. We decided to evaluate the combination of the DermliteDL4™ (3Gen Inc., San Juan Capistrano, CA), again with the iPhone6™ digital camera. The Macro Pro Lens™ cost $125USD and the DermliteDL4™ $1395USD and both are handheld devices which would allow for bedside or clinic room examinations. The magnification achieved with both devices is around 20× which, combined with the smartphone digital zoom, allows for evaluation of capillaries at higher magnification (maximum 80×). Digital photographs can then be examined at even higher magnification, albeit without improved resolution, using standard photography software.

The primary study aim was to evaluate the diagnostic performance of digital photographs taken with both Macro Pro Lens™ and DermliteDL4™ combined with a smartphone (iPhone6™) camera, using widefield microscopy as a reference standard. The secondary aims were to assess the influence of prior capillaroscopy experience and familiarity with the novel devices on diagnostic accuracy.

Materials and methods

This was a single-centre cross-sectional study performed in accordance with Good Clinical Practice. Ethical approval for this study was granted by the Sydney Local Health District Ethics board and all patients provided informed written consent according to the Declaration of Helsinki. Our institution has a long established nailfold capillaroscopy clinic and receives referrals from a large mainly metropolitan area (Sydney, Australia) for a diagnostic examination. All patients referred to the clinic between May 2016 and January 2017 were approached for inclusion in the study and patients were excluded if our reference standard, widefield microscopy, was unable to be performed because of, for example, severe contractures or distal digit amputation.

Reference standard procedure

Participants were first assessed with widefield microscopy by an expert assessor (NM) blinded to any clinical information. A drop of paraffin oil applied to the nailfold of each finger is used in all techniques to help to reduce surface reflection and better visualise the capillaries. The index, middle, ring and little finger nailfolds of each hand were visualised before classifying the overall impression as normal, non-specifically abnormal or the scleroderma pattern (‘SD-pattern’) in keeping with the classification of Maricq et al. [3]. Both the patient and the other investigators were blinded to this initial result until the result was communicated to the referring physician once all examinations were concluded.

Image acquisition

Participants then proceeded to a separate clinic room with a co-investigator (MP or MO) for assessment with the two novel techniques. A single representative digital photograph was taken for each of the eight nailfolds previously examined by widefield microscopy using each technique (16 photographs per patient). The combination of the Macro Pro Lens™ and iPhone6™ will be referred to henceforth as the ‘smartphone-lens’ technique and the combination of DermliteDL4™ and iPhone6™ referred to as the ‘smartphone-dermatoscope’ technique. The images were de-identified, transferred onto a study computer and later randomised for assessment. MO had no prior experience with either novel technique to simulate a physician using the devices for the first time in clinical practice.

Image classification

After enrolment concluded, all assessors undertook a 30-min computer-based tutorial created in-house to calibrate them to the classification criteria. Each assessor, henceforth referred to as novice (MO), intermediate (MP) and expert (NM), classified each patient photograph folder as normal, non-specifically abnormal, SD-pattern or unclassifiable (if the images were not of adequate quality to discriminate). As nailfold abnormalities are known to sometimes vary considerably between fingers multiple images allows the assessor to build an overall impression. The presence of certain specific abnormal findings (‘giant capillaries’ or ‘avascular areas’) in a single nailfold is sufficient to classify the folder as SD-pattern. Only folders classified as SD-pattern were considered diagnostically abnormal in the subsequent statistical analysis, as is the interpretation applied in our clinical practice using widefield microscopy. All assessors repeated classification of a re-randomised series of the patient folders 3 months later to provide a measure of intra-observer variability.

Analysis

IBM SPSS Statisitics^® ver. 22 (IBM, Armonk, NY) was used for analysis. Sensitivity and specificity were calculated for each technique using widefield microscopy as the reference standard, presented with 95% confidence intervals. Cohen’s kappa coefficient was calculated to assess the inter- and intra-observer reliability; a value of >0.41 is considered moderate agreement, >0.61 substantial and >0.81 almost perfect [12]. Pearson Chi-squared test was used to assess the significance of previous photography experience.

Results

All sixty-five patients attending the clinic during the enrolment period consented and none was excluded. Fifty (76.9%) were female and the mean age was 52.3 years (range 22–79). One thousand and forty individual photographs were later assessed by each of the three assessors. Figure 1 shows an example of the images obtained using each technique.

Figure 1. Digital photographs of an identical nailfold taken with each novel technique in two patients. (a, b) Normal appearance; (c, d) SD-pattern (abnormal showing multiple giant capillaries and avascular areas).

Table 1 shows the performance of the two techniques calculated for each assessor individually to investigate the potential modulating effect of prior nailfold assessment experience.

Table 1. The diagnostic performance of the smartphone-lens and smartphone-dermatoscope techniques stratified by assessor’s prior experience of nailfold capillaroscopy.

		True positive (n)	True negative (n)	False positive (n)	False negative (n)	Sensitivity (95% CI)	Specificity (95% CI)
Expert	Smartphone-lens	15	38	4	8	65% (58–72)	90% (84–96)
	Smartphone-dermatoscope	17	40	2	6	74% (66–82)	95% (88–100)
Intermediate	Smartphone-lens	15	41	1	8	65% (58–72)	98% (91–100)
	Smartphone-dermatoscope	17	41	1	6	74% (66–82)	98% (91–100)
Novice	Smartphone-lens	12	36	6	11	52% (45–59)	86% (80–92)
	Smartphone-dermatoscope	16	38	4	7	70% (62–78)	90% (84–96)

Twenty percent of photographs were taken by a co-investigator investigator with no prior experience using the novel devices (MO) and a significant increased proportion of these photographs were assessed as unclassifiable. Fifty-four percent of the photographs taken by MO using the ‘smartphone-lens’ technique were assessed as unclassifiable compared to 10% (p = .000) when taken by the investigator with more prior experience (MP). For the ‘smartphone-dermatoscope’ technique there was no statistically significant difference; 8% of photographs taken by MO were deemed unclassifiable compared to 5% when taken by MP (p = .385), respectively.

A measure of inter-observer variability, Cohen’s Kappa coefficient was 0.805 between expert and intermediate, 0.682 between intermediate and novice and 0.509 between expert and novice for the ‘smartphone-lens’. For the smartphone-dermatoscope technique the kappa values were 0.736, 0.703 and 0.670, respectively.

Intra-observer correlation for the ‘smartphone-lens’ was 1.00 for the expert assessor, 0.91 for the intermediate and 0.83 for novice. For the ‘smartphone-dermatoscope’ the calculations were 0.86, 1.00 and 0.81, respectively.

An unclassifiable assessment was made for 19.0% of images taken with the ‘smartphone-lens technique’ and 5.6% of those taken with the ‘smartphone-dermatoscope’ technique. A post-hoc analysis of these cases was performed. 78.4%, 10.8% and 10.8% of unclassifiable cases with the ‘smartphone-lens’ technique were originally classified by widefield microscopy as normal, non-specifically abnormal or SD-pattern respectively. Although smaller numbers limit comparisons with the ‘smartphone-dermatoscope’ technique, 72.7%, 0% and 27.3% were originally classified by widefield microscopy as normal, non-specifically abnormal and SD-pattern, respectively. This is summarised in Figure 2.

Figure 2. A post hoc analysis of the unclassifiable images combined for all assessors stratified by technique.

A post hoc analysis was also undertaken in to the features of false-positive and false-negative cases. In all but one of the false-positive assessments (94.4%) by both techniques the original finding at widefield microscopy was non-specific abnormalities. Additionally, all false-positives reported for the ‘smartphone-dermatoscope technique’ were also found by the same assessor as false-positive for the ‘smartphone-lens technique’, suggesting a systematic difference in interpretation. Reviewing the relevant images again they all show a mixture of capillary enlargement and microhaemorrhages and are none appear floridly SD-pattern with, for example, multiple giant capillaries or avascular areas.

Similarly, regarding the false negative cases, in 87.1% the assessor(s) recognised abnormalities but felt these were non-specific rather than SD-pattern. In the remaining cases, all three assessors agreed in classifying each individual case as normal. Reviewing the relevant images, three of the participants had darkly pigmented skin which may have affected the techniques and the majority of the other cases appeared to have a single giant capillary which may have been missed or not attached appropriate significance.

Discussion

We are not aware of any previous study combining a smartphone digital camera with other technologies to study nailfold capillaries. Both novel techniques performed adequately in identifying patients with the diagnostically abnormal nailfold capillary changes linked to scleroderma-spectrum disease, retaining specificity which is important to minimise concern arising from false positive results. The ‘smartphone-dermatoscope’ technique performed better than the ‘smartphone-lens technique’. Both achieved a similar correlation with the results from widefield microscopy as have reports in the literature of the comparison of videocapillaroscopy versus dermoscopy, videocapillaroscopy versus widefield microscopy and a recent report of the reproducibility of videocapillaroscopy [11,13,14]. The reported variability in all techniques supports the need for more accurate definitions of capillary abnormalities and classification criteria, and demonstrates the lack of a true gold standard.

These data suggest that there are factors important to diagnostic performance involving both image acquisition and image interpretation. A major factor we identified for the difference in performance between the techniques was the large proportion of photographs taken with the ‘smartphone-lens’ technique by the co-investigator with no prior experience that were unclassifiable. In contrast, even in inexperienced hands, photographs taken using the ‘smartphone-dermatoscope’ technique were almost always deemed classifiable. The postulated reasons for this include the integrated polarised light source in the DermliteDL4™, improving illumination for photography, as well as a familiarity with dermatoscopes from other clinical practice.

The presence of unclassifiable cases affects the utility of the techniques but also may have an impact on accuracy if certain groups are systematically found to be unclassifiable. Our post hoc analyses suggest that the majority of these cases were found to be normal at widefield microscopy and were therefore appropriately included in the non-SD-pattern results. However, a proportion of unclassifiable images were from patients who had diagnostically abnormal nailfolds (2.1% of photographs from the ‘smartphone-lens’ and 1.5% from the ‘smartphone-dermatoscope’ techniques). In these cases potential clinically important information has been missed.

In our study, no patients were deemed unclassifiable by widefield microscopy as this was a pre-requisite exclusion criteria. However, the occurrence of unclassifiable images is reported in the literature for all other techniques such as nailfold videocapillaroscopy, used increasingly as a gold standard investigation. In a recent study by Dinsdale et al. [14], a total of 3401 photographs of healthy controls, patients with primary Raynaud’s phenomenon and patients with scleroderma were shown to ten capillaroscopy experts. 18.5%, 20.0% and 15.0% of images from each group, respectively, were of an ‘ungradable quality’. In another study by Hughes et al. using images obtained via nailfold videocapillaroscopy and dermoscopy, 14% and 30%, respectively, of images were deemed not classifiable by the 48 mixed specialist and general rheumatologist assessors [11]. When put in context of existing gold-standard performance our unclassifiable rates for the smartphone-lens of 19.0% and 5.6% for the smartphone-dermatoscope techniques we believe are acceptable albeit not ideal.

A post hoc analysis of the false-positive and false-negative cases reveals that in almost every such occurrence these were the result of the assessor(s) choosing the adjacent categorical group (i.e. choosing SD-pattern when widefield microscopy found non-specific abnormalities or vice versa). Such variation is to some extent inherent when applying categorical classification to effectively continuously variable abnormalities, borne-out in the intra-observer variability.

The results support a modulation of the diagnostic performance by prior capillaroscopy experience with the same photographs being classified more accurately by the expert assessor than the novice. The gradient of intra-observer variability between experience levels suggests the more experience a clinician obtains, the more reproducible their assessments will likely become. Our results demonstrate, however, that a novice with very brief capillaroscopy education and no prior experience with the devices can complete a diagnostic examination accurately, important when considering how to expand access to capillaroscopy more broadly.

An advantage of digital photography over real-time examination is the photographs themselves. If there is diagnostic uncertainty, the images could be shared with an expert colleague for their opinion, helping to improve the accuracy of the study and educate the original assessor. If uncertainty remains, the patient can be referred for an alternative procedure, although all studied capillaroscopy techniques result in images which are unclassifiable [15]. Photographs allow for a direct comparison of serial examinations which may become increasingly desirable as evidence emerges of nailfold appearances changing in response to therapy and over time [15,16]. Another potential advantage is our observation during the study that patients engage in these techniques, perhaps because of the use of technology that is largely familiar to them. The process of a detailed examination and resultant photographs could be used as a powerful adjunct to a considered discussion to, for example, motivate a patient to employ protective measures to prevent digital ischaemia or reassure another patient with a normal examination. The ethical implications of a photographic record outside of a clinical trial should be considered carefully, however.

The novel technology offers considerable potential cost saving, even when factoring in the cost of a smartphone. Both are small, robust and lend themselves to transport between clinic rooms and hospital wards if necessary. They are both quick to use; each examination taking under 5 min and typically only 2–3 min per patient.

The strengths of this study include its novelty and clinical focus. We attempted to evaluate the techniques in a ‘real-world’ setting, using patients referred to our existing clinical service and a range of prior experience to simulate the range of experience in the rheumatology community. The data are reproducible, simple to interpret and consider in a broader perspective. However, there are some significant limitations. This was a relatively small cohort of patients from a single-centre. The study did not record clinical information (such as autoantibody positivity or subsequent diagnostic classification) which would enrich an assessment of its clinical significance and is an area for potential further work. As referred to above there is no gold standard investigation with which to compare the new techniques against but we hope that this preliminary work helps to encourage others to look at how to harness novel technologies to improve accessibility and accuracy of nailfold capillaroscopy in the broader rheumatology setting.

In summary, the ‘smartphone-dermatoscope’ technique performed marginally better than the ‘smartphone-lens’ technique in the identification of abnormal nailfold capillaries in patients with suspected scleroderma-spectrum disorders. Both novel techniques offer the potential benefits of cheaper and more portable equipment than existing techniques and performed adequately even in the hands of a novice.

Conflict of interest

None.

Acknowledgements

The authors would like to thank Mrs. Dorothy Fowler CNC for her assistance in the conduct of the study. MP is supported by the Leanne Stafford Scholarship provided by the Australasian Rheumatology Association and Arthritis Australia.