An Adjusted Error Score Calculation for the Farnsworth-Munsell 100 Hue Test

ABSTRACT

The Farnsworth-Munsell 100 Hue Test, a test that measures an individual’s hue discrimination ability, operates with the fundamental assumption that it is administered using a fixed, standard illuminant. This assumption is violated when the testing illuminant is changed—as is common when testing color discrimination ability of an illuminant—which likely causes a reordering of the caps in the test. To ensure that a participant is not falsely penalized for correctly responding to a hue transposition caused by the new testing light source, an adjusted error score is proposed that reconciles light source–induced hue transpositions and participant performance on the test.

KEYWORDS:

• Adjusted total error score
• color discrimination
• Farnsworth-Munsell 100 Hue Test
• total light source error score
• transposition

1. Introduction

The Farnsworth-Munsell 100 Hue Test (FM-100) is a physical test of hue discrimination, consisting of 85 colored caps whose chromaticities are distributed around the hue circle. It contains four separate trays of caps—tray A is nominally red to red–orange, B is yellow to yellow–green, C is green to green–blue, and D is indigo to indigo–magenta—that are moveable between two fixed, colored end caps (Fig. 1). The test is used to evaluate the hue discrimination ability of an individual, to classify that individual into superior, average, or low discrimination ability, and to identify individual color anomaly (Farnsworth 1957). The FM-100 hue test is largely used as a research tool to understand normal color vision (Kinnear and Sahraie 2002; Malone and Hannay 1977) and to detect and study its deficiencies (Farnsworth 1943; Moreland et al. 2014; Verriest 1963, 1974; Vingrys et al. 1992). It is also used as a clinical screening test of color vision for professionals in color-critical applications such as manufacturing (paints, dyes, et cetera) and health care.

Fig. 1. (Top) The distribution of caps of the FM-100 hue test in the a′b′ plane of CAM02-UCS (using the CIE 1964 10° standard observer) illuminated by CIE standard illuminant C. The caps have an average lightness correlate (J′) of 60.15 and an average colorfulness correlate (M′) of 19.73 (Luo et al. 2006). The dashed circle shows an equal colorfulness of 19.73 across all hues. Number labels indicate cap number. (Bottom) Top view of the four physical test trays of the FM-100 test. Tray A has 22 moveable caps between two fixed end caps; trays B, C, and D have 21.

Concerning a light source, color discrimination describes the ability of a light source to allow observers to distinguish between colors of slightly different hue when viewed simultaneously (Thornton 1972). To quantify this ability, past researchers have used an experimental design that has participants complete the FM-100 hue test under systematically varied light spectra (Boyce and Simons 1977; Esposito 2016; Esposito and Houser 2017; Mahler et al. 2009; Rea and Freyssinier-Nova 2008; Royer et al. 2012; Wei and Houser 2012). The ultimate goal is to link light source colorimetric performance parameters—such as color fidelity (CIE 1995; Davis and Ohno 2010; IES 2015; Smet et al. 2015) and gamut area (Davis and Ohno 2010; IES 2015; Rea and Freyssinier-Nova 2008; Thornton 1972)—to FM-100 error scores, to predict color discrimination ability of light sources. Absolute thresholds for classifying a light source’s color discrimination ability have yet to be determined. Esposito and Houser (2017) offered a preliminary proposal using a measure of light source–induced cap transpositions called the Total Light Source Error Score, R_d (described in Section 3).

Though varying the light source is necessary for determining a causal link between light source performance parameters and light source color discrimination, doing so violates the intrinsic requirement that the FM-100 hue test be administered with a fixed testing source. In 1957, Farnsworth stated: “reliable results cannot be expected from this test unless standard illumination is used.” That standard illumination is CIE Standard Illuminant C (or daylight), because it retains an optimal spacing of the caps in chromaticity space (Farnsworth 1943), which also preserves their numerical order.

Changing the testing light source will alter the hue, chroma, and lightness of the test chips, but it may also swap colored caps or significantly reorder them altogether (Esposito 2016; Esposito and Houser 2017; Moreland et al. 2014). When the test is administered using a light source that causes such transpositions of caps, errors will be miscalculated and the experimental results distorted.

To right this discrepancy, an adjusted Total Error Score is detailed that accounts for the interaction between the light source spectral power distribution (SPD) and the spectral reflectance distribution (SRD) of the FM-100 hue test chips.

2. Standard total error score

The score for any individual cap of the FM-100 test “is the sum of the [absolute] difference between the number of that cap and the numbers of the caps adjacent to it” minus 2 (Farnsworth 1957). For example, the arrangement 29–30–31–32 has an error score of zero, and 29–31–30–32 (one transposition) has an error score of 4 (Appendix, Scenario 1). Two transpositions correspond to an error score of 8, three to an error of 12, four to an error of 16, and so on. The Total Error Score (TES) is computed as the sum of the error scores for each of the four separate trays of the test:

(1) $TotalErrorScore\left(TES\right)=\sum _{i=1}^{4}iES=\sum _{i=1}^4\left(\left(\sum _{j=1}^{n+2}C{E}_j\right)-\left(\left(n+2\right)\ast 2\right)\right)$(1)

$C{E}_j=\left|C_j-C_{j-1}\right|+\left|C_j-C_{j+1}\right|,$

where

i is a counter for the four trays (i = 1 is A, 2 is B, 3 is C, and 4 is D);

C_j is the cap number of the jth cap ¹;

CE_j is the cap error of the jth cap ²;

n is the number of moveable caps in the tray corresponding to i (n = 22 for tray A and 21 for trays B, C, and D);

AES is the standard error score for tray A;

BES is the standard error score for tray B;

CES is the standard error score for tray C;

DES is the standard error score for tray D.

The calculation software that accompanies the physical FM-100 hue test uses (1) and is only applicable when the testing light source preserves the numerical order of caps.

3. An adjusted total error score

The adjusted Total Error Score (TES_adj) (Esposito and Houser 2017) is a modified Farnsworth-type error score that compares the order of the FM-100 hue test chips illuminated by the testing source—calculated based on hue angle in the a′b′ plane of the CAM02-UCS (Fairchild 2013; Luo et al. 2006)—to a participant’s order of caps. TES_adj guarantees that a participant is not penalized for correctly responding to a hue transposition caused by the light source or mistakenly rewarded because the analysis does not consider the light source’s impact on cap order. TES_adj reconciles the discrepancy between a light source–induced transposition and a transposition arranged by a participant performing the test:

(2) $TE{S}_{\mathrm{a}\mathrm{d}\mathrm{j}}=\sum _{i=1}^{4}iE{S}_{\mathrm{a}\mathrm{d}\mathrm{j}}=\sum _{i=1}^4\left(\left(\sum _{j=1}^{n+2}P{E}_j\right)-\left(\left(n+2\right)\ast 2\right)\right)$(2)

$P{E}_j=\left|P_j-P_{j-1}\right|+\left|P_j-P_{j+1}\right|,$

where

i is a counter for the four trays (i = 1 is A, 2 is B, 3 is C, and 4 is D);

P_j is the place number of the jth cap (demonstrated in Appendix, Scenario 2);

PE_j is the place error of the jth cap;

n is the number of moveable caps in the tray corresponding to i (n = 22 for tray A and 21 for trays B, C, and D);

AES_adj is the adjusted error score for tray A;

BES_adj is the adjusted error score for tray B;

CES_adj is the adjusted error score for tray C;

DES_adj is the adjusted error score for tray D.

The principal difference between (1) and (2) is the place number, which is an n × 1 array that is determined by cross-referencing the numerical cap order with their correct place in the array of caps illuminated by the testing illuminant.

As an example, if a light source causes the order of caps 29–31–30–32 and a participant arranges these caps accordingly in tray B of the test, he or she would not be attributed an error (as opposed to an error score of 4 using (1)). In this example, cap number 31 is the ninth cap in the correct order (P_j = 9) and cap number 30 is the 10th cap in the correct order (P_j = 10). When the light source causes no cap transpositions, place number is equal to cap number (P_j = C_j), the adjusted Total Error Score is equal to the standard error score (TES_adj = TES), and (2) collapses to (1).

4. Light source error score

Applying (1) directly to the cap order under the testing illuminant produces an objective measure of light source–induced cap transpositions, called the Total Light Source Error Score, R_d (Esposito 2016; Esposito and Houser 2017); Esposito and Houser (2017) proposed R_d as a measure of color discrimination. R_d is a source-specific error score and is the sum of the source-specific error scores for each of the four trays of the FM-100 hue test:

(3) $R_{\mathrm{d}}=\sum _{i=1}^4{R}_{\mathrm{d},i}=\sum _{i=1}^4\left(\left(\sum _{j=1}^{n+2}CE{t}_j\right)-\left(\left(n+2\right)\ast 2\right)\right)$(3)

$CE{t}_j=\left|C{t}_j-C{t}_{j-1}\right|+\left|C{t}_j-C{t}_{j+1}\right|,$

where

i is a counter for the four trays (i = 1 is A, 2 is B, 3 is C, and 4 is D);

Ct_j is the cap number of the jth cap, as ordered by the testing light source;

CEt_j is the cap error of the jth cap, as ordered by the testing light source;

n is the number of moveable caps in the tray corresponding to i (n = 22 for tray A and 21 for trays B, C, and D);

R_d is the total light source error score;

R_d,_A is the light source error score for tray A;

R_d,_B is the light source error score for tray B;

R_d,_C is the light source error score for tray C;

R_d,_D is the light source error score for tray D.

An R_d of 4 indicates that the light source causes one transposition, a value of 8 is two transpositions, 12 is three transpositions, and so on. Table 1 shows R_d values for select light sources.

Table 1. R_d for select light sources from the library of the IES TM-30-15 calculator. Multiple spectra of the same source type sometimes exhibited different R_d values. For example, there were multiple phosphor-based LED sources in the library that collectively exhibited all R_d values between 0 and 28.

Source type	Source name	Rd
Incandescent	75WA19 neodymium	12
Incandescent	Halogen/halogen MR16	4, 8
Incandescent	Filtered halogen	0
HID	HPS standard	40, 48
HID	HPS deluxe	36
HID	Super HPS	40
HID	Mercury	36, 44, 52
HID	CDM940, MHC100UMP4K–metal halide	0
HID	CDM830, MHC100/U/MP/3K	4, 8
LED	Mixed (experimental)	0–40
LED	Mixed (commercial)	36
LED	Hybrid (commercial)	0–16
LED	Phosphor	0–28
Fluorescent	Narrowband–F32T8/7XX	8, 12, 24
Fluorescent	Narrowband–F32T8/8XX	0–16, 24
Fluorescent	Narrowband–F32T8/9XX	0, 8
Fluorescent	Narrowband–F40T12/XXU	8, 12, 24
Fluorescent	Broadband	0, 4, 8, 20
Fluorescent	CIE C	0
Fluorescent	CIE F1–F12	0, 8, 12, 20
Model	CIE D-Series (5000–8000 K)	0
Model	Equal energy	0

The R_d calculation compares the order of caps illuminated by the light source to the numerical order of caps that occurs, by design, under CIE standard illuminant C. CIE C, then, becomes the de facto reference.

5. Discussion

Table 1 shows that many common light sources, as well as many experimental spectra, transpose at least one cap of the FM-100 hue test. A recent study by Esposito and Houser (2017) evaluated the color discrimination ability of 24 experimental light emitting diode (LED) spectra, 17 of which transposed at least one cap of the FM-100 hue test (R_d ≥ 4). On average, the difference between TES and TES_adj for those 17 sources was 7.6 (approximately two transpositions) and as large as 40. Figure 2 shows a comparison of TES and TES_adj for their spectra; the comparison suggests that if the order of caps illuminated by the testing source is not considered, results will be significantly distorted. This may lead to erroneous conclusions about the color discrimination ability of a light source.

Fig. 2. (Top left) A generalized comparison of standard error score and adjusted error score. Point 1 indicates that the standard error score equals the adjusted error score (that is, ES = ES_adj), which only occurs when a light source causes no cap transpositions (R_d = 0). Equation (1) assumes that the light source causes no cap transpositions and that all resulting participant error scores fall on the c = 1 line. For point 2, the adjusted error score is higher than a standard error score of zero (that is, ES_adj > ES = 0), which occurs when a light source causes many transpositions and the participant orders his or her caps in exact numerical order; in this scenario, many errors would be uncounted using the standard error score calculation. For point 3, the standard error score is higher than an adjusted error score of zero (that is, ES > ES_adj = 0), which occurs when the light source causes several transpositions and the participant exactly responds. In this scenario, errors are misapplied to the participant using the standard error score calculation because the correct order of caps is not the numerical order. (Top right/bottom left) Trays B and C error score comparison, respectively, for 480 responses (24 spectra × 20 participants) from Esposito and Houser (2017). (Bottom right) Total error score comparison for the same study. Seventeen of their 24 experimental spectra caused at least one hue transposition, which resulted in a large overall discrepancy between TES and TES_adj. It is clear that the assumption that all responses fall on the c = 1 line will result in significant error. Note that these panels are a new calculation from the data of Esposito and Houser (2017).

It is possible for a light source to transpose a free cap with a fixed end cap of its given tray. In this scenario, any placement of that cap on the tray is erroneous. Because participants are not permitted to omit caps from the tray—and allowing them to do so is not advisable in order to maintain the simplicity of the testing and scoring procedure—the participant should not be penalized for the error. In this scenario, the transposed cap can simply be removed from the tray before administering the test. Equation (2) will therefore need to be modified to account for the reduced number of caps in the tray; the calculation is otherwise straightforward.

End cap transpositions are, however, seemingly uncommon; none of the 24 structured LED spectra of Esposito and Houser (2017) transposed any of the free caps of the FM-100 hue test with any of the fixed caps; six high-pressure sodium and four mercury vapor lamps (spectra taken from the IES TM-30-15 calculator)—which are known to have poor color discrimination ability and transpose many caps (Table 1)—also cause no end cap transpositions.

6. Conclusion

This article details an adjusted error score calculation for the Farnsworth-Munsell 100 Hue Test that considers the impact of a light source’s spectra on cap order, which corrects a fundamental discrepancy between actual cap order and participant performance in color discrimination research. Studies that administer the FM-100 hue test without the recommended standard illuminant (CIE C or daylight) should consider the light source’s impact on cap order and utilize the adjusted error score calculation to avoid potentially distorted results.

Acknowledgments

The author thanks X-rite for supplying the spectral reflectance distributions of the caps of the FM-100 test and for granting permission to publish the supplemental calculator that accompanies this article. Also, the author gratefully acknowledges Kevin Houser for his help facilitating the discussion to acquire permission.

Supplemental material

An Excel spreadsheet, which performs the computations described herein, accompanies this manuscript and is available online and linked by the same DOI as this manuscript.

Disclosure statement

The author reported no declarations of interest.

Supplementary material

Supplemental data for this article can be accessed on the publisher’s website.

Additional information

Funding

This work is derived from Esposito (2016) and Esposito and Houser (2017), funded by Project CANDLE partners: Cline Betteridge Bernstein Lighting Design, Digital Filaments, Domingo Gonzalez Associates, Eaton’s Cooper Lighting Business, Fisher Marantz Stone, Focal Point, Horton Lees Brogden Lighting Design, Lam Partners, Landscape Forms, Lighting Design Alliance, The Lighting Practice, Lutron Electronics, Naomi Miller, Office for Visual Interaction Inc., Philips Lighting, Randy Burkett Lighting Design, Schuler Shook, studio i, Tillotson Design Associates, and Traxon Technologies.

Notes

¹. For this equation to work properly, the terms |C_j − C_{j + 1}| and |C_j − C_{j − 1}| must each be equal to 1 when the caps are ordered correctly. Because the first free cap in tray A is numbered 85, not 1, the calculation requires a dummy array that assigns cap 85 a value of 1, cap 1 a value of 2, and so on.

². The cap error must be calculated for the end caps of the try; otherwise, results will be incorrect when errors are made near the tray’s ends; this necessitates the n + 2 term in the equation, the 2 accounting for each end cap.

AppendixScenario 1

In this example, the light source does not cause any cap transpositions (R_d = 0; see Section 4), and a participant makes one transposition in physical test tray B (BES = 4). Because the light source does not cause any transpositions, (1) applies.

• j is the cap counter from (1);

• Response order is the order of caps of tray B, as ordered by a hypothetical participant;

• CE_j is the cap error associated with the jth cap.

  Table A1. Worked example where R_d = 0 and BES = 4.

  j	Response order	CEj = |Cj − Cj − 1| + |Cj − Cj + 1|
  1	21	= ABS(21–20) + ABS(21–22)	= 2
  2	22	= ABS(22–21) + ABS(22–23)	= 2
  3	23	= ABS(23–22) + ABS(23–24)	= 2
  4	24	= ABS(24–23) + ABS(24–25)	= 2
  5	25	= ABS(25–24) + ABS(25–26)	= 2
  6	26	= ABS(26–25) + ABS(26–27)	= 2
  7	27	= ABS(27–26) + ABS(27–28)	= 2
  8	28	= ABS(28–27) + ABS(28–29)	= 2
  9	29	= ABS(29–28) + ABS(29–31)	= 3
  10	31	= ABS(31–29) + ABS(31–30)	= 3
  11	30	= ABS(30–31) + ABS(30–32)	= 3
  12	32	= ABS(32–30) + ABS(32–33)	= 3
  13	33	= ABS(33–32) + ABS(33–34)	= 2
  14	34	= ABS(34–33) + ABS(34–35)	= 2
  15	35	= ABS(35–34) + ABS(35–36)	= 2
  16	36	= ABS(36–35) + ABS(36–37)	= 2
  17	37	= ABS(37–36) + ABS(37–38)	= 2
  18	38	= ABS(38–37) + ABS(38–39)	= 2
  19	39	= ABS(39–38) + ABS(39–40)	= 2
  20	40	= ABS(40–39) + ABS(40–41)	= 2
  21	41	= ABS(41–40) + ABS(41–42)	= 2
  22	42	= ABS(42–21) + ABS(42–43)	= 2
  23	43	= ABS(43–42) + ABS(43–44)	= 2

$BES=\left(\sum _{j=1}^{n+2}C{E}_j\right)-\left(\left(n+2\right)\ast 2\right)=50-\left(21+2\right)\ast 2=50-46=4.$

Scenario 2

In this example, the light source causes one transposition in tray B (R_d,_B = 4; see Section 4), and the participant does not correctly respond to that transposition (BES_adj = 4). Note that if (1) was applied to this participant’s response, he or she would have an error score of zero (BES = 0), which would be erroneous because the participant did not correctly respond to the light source.

• j is the cap counter from (2);

• Source is the order of the caps of tray B, under a hypothetical light source;

• Response order is the order of caps of tray B, as ordered by a hypothetical participant;

• ID is a fixed dummy array of numbers used as an intermediate variable to determine P_j;

• P_j is the place number from (2), which renumbers the participant’s order of caps based on their correct place compared to the source order. In this example, Excel function VLOOKUP was used to calculate P_j;

• PE_j is the error associated with the jth place.

  Table A2. Worked example where R_d,B=4 and BES_adj=4.

  j	Source	ID	Response order	Pj	PEj = |Pj − Pj − 1| + |Pj − Pj + 1|
  1	21			0	= ABS(0−(−1)) + ABS(0–1)	= 2
  2	22	1	22	1	= ABS(1–0) + ABS(1–2)	= 2
  3	23	2	23	2	= ABS(2–1) + ABS(2–3)	= 2
  4	24	3	24	3	= ABS(3–2) + ABS(3–4)	= 2
  5	25	4	25	4	= ABS(4–3) + ABS(4–5)	= 2
  6	26	5	26	5	= ABS(5–4) + ABS(5–6)	= 2
  7	27	6	27	6	= ABS(6–5) + ABS(6–7)	= 2
  8	28	7	28	7	= ABS(7–6) + ABS(7–8)	= 2
  9	29	8	29	8	= ABS(8–7) + ABS(8–10)	= 3
  10	31	9	30	10	= ABS(10–8) + ABS(10–9)	= 3
  11	30	10	31	9	= ABS(9–10) + ABS(9–11)	= 3
  12	32	11	32	11	= ABS(11–9) + ABS(11–12)	= 3
  13	33	12	33	12	= ABS(12–11) + ABS(12–13)	= 2
  14	34	13	34	13	= ABS(13–12) + ABS(13–14)	= 2
  15	35	14	35	14	= ABS(14–13) + ABS(14–15)	= 2
  16	36	15	36	15	= ABS(15–14) + ABS(15–16)	= 2
  17	37	16	37	16	= ABS(16–15) + ABS(16–17)	= 2
  18	38	17	38	17	= ABS(17–16) + ABS(17–18)	= 2
  19	39	18	39	18	= ABS(18–17) + ABS(18–19)	= 2
  20	40	19	40	19	= ABS(19–18) + ABS(19–20)	= 2
  21	41	20	41	20	= ABS(20–19) + ABS(20–21)	= 2
  22	42	21	42	21	= ABS(21–20) + ABS(21–22)	= 2
  23	43			22	= ABS(22–21) + ABS(22–23)	= 2

$BE{S}_{\mathrm{a}\mathrm{d}\mathrm{j}}=\left(\sum _{j=1}^{n+2}P{E}_j\right)-\left(\left(n+2\right)\ast 2\right)=50-\left(21+2\right)\ast 2=50-46=4.$