Subjective Assessments of Lighting Quality: A Measurement Review

ABSTRACT

Lighting is an important component of indoor environmental quality that can affect occupant satisfaction, well-being and productivity. Lighting quality is a broad abstract concept and this has implications for its assessment. Subjective evaluations of lighting are an important complement to objective photometric information; however, there is limited existing guidance for the selection of such measures. We review and highlight the advantages and limitations associated with measures of general lighting quality and discomfort glare. Existing measures of lighting quality have broad coverage of individual lighting features but do not always clearly form cohesive scales measuring an underlying construct. Questions used in experimental glare research focus narrowly on glare severity, with ambiguous response rating scales. There is a need for the development of reliable and valid tools to assess lighting quality and its components, with clearly defined definitions and constructs, and explicit reporting of psychometric scale properties. The development of rigorous self-report tools will improve the understanding and design of quality lighting environments.

KEYWORDS:

• Glare
• lighting quality
• lighting satisfaction
• postoccupancy evaluation
• visual discomfort

1. Introduction

Indoor environmental quality has implications for the satisfaction, well-being, and productivity of building occupants (Humphreys and Nicol 2007; Newsham et al. 2009; Singh et al. 2010). Together with thermal, acoustic, and air quality components, lighting is a key aspect of indoor environmental quality. Lighting can affect occupant well-being and productivity, and dissatisfaction with lighting can influence operational costs if it results in redesign or unanticipated occupant intervention into the environment. Although a desirable outcome, lighting quality remains an abstract concept without a widely agreed definition. Boyce (2014) suggests that lighting quality is best considered in terms of its impact on outcomes such as visual performance and behavior, and Veitch and Newsham (1996) define lighting quality more broadly, as lighting that supports visual performance, task and behavioral performance, social interactions, mood, health and safety, and aesthetic judgments. Integral to both definitions are acknowledgement of the purpose of the space (the tasks or activities that take place in a particular setting) and the user of a space (the individual completing the tasks and activities), suggesting that lighting quality depends on an interaction between lighting, place, and person. Evaluation of lighting typically involves detailed photometric measures of the environment; however, photometric measures alone do not fully capture the “person” component of lighting quality. Established protocols for describing lighting (CIE 213: 2014; International Energy Agency [IEA] 2016) acknowledge the importance of capturing subjective evaluations in addition to objective photometric information; however, they do not provide any summary, comparison, or guidance for selecting measures. Therefore, it is not clear how existing subjective evaluation measures overlap and what gaps remain. This review aims to summarize existing measures of subjective lighting quality, identify advantages and limitations of existing surveys to inform measure selection, and provide recommendations for future measure development.

2. Review of measures

To identify measures, we conducted database searches, examined established protocol and guideline documents from international lighting bodies, and examined methods and reference lists of articles reporting subjective evaluation. We identified four types of subjective measures: general evaluations of lighting quality, questions examining discomfort glare (one specific feature of the lighting environment), indoor environmental quality scales, and affective response scales. Of these, general lighting quality measures and measures of glare were the most prevalent and are reviewed in detail.

2.1. Measures of general lighting quality

Subjective assessments form an important component of postoccupancy evaluations (POEs) of the built environment. There are a number of occupant self-report measures that include general lighting quality as either a substantial component or the sole focus. Tools that have not been published in a peer-reviewed source or have an associated fee were not included, because these restrictions limit utility to researchers, widespread adoption, and comparability of results. The CIE guide to protocols for describing lighting (CIE 2014) provides examples of existing measures, two of which are reviewed here (the Thorn Lighting Work Space Appraisal was excluded because it is a proprietary unpublished tool, and the Ergonomic Lighting indicator [Tralau et al. 2009] is a lighting design tool that does not capture occupant ratings). In addition to the CIE protocol examples, three other measures were included.

Table 1 summarizes the key details of the five reviewed measures, including the number of questions, whether non-lighting questions are included, the type of response format used, the time frame participants are asked to consider when responding, and whether psychometric scale properties (e.g., Cronbach’s alpha for evaluation of internal consistency of scale items or results of structural validity via factor analysis) were reported. The remaining columns indicate what specific features of the lighting environment are addressed by the tool.

Table 1. Characteristics of existing lighting quality surveys.^a

						Lighting characteristics
Survey	Number of questions	Other areas?	Response type	Time frame	Scale properties reported?	Comfort	Performance	Glare	Artificial light	Natural light	Windows	Brightness	Reflections	Distribution	Flicker	Control	Task lighting	Color	Shading	View	Overall satisfaction
Office Lighting Survey (Eklund and Boyce 1996)	10	No	Agree/disagree three-point rating scale	LT	Selected	x		P				x	x	x	x			x			x
Lighting Conditions Survey (IEA 1999)	37	Yes	Mixed (rating scales, multiple choice, free-text)	LT	No			F	x	x	x	x	x	x			x	x		x	x
National Research Council (NRC) Canada Lighting Quality Scale (Veitch and Newsham 2000)	10	No	Five-point rating scale	CC	Yes		x	P, L, S					x								x
IEA retrofit monitoring user assessment survey (IEA 2016)	36	Yes	Mixed, (rating scale and free-text)	LT & CC	No			F	x	x	x	x			x	x		x	x	x
Satisfaction with Environmental Features Veitch et al. (2007)	18	Yes	Seven-point rating scale	CC	Yes			S				x	x							x

^aP = presence, L = location, S = severity, LT = long-term, CC = current conditions.

2.1.1. Office lighting survey

The office lighting survey includes evaluations of broad (overall satisfaction and comfort; for example, “Overall, the lighting is comfortable”) and specific characteristics of lighting (illuminance, brightness ratio, veiling reflections, discomfort glare, color, and flicker) over the long term. It includes statements that assess quality (e.g., “The lights flicker throughout the day”), comfort (e.g., “Overall, the lighting is comfortable”), and performance (e.g., “Reflections from the light fixtures hinder my work”). It has been used in both field and experimental settings (Borisuit et al. 2014; Boyce et al. 2006; Eklund and Boyce 1996).

2.1.1.1. Advantages

The survey allows diagnosis of issues resulting from individual lighting features, as well as an overall assessment of lighting quality. Convergent validity is thoroughly demonstrated via associations with previously developed tools, as is test–retest reliability (the stability of a test over time) over one week. The authors clearly identify the aim of their measure and rationale for its development. Normative data for this survey are reported, providing a practical comparison point for those using the tool in the field.

2.1.1.2. Limitations

The survey does not evaluate daylighting; however, an additional daylighting question is described by Veitch (2010). The item evaluating glare (“The light fixtures are too bright”) does not explicitly address discomfort and excludes daylighting. One question requires a comparative quality judgment (i.e., “How does the lighting compare to similar workplaces in other buildings?”), which may produce unreliable responses. Some scale properties such as internal reliability are not reported, although many aspects of reliability and validity are examined. Finally, the norms provided are based on surveys of offices in the mid-1990s. Given the changes in building design and lighting technology in the intervening period, these norms may no longer be a useful reference point.

2.1.2. Lighting conditions survey

Developed as part of the Daylight in Buildings task for the IEA Solar Heating and Cooling Programme (IEA 1999), the Lighting Conditions Survey consists of 37 items that assess long-term lighting satisfaction, quality, and other features of the workplace (i.e., office layout, orientation) in a field setting. It includes broad questions relating to overall satisfaction, as well as questions about individual aspects of lighting (e.g., “Does the lighting cause reflections in your work material?”).

2.1.2.1. Advantages

Selected items from this survey have been used or adapted in subsequent research (Aries et al. 2010; Velds 2002), providing the potential for comparison across different settings. The survey covers many different individual lighting characteristics, including a particular focus on daylighting (see Table 1 for a summary of individual lighting characteristics covered).

2.1.2.2. Limitations

A small number of specific lighting features are not addressed (e.g., flicker and control), nor is an overall rating of visual comfort. Questions focus on identification of individual problematic lighting features, and questions describing the environment and environmental aspects (e.g., “Do you ever notice cold draughts near the windows?”) are interspersed with judgments of satisfaction and quality, without an attempt to clearly distinguish how questions group together or to measure defined constructs.

2.1.3. NRC Canada Lighting Quality Scale

Veitch and Newsham (2000) developed this scale as part of an office simulation experiment. It includes broad overall satisfaction questions, as well more specific assessment of individual features (i.e., visual performance, presence, location and severity of glare, and the presence of reflections). The phrasing of the questions relates to lighting during the current day.

2.1.3.1. Advantages

The authors performed factor analysis to identify two subcomponents of their scale, lighting quality and glare, each of which consists of multiple questions that can be averaged to give an overall subscale score and has high internal reliability (as demonstrated via Cronbach’s alpha). Multiple characteristics of glare are recorded in this survey.

2.1.3.2. Limitations

This survey was designed for use in a mock-up experimental setting. Therefore, it measures only selected lighting features, uses a day-long reference period, and does not include any daylighting questions.

2.1.4. IEA retrofit monitoring user assessment survey

The lighting monitoring protocol for retrofits described by the Solar Heating and Cooling Programme of the International Energy Agency includes a “user assessment” portion where occupant ratings are captured (IEA 2016). A case report using this tool has also been described (Gentile et al. 2016). It assesses individual features but does not include a broad question assessing overall satisfaction with lighting. Assessments are made about both current and long-term conditions, and it includes judgments of satisfaction (e.g., “General satisfaction with the following aspects of this room”) and quality (e.g., “Do you ever experience flickering from the electric light sources in this room?”).

2.1.4.1. Advantages

This tool captures occupant satisfaction and evaluations of a range of specific lighting features (see Table 1 for a summary of coverage), including daylighting. The inclusion of questions relating both to current conditions and to long-term experience provides multiple perspectives to the administrator.

2.1.4.2. Limitations

Some lighting features are assessed only as a point-in-time rating. Although the authors report using scale design methodology, environmental feature questions are interspersed with lighting questions, and it is not clear how multiple items could be combined. As such, the constructs measured are not clearly identified, nor are psychometric properties reported.

2.2. Indoor environmental quality surveys

Broader surveys of indoor environmental quality generally include one or two questions addressing lighting in addition to items regarding acoustics, noise, and other characteristics of the indoor environment. Indoor environmental quality scales are generally reported as an overall environmental satisfaction or utility score without an individually identifiable lighting section and therefore most are not included in this review. However, the Satisfaction with Environmental Features survey includes a substantial lighting assessment component reported as a distinct construct and is therefore included in Table 1.

2.2.1. Satisfaction with environmental features

This survey assesses occupants’ satisfaction with three separate environmental features: privacy/noise, ventilation/temperature, and lighting (Veitch et al. 2007). Five of the 181 items relate to lighting, specifically satisfaction with brightness, reflections, glare, and view. Participants are asked to respond based on conditions at the time of evaluation.

2.2.1.1. Advantages

The authors report on field validation results and use an appropriate psychometric development approach and describe the rationale, constructs assessed, scale factor analysis, and relevant scale properties.

2.2.1.2. Limitations

Because it is part of a broader questionnaire on environmental features, this survey is brief and measures few aspects of lighting quality.

2.3. Subjective ratings of discomfort glare

All of the general lighting quality scales reviewed included questions relating to glare, indicating its importance for any occupant assessment of lighting. Glare is one specific component of subjective lighting quality that has received considerable experimental attention. Glare occurs when the range of simultaneous luminances within the field of view causes discomfort or loss of visual performance (Boyce 2014). Discomfort glare causes annoyance or discomfort to the viewer, and a large body of experimental work has attempted to describe the physiological mechanisms involved. Although it is a subjective process, there has been substantial work in the development of glare indices in an effort to quantify the likelihood of glare occurring in a given luminous scene. Glare indices such as the daylight glare probability index (Wienold and Christoffersen 2006) assess the luminances in a visual field to estimate the likelihood of glare occurring. However, glare prediction indices do not entirely explain the perception of glare, and their performance degrades the further they are applied from highly controlled settings (Jakubiec and Reinhart 2015; Van Den Wymelenberg and Inanici 2016).

As Boyce (2014) comments, even with the best characterization of the physiological processes associated with glare “… there would still be the fundamental fact that discomfort glare involves a psychological element in the form of the different criteria used by individuals” (179). Glare indices represent only a measure of glare potential rather than being a direct measure of the experience of glare itself, which requires assessment from a viewer. Further, subjective observer ratings are used to validate glare prediction indices and therefore directly inform their efficacy. As a result, it is important to consider how subjective assessments are obtained. The majority of studies assessing glare are completed in highly controlled experimental settings, where specific lighting conditions or individual fixtures are manipulated carefully in order to establish photometric thresholds for glare response. Table 2 describes individual items that have been developed for this purpose. Although some are administered alongside other questions relating to lighting and glare, single items are used as the outcome variable capturing glare severity. All of the items in Table 2 were developed for use in experimental settings to gather point-in-time ratings of current conditions.

Table 2. Items used to assess glare severity in experimental settings.

Source	Question details	Examples of subsequent use
De Boer (1967)	Nine-point rating scale, cues at odd numbers: 1 = unbearable, 3 = disturbing, 5 = just acceptable, 7 = satisfactory, 9 = just noticeable	Bullough et al. (2008); Balk and Tyrrell (2011)
Osterhaus and Bailey (1992)	Exact question wording not always reported 4-point rating scale: imperceptible, noticeable, disturbing, intolerable Definitions quantifying the amount of time individuals could work with glare present often provided	Christoffersen and Wienold (2004); Wienold and Christoffersen (2006); Jakubiec and Reinhart (2015); Karlsen et al. (2015); Rodriguez et al. (2015)
Iwata et al. (1992)	Exact question wording not always reported Four-point response scale: just perceptible, just acceptable, just uncomfortable, just intolerable	Kent,  Altomonte et al. (2017); Tuaycharoen and Tregenza (2007; with additional scale points); Velds (2002; with behavioural definitions and a “no glare” response)
Van Den Wymelenberg and Iancini (2014)	Visual comfort measured using the question: “This is a visually comfortable environment for the intended work” Seven-point rating scale (strongly disagree to strongly agree)	Van Den Wymelenberg and Inanici (2016); Mahić et al. (2017)

2.3.1. De Boer scale

One of the oldest and most widely used glare scales is that developed by De Boer (1967). This question asks participants to rate currently experienced glare on a nine-point scale, with anchors at 1 = unbearable, 3 = disturbing, 5 = just acceptable, 7 = satisfactory, and 9 = just noticeable. The scale was originally designed for the evaluation of visual perception in road traffic in motorists.

2.3.1.1. Advantages

This scale has been used in research assessing glare from indoor and outdoor luminaire sources.

2.3.1.2. Limitations

The limitations associated with the De Boer scale (and other glare scales) are summarized thoroughly by Fotios (2015). Of most concern is its failure to provide an option for reporting the absence of glare—the response scale presumes some level of glare. This is a major drawback—asking participants to rate the extent of the glare they are currently experiencing is inappropriate if there is none present. In addition, the scale has very ambiguous descriptive anchors, which may not be consistently interpreted by participants. It was designed for the assessment of the glare following direct exposure to luminaires and therefore may not transfer well to capturing the overall experience of glare in a field setting, particularly where daylight components are present.

2.3.2. Imperceptible–intolerable four-point scale

This four-point rating scale, as originally described by Osterhaus and Bailey (1992), is frequently used to capture subjective assessments of glare in experimental settings. The exact wording of the question varies across studies; however, the verbal cues for the four-point rating scale are generally imperceptible, noticeable, disturbing, and intolerable. Many studies, but not all, provide participants with an extended descriptive definition for each scale point describing the length of time that glare could be tolerated. Ratings on this four-point scale are often dichotomized at the point between imperceptible and noticeable or between noticeable and disturbing (Rodriguez et al. 2017; Van Den Wymelenberg 2014). An adaptation of this scale has been used by Jakubiec and Reinhart (2015) to assess the experience of glare over the long term (an academic semester), asking participants to rate the typical amount of discomfort glare they experienced in the morning, midday, and afternoon, with additional questions to assess glare source and intervention strategies. Some subsequent reports use this in conjunction with questions assessing other aspects of glare and lighting (Wienold and Christoffersen 2006).

2.3.2.1. Advantages

This rating scale has been widely used, which facilitates the comparability and statistical comparison and combination of studies conducted in different settings. Additionally, glare indices used to estimate the glare potential of the luminous environment (such as the daylight glare probability index) have been validated against these scale anchors in experimental settings. Therefore, if a main aim is to compare and combine results with other studies, this scale is desirable. The adaptation of this question to a long-term rating of the environment (Jakubiec and Reinhart 2015) provides a useful bridge between the measures used in experimental glare research and field settings.

2.3.2.2. Limitations

The question used to elicit participant responses is not always the same, which may impede comparability between studies. The phrasing often implies the presence of glare, and the response scale points are ambiguous. The lowest rating (imperceptible glare) is not plain English and is an oxymoron—by definition, glare is a perceptual process that requires a sensation of annoyance or discomfort to exist. This could be rectified by substituting imperceptible with another scale anchor that clearly communicates that no glare is present. The use of unnoticeable by Rodriguez et al. (2015) as the lowest response scale point improves but does not entirely remove this issue. The use of extended behaviorally anchored definitions is likely to provide more reliable judgments; however, including definitions may not be feasible or desirable in a field setting.

2.3.3. Glare sensation vote

A glare sensation vote subjective rating (Hopkinson 1972; further adapted by Iwata et al. 1992) is similar to the above imperceptible–intolerable scale (Osterhaus and Bailey 1992), with slightly different scale presentation and labels. It has also been used with extended behavioral definitions (e.g., Kent, Fotios et al. 2017).

2.3.3.1. Advantages

Versions of this scale have been used in numerous studies, sometimes with the addition of a no glare response and behavioral definitions for each of the scale points, which improves clarity for respondents.

2.3.3.2. Limitations

As with the previous scale, there is not always an option to indicate that there is no glare present. The question is not always clearly described, and the semantic descriptors of the response scale do not by themselves communicate a clear order of increasing severity.

2.3.4. Visual comfort rating

Van Den Wymelenberg (2014) proposes that glare may be better captured by phrasing questions not in terms of discomfort but in terms of comfort. A series of studies (Mahić et al. 2017; Van Den Wymelenberg and Inanici 2014, 2016) have used a positively worded question with a seven-point response scale to assess glare. This question has been rated alongside more general questions of lighting quality that address visual comfort, visual appearance, brightness, reflections, and distribution.

2.3.4.1. Advantages

This question is easy to understand and does not presume the existence of glare in the phrasing of the question. The verbal anchors are clearly ordered, unambiguous, and do not require extended behavioral definitions.

2.3.4.2. Limitations

The construct of visual comfort can be interpreted more broadly than the absence of glare. This question is suitable for assessing glare in experimental settings where the lighting conditions are carefully managed and manipulated (the setting this item was originally applied in). However, visual comfort can be influenced by other aspects of lighting (e.g., flicker), and in a field setting this question may not be specific to glare.

2.4. Other subjective measures of lighting

2.4.1. Descriptive scales

Flynn et al. (1979) provide a series of semantic differential scales to evaluate user impressions resulting from a series of lighting scenarios. Descriptive prompts such as “clarity,” “order,” “spaciousness,” “relaxation,” and “privacy” are rated by occupants to characterize their affective response to the environment. These scales allow a more impressionistic description of the lighting in a space in a way that the previously discussed surveys do not. They have been used to help understand how lighting design affects occupants (Durak et al. 2007; Manav and Yener 1999). However, this descriptive approach does not provide any indication of whether these characteristics match the observer’s aesthetic preferences, making them difficult to translate into a measure of lighting quality or satisfaction.

2.4.2. Lighting preferences, beliefs, and behavioral consequences

Finally, there are also studies of general lighting preferences (Haans 2014; Veitch et al. 2010) or beliefs (Veitch and Gifford 1996); however, we believe that these are better conceptualized as individual differences that predict subjective assessments of lighting quality, rather than forming part of the lighting quality assessment itself. Additionally, behavioral consequences of poor lighting quality (e.g., use of blinds) are a related yet conceptually distinct set of behaviors. As a result, none of these have been included in this review.

2.5. Gaps and recommendations for future measure development

2.5.1. General lighting quality scales

The reviewed general lighting quality surveys differed considerably in terms of type and breadth of content (i.e., lighting characteristics), the time period they asked respondents to consider (i.e., current conditions vs. long-term assessments), and the type of judgment they asked people to make. Readers can use Table 1 in combination with the in-text descriptions to select or adapt a measure that may be useful for their particular setting. We identified a number of overall limitations with existing subjective assessments of lighting and provide some suggestions for the future development of measures.

2.5.2. Consensus on lighting characteristics

There is not currently agreement on which individual lighting characteristics should be included in a subjective assessment of lighting quality. Though all addressed glare, questions regarding daylighting/natural lighting, flicker, lighting control, and task lighting were inconsistently included in measures (see Table 1). None of the measures included items relating to aesthetic judgments or satisfaction, nor did any address variability of light across the day, which may be relevant in heavily daylit spaces or as a result of variable or smart lighting schemes.

The field would benefit from some consensus on what specific lighting characteristics should be included in subjective assessments of lighting environments. Future protocol documents could recommend not only that subjective evaluations should be gathered but also specify what lighting characteristics should be assessed by occupants. The lighting characteristics listed in Table 1 could serve as a starting point for this discussion. We suggest that a broad coverage of characteristics would be of most value and that questions relating to daylighting are critical inclusions for field surveys, given the heavy reliance on daylighting strategies in modern buildings.

2.5.3. Evaluation type

Judgments of lighting quality (rating from bad to good), judgments of satisfaction (unsatisfactory to satisfactory), or endorsement of a problem or issue (e.g., “The lighting causes deep shadows,” as included in the Office Lighting Survey) were all used within surveys. Authors should think critically about the relevance of the type of judgments they are asking occupants to make about the lighting and whether or not it is useful to combine different types of evaluations. Depending on context, satisfaction ratings may be more meaningful than quality ratings, which rely on knowledge and may not always be suitable for nonexpert occupant respondents (Veitch et al. 2007).

2.5.4. Articulation of constructs and use of psychometric scale development methods

There is a focus on asking respondents to rate or identify problems with individual lighting characteristics at the expense of broader evaluations (e.g., of satisfaction or comfort). It is not always clear how items group together or how existing questionnaires could map onto broader components and existing definitions of lighting quality (e.g., including visual comfort, performance, and aesthetics). Two of the measures reviewed reported no psychometric properties of their scale (such as internal reliability, factor structure, or correlations with other related measures) to assess validity and reliability, despite previous calls for authors to use psychometric methods for measure development (CIE 2014). In these measures, lighting appraisal questions are presented intermixed with questions assessing other aspects of the physical environment (e.g., type of office or proximity of windows). Although this broader environmental information is critical to collect in a POE evaluation, these questions do not themselves directly address lighting quality and should be clearly differentiated.

There are conventional statistical approaches for the creation of self-report measures that are described extensively in psychological literature and have been widely adopted within other disciplines (see Cronbach and Meehl 1955; DeVellis 2012). Because subjective experiences are intangible and not directly observable, we rely on self-report instruments to measure underlying constructs—in this case, lighting quality—and their relationships. At a minimum, scale development requires clear articulation of the theoretical constructs to be measured and a sound argument for why items relate to that construct. Following this, a comprehensive pool of items (larger than the eventual intended length of the measure) is generally administered to an initial group and reduced based on initial validation. Statistical psychometric methods are used to refine the item pool. Factor analysis can be used to identify and confirm the underlying structure of a scale (and possible subscales) that estimates levels of a more abstract theoretical construct (e.g., satisfaction with lighting). Future measures should document the rationale and conceptual basis for their development and overtly label the variable being measured by each question or group of questions. Methods and rationale for measure development should be published with the results of psychometric analyses such as internal consistency of the items in a scale, factor analysis, or item response analyses.

Rather than being a perfunctory exercise, engaging in formal statistical analysis of scale properties helps authors determine what construct is being measured, reveal the structure of any subcomponents, explore whether questions group together effectively, assess similarity to and difference from other related constructs and measures, and identify whether there are particular items that are not performing as intended. These analyses help the authors to demonstrate the quality of their measures and identify relationships and hierarchies between the individual items and broader constructs and theoretical frameworks. They can also be used to identify which questions are most valuable to retain in shorter forms or in broader surveys of indoor environmental quality.

2.6. Glare scales

Glare is a more specific construct representing just one component of lighting quality. The single-item questions shown in Table 2 have generally been applied in more controlled research settings. As such, there is more clarity regarding what is being measured and how it is being measured. Despite this, there remains a lack of agreement on a universal rating scale (Fotios 2015; Van Den Wymelenberg), which limits comparability between experimental studies assessing glare. Fotios (2015) notes that despite the creation of glare indices based on the results of subjective ratings, there is still considerable ambiguity in the methods used to elicit those ratings.

It may be that measurement of behavioral or physiological responses is a more useful indicator of glare in some settings than stated preferences (Fotios 2018). However, behaviors are not themselves preferences and can be influenced by other factors, particularly in a field setting. For example, the use of blinds may serve as a practical indicator of glare presence in a field setting but is also dictated by external factors such as view desirability and social dynamics. Therefore, rating scales will likely remain an important component of glare evaluation.

2.6.1. Dichotomization of multipoint scales

Multipoint ratings of glare are often dichotomized to create categories of comfort and discomfort; however, there is disagreement on the placement of this border, creating inconsistencies and lack of comparability between studies (Van Den Wymelenberg). The dichotomization of continuous variables can be problematic for both conceptual and statistical reasons. Dichotomization considerably reduces statistical power and increases the likelihood of false-positive results (i.e., detecting discomfort where none exists). Dichotomized results do not accurately represent the differences between groups, because they treat respondents straddling the borderline as being very different, when in fact they are very similar (Altman 2006). However, the ongoing difficulty in identifying a clean threshold that differentiates luminous scenes that are comfortable from those that are problematic suggests that imposing a cut-point does not solve these issues and instead runs the risk of creating spurious results. Analyzing interval or multipoint data requires different statistical approaches but is very feasible, and we recommend increased analysis of these scales in their gathered format.

2.6.2. Question construction

Most existing glare questions do not provide respondents with an option to indicate absence of discomfort or glare, and some response scale anchors (i.e., the descriptive words associated with each scale point) are ambiguous and not clearly ordered. Further, the exact wording of questions used to elicit ratings is not always reported in full, leaving room for procedural discrepancies and reduced comparability between studies. A sizeable body of literature has shown that the wording and order of questions has a demonstrable impact on participant responses, and question wording has implications for readability, translations, and study replication.

The exact wording of questions posed to participants and the associated response scale anchors should always be reported in full. Any changes to the wording of previously published measures should be explicitly described, and questions should be phrased to avoid inferring the presence of discomfort glare. Response options must include a clear option for participants to indicate that they are not experiencing discomfort or glare, and response scale descriptors should have an unambiguous order that indicates increasing or decreasing severity.

2.6.3. Additional glare characteristics

Glare research has focused heavily on luminous conditions that lead to the acute experience of glare. When glare is assessed at a point in time (i.e., observers are asked to rate current conditions) in a highly controlled experimental setting, the primary interest is the severity of glare, and the questions in Table 2 address glare severity (or simply glare presence when they are dichotomized). However, in POE or field settings, researchers, practitioners, and building managers may be more interested in occupants’ experiences of glare over the long term.

Across this broader time frame, other glare characteristics may be of interest, such as glare source, which is often controlled in an experimental setting (Christoffersen and Wienold 2004; Hirning et al. 2013), glare frequency (IEA 2016), glare duration, and glare timing (Jakubiec and Reinhart 2015), both across the day and across the year. The general lighting quality surveys all include glare questions. Some ask occupants to rate the frequency rather than severity of glare; for example, “Does the artificial light ever cause glare strong enough to bother you?” (IEA 1999) or, “In general, how often do you experience glare from direct sunlight?” (IEA 2016). These additional glare characteristics have not been systematically described and are therefore not reviewed separately. However, describing and defining these additional glare characteristics and measuring them in field settings will allow us to better determine their importance and better predict the experience of glare. We recommend that authors consider which glare characteristic they are assessing and explicitly state this when describing their measures.

2.7. Broad recommendations for lighting research

2.7.1. Consistency across research areas

The field would benefit from greater consistency in measures used to assess glare and lighting quality in general. Questions designed for administration in experimental contexts for moment-in-time ratings may differ from those designed for long-term assessments in field settings. However, there could be greater consistency in the structure or phrasing of questions to allow both types of measurements to be meaningfully integrated and compared. Additionally, the discomfort glare literature often uses the terms visual discomfort and glare interchangeably. Visual comfort can be influenced by a range of lighting issues, including insufficient light and flicker (Boyce 2014), and may represent a slightly broader construct. Consideration of theoretical definitions and careful construction of lighting quality scales could help to resolve this hierarchy of constructs.

2.7.2. General advice for scale construction

We strongly recommend that scale development texts are consulted in the development of future measures, because there is a wealth information available regarding the construction of questions for self-report scales (see DeVellis 2012; Kline 1986). In general, questions should avoid using technical jargon and ambiguity, remain relatively short, and ensure that they address only one aspect or issue. Questions should avoid leading the respondent to a particular conclusion (e.g., inferring the presence of glare), and negatively worded questions should be written with care, because they can be confusing to readers. Response rating scales should be labeled in a way that implies approximately equal weighting across the scale, is unambiguous, and has a clear increasing or decreasing order. There are mixed views regarding the optimal number of scale points in Likert-type responses; however, five- to seven-point scales are generally desirable, because higher numbers can become impractical and lower numbers can provide inadequate variation.

Although single-item measures are desirable for their brevity, asking multiple questions measuring the same underlying variable reduces the likelihood that the specific phrasing of the question results in misinterpretation by the respondent. Further, multi-item measures allow researchers to assess the internal consistency of responses using Cronbach’s alpha. Single items that effectively measure a construct can emerge from and be validated against longer measures, although even carefully designed single-item responses are typically less stable than those obtained from multi-item measures (DeVellis 2012).

In experimental settings where it is practical to obtain only one or a handful of ratings per condition, identical conditions can be repeated and responses averaged to give a more stable subjective estimate of the measured variable. However, even well-designed subjective scales cannot rectify oversights in experimental design that allow for the influence of confounding variables, order effects, and anchoring biases (see Kent, Fotios et al. 2017; Kent et al. 2018). These effects can be avoided through careful experimental design (e.g., via randomization or counterbalancing of conditions) and are not specific to rating scales—they can also affect behavioral indicators or physiological responses.

Finally, though questions that name a specific task (e.g., computer work) or setting (e.g., offices) increase the tangibility of questions to participants, they have narrow applicability. Increased transferability to a wider variety of settings could be achieved by writing questions that are task and setting agnostic (i.e., use of language such as “this space” or “usual tasks or activities”) or by phrasing questions such that the named task or setting can be easily substituted without changing question structure. More broadly applicable questions may encourage increased examination of lighting quality outside of an office building context.

3. Conclusions

Subjective evaluations of lighting are an important complement to photometric measures when determining lighting quality. Existing protocols acknowledge the role of subjective measures of lighting, and this review reveals that although a number of such measures exist, there is room for further development of reliable and valid tools to assess lighting quality and its components. Therefore, development of future measures should be based on clear definitions of lighting quality and its component parts, follow established methods for scale development, and explicitly report the psychometric scale.

The lack of consensus on scales assessing lighting quality greatly limits the comparability of results and the pooling of data from multiple studies to compare and contrast lighting quality in different settings. Greater consensus and consistency in measures would enable broader examination of how different environmental, architectural, and lighting features influence occupants’ judgments of lighting quality and satisfaction. Further, consistent implementation of standard measures would allow establishment of norms as originally undertaken with the Office Lighting Survey (Eklund and Boyce 1996). This would dramatically improve the utility of such scales for practitioners, who could compare results from individual postoccupancy evaluations to existing published norms and report benchmarks to clients.

Gathering subjective assessments is inherently less precise than capturing physical measurements but is no less important. The careful design of subjective measures can greatly reduce measurement uncertainty. By developing reliable and valid tools, we can improve our understanding of lighting quality and establish design solutions that lead to more satisfied building occupants.

Disclosure statement

Gillian Isoardi is a consultant for Light Naturally, who have provided support to an associated project. Light Naturally has no business or financial benefits directly relating to the contents of this review. The other authors have no interests to declare.

Additional information

Funding

This work was supported by Australian Research Council Linkage grant “Designing Healthy and Efficient Lighting Environments in Green Buildings”(LP150100179) in partnership with AECOM and Light Naturally.