Judging the Scientific Quality of Applied Lighting Research

Figures & data

Fig. 1 The original 1941 Kruithof paper includes descriptions of linear fluorescent lamps and their operation in addition to the consideration of light source color temperature and illuminance. Extract reproduced here by kind permission of Royal Philips.

Fig. 2 The research quality considerations discussed here. These interrelate and leave a hole if one is not addressed in sufficient detail.

Table 1. Examples of the questions that might be considered when reviewing the work of others that reports scientific research results. Manuscripts of this type should contain the usual sections of introduction, methods, results, discussion, conclusions, and references. Authors can expect to be challenged by reviewers and editors if critical details are missing.

Review issue mapped to manuscript section	Questions addressed by reviewers
General considerations	• Is the topic original, novel, and sufficiently important for archiving and broadcasting to the lighting community? • Are all components of the manuscript presented with clarity and completeness? Is the quality of the writing clear and concise? • How well organized is the material? • Are figures and tables well crafted and sufficiently captioned? Do they support comprehension of the study?
Introduction	• Is the research question clear? • Is the development from previous research clear? To what extent is relevant prior work known, used, and cited? How extensive are the references? • Was there a specific hypothesis being tested? If so, what was it? • Is the hypothesis logically related to an established theory or to a well-considered idea?
Methods: Definition of variables	• What were the independent variables: What did the researcher vary or measure as a predictor variable? Are the variables reported in sufficient detail to permit replication? • What were the dependent variables: What outcomes or effects did the researcher measure? Are the dependent variables clearly related to the hypothesis or theory being tested? • What were the control variables: What quantities were intentionally limited in their variation?
Methods: Procedures and credibility	• Was the choice of procedure(s) defended, with anticipated limitations stated? • Were the setup and procedure reported in sufficient detail to permit repetition? • What steps were built into the test procedure to ensure internal validity? • How many test participants were there, and why was this considered sufficient? What were their characteristics?
Results: Strength and extent of data analysis	• What were the results? Are measures of central tendency and variance reported or only the central tendency? • Where results are presented graphically, is it possible to discriminate between the test results and assumed intervening values (i.e., are the data points shown)? • How appropriate and thorough is the analysis of the data? Is it sufficient to support the claimed relationships? What inferential statistics were performed? Are effect sizes reported along with the test results?
Discussion	• Are sufficient data presented to support the claims and interpretations? • Are the caveats (limitations) of the study acknowledged? • Are the results extended appropriately to other people, places, or circumstances? • How do the findings compare with those from previously published work? • How and to what extent do the results advance the state of knowledge, and is this expressed clearly?
Conclusions	• Was the research question supported or refuted? Are the conclusions supported by the results?