Construct validity and test-retest reliability of a chrononutrition questionnaire for shift work and non-shift work populations

ABSTRACT

The irregular eating patterns of both shift workers and evening chronotypes adversely affect cardiometabolic health. A tool that conveniently captures temporal patterns of eating alongside an indicator of circadian rhythm such as chronotype will enable researchers to explore relationships with diverse health outcome measures. We aimed to investigate the test-retest reliability and convergent validity of a Chrononutrition Questionnaire (CNQ) that captures temporal patterns of eating and chronotype in the general population (non-shift workers, university students, retirees, unemployed individuals) and shift work population. Participants attended two face-to-face/virtual sessions and completed the CNQ and food/sleep/work diaries. Outcomes included subjective chronotype, wake/sleep/mid-sleep time, sleep duration, meal/snack regularity, meal/snack/total frequency, times of first/last/largest eating occasions (EO), main meal (MM) 1/2/3, and duration of eating window (DEW). 116 participants enrolled (44.5 ± 16.5 years, BMI: 27.3 ± 5.8 kg/m², 73% female, 52% general population); 105 completed the study. Reliability was acceptable for chronotype, sleep, and all temporal eating patterns except on night shifts. Convergent validity was good for chronotype and sleep except for certain shift/shift-free days. Generally, meal/snack regularity and frequency, and times of first/last EO showed good validity for the general population but not shift workers. Validity was good for DEW (except work-free days and afternoon shifts) and times of MM 1/2/3 (except afternoon and night shifts), while time of largest EO had poor validity. The CNQ has good test-retest reliability and acceptable convergent validity for the general and shift work population, although it will benefit from further validation, especially regarding regularity, frequency, and times of first and last eating occasions across more days amongst a larger sample size of shift workers. Use of the CNQ by researchers will expand our current understanding of chrononutrition as relationships between timing of food intake and the multitude of health outcomes are examined.

KEYWORDS:

• Chrononutrition
• temporal patterns of eating
• meal timing
• meal regularity
• meal frequency
• dietary questionnaire
• dietary pattern

Introduction

When we eat may be as important as what we eat for optimal cardiometabolic health. Eating later rather than earlier in the day and over extended periods is linked with impaired glycaemic control and obesity (Leung et al. 2020; Popp et al. 2021). In contrast, time-restricted eating has shown in meta-analyses to achieve similar weight loss to continuous energy restriction (Elortegui Pascual et al. 2023), and improve blood pressure, glucose, and lipid profile (Liu et al. 2022; Moon et al. 2020). These data show that efficiency of metabolic processes are aligned with the 24-h day (Bonham et al. 2019; Poggiogalle et al. 2018) due to our circadian rhythm (Johnston et al. 2016). Hence, long-term consequences of eating outside typical diurnal rhythms manifest as higher rates of chronic disease, as observed in shift workers (Hemmer et al. 2021). Similar consequences are also observed in non-shift workers, particularly among evening chronotypes (i.e., those with a later circadian preference who eat later in the day) through poorer weight loss success (Garaulet et al. 2013) and greater odds of obesity (Xiao et al. 2019). In all, these data support the American Heart Association (AHA) recommendation to optimise timing, regularity, and frequency of food intake, or chrononutrition, to better manage cardiometabolic disease risk (St-Onge et al. 2017).

However, accurate assessment of these temporal eating aspects is challenging. Current approaches to describe temporal patterns involve food diaries or phone applications that time-stamp food intake (Gill and Panda 2015; Shaw et al. 2019). However, they can be burdensome for participants and tedious for researchers in large studies (Ortega et al. 2015) or where participants’ workplaces prohibit use of electronic devices (e.g., military bases). Furthermore, electronic methods to assess consumption patterns (i.e., timing, frequency, regularity, spacing, skipping) were found to be unvalidated (Pendergast et al. 2017). Reviews of temporal eating patterns concluded that dietary assessment tools were heterogenous, preventing data aggregation, meta-analysis (Peplonska et al. 2019), and direct comparisons between studies (Mazri et al. 2019; Phoi et al. 2021). Conventional dietary assessment tools were simply not designed to capture meal timings or identify variability between days (Almoosawi et al. 2019). Moving forward, a standardized approach to data collection on temporal eating patterns is warranted to understand its impact on health and develop national guidelines regarding optimal meal timing. We developed a Chrononutrition Questionnaire (CNQ) for this purpose.

Through a scoping review, we identified relevant aspects of temporal eating patterns: meal timings, duration of eating window, and meal regularity (Phoi et al. 2021). We discussed reasons they should be characterised in relation to sleep/wake times and individual circadian timing preferences, in the form of chronotype (Roenneberg and Merrow 2016; Roenneberg et al. 2003). It is also important that the CNQ is suitable for shift workers, who form 15–20% of the workforce (Ker et al. 2010) and whose longer eating windows (Terada et al. 2019) and common practice of eating late on night shifts (Kosmadopoulos et al. 2020; Lauren et al. 2020; Molzof et al. 2017; Shaw et al. 2019) increase their health risk. Finally, in a Delphi study, experts provided feedback on the CNQ, which achieved content validation after three rounds (Phoi et al. 2021).

To our knowledge, only four questionnaires to date capture temporal eating patterns (Bertéus Forslund et al. 2002; Chakradeo et al. 2021; Gioia et al. 2022; Veronda et al. 2020). However, they have not been validated to capture chronotype, and do not identify all aspects of temporal eating patterns (e.g., regularity) or variability in food intake across shift types for shift workers (Chakradeo et al. 2021; Veronda et al. 2020). Hence, we aimed to validate our CNQ, which identifies temporal patterns of eating, sleep/wake times, and chronotype against gold standard measures of dietary intake (food diaries) and sleep patterns (sleep diaries), in both shift and non-shift working individuals. The use of this questionnaire is multi-pronged; it will assist researchers in exploring relationships between timing of food intake and disease development or to assess compliance to time-restricted feeding protocols, clinicians may use it as a periodic assessment tool to monitor changes in temporal eating patterns and effects on health indicators like glucose levels, and shift work industries may use it as a screening tool to identify employees with suboptimal temporal eating patterns who require nutritional intervention.

Materials and methods

Recruitment and participants

The study population consisted of individuals from the general population (non-shift workers, university students, retirees, and unemployed individuals) and shift work population (shift workers who performed morning, afternoon, night or split shifts). They were recruited from the University of South Australia (UniSA), the public, and shift work industries across South Australia and Victoria through a combination of e-mails, flyers, and Facebook posts, which linked volunteers to a participant information sheet and a screening questionnaire. Individuals deemed eligible were contacted to schedule their first visit. Individuals 18 years or older and who were able to read and write English were included; exclusion criteria were determined prior to recruitment of participants and are listed in Figure 1. The study was approved by the University of South Australia Human Research Ethics Committee (register number 203281).

Figure 1. Flow diagram of individuals who underwent screening, enrolment, and completed the study, including exclusion criteria.

Study procedure

Participants attended two visits during the study period, either in a clinic at the University of South Australia City East Campus or online via Zoom. At their first visit, participants provided informed consent and demographic information; including date of birth, gender, ethnicity, height, and weight. Participants who attended clinic visits had height measured twice to the nearest 1 mm using a stadiometer (SECA 216 Height Measuring Rod, SECA) and body weight measured twice to the nearest 100 g using calibrated electronic scales (TANITA Ultimate Scale 2000, Tanita Corporation, Tokyo, Japan) while individuals who participated online estimated their height and weight; the values were averaged and used to calculate body mass index (BMI). They were then asked to fill in the CNQ and a feedback form regarding its usability. These data were all collected using REDCap – a secure web-based interface for research data collection (Harris et al. 2009, 2019). Participants were then provided either hard or soft copies of the 7-day food diary, 14-day work diary, and 14-day sleep diary with instructions on completion prior to their second visit.

Visit two was scheduled two weeks after visit one. The diaries were checked for details with the participants and any incomplete entries clarified and updated. Participants were then asked to fill in the CNQ and feedback form on REDCap for a second time. Test conditions were kept similar to the first visit in terms of instructions provided and environmental setting. Participants were also reminded not to recall answers that they input into the questionnaire at their first visit, to maintain independence of each administration.

Measures

CNQ

Questions were based on findings of our scoping review (Phoi et al. 2021) and adjusted according to recommendations by experts in circadian biology and chrononutrition from our Delphi study (Phoi et al. 2021). Participants were asked in three separate sections about their: 1) wake and sleep times, 2) regularity of meals and snacks, 3) time of main meals and snacks, and time of largest meal. Each of these aspects were identified on each of work/school (i.e., tertiary education) and work-free/school-free days for non-shift workers, and morning/afternoon/night/split shift days (as relevant) and work-free days for shift workers. The language and format of the questionnaire were also refined by the expert panel to optimise readability and understanding, especially to minimise error when responses were answered by participants. A small but key format deviation from the Delphi study recommendations is the use of 24-h time instead of am/pm format, as REDCap does not support collection of time in am/pm format. Refer to Supplementary A for the CNQ, and Supplementary B for definition of terms used in the CNQ.

Feedback form for the CNQ

In order to capture face validity of the CNQ to a layperson based on readability, clarity of language and consistency of format (Taherdoost 2016), participants were asked whether: 1) the questions asked were clear, 2) the language used was easy to understand, and 3) the format of the questionnaire made it easy to answer. They were asked to rate each question on a Likert scale ranging from strongly disagree (1) to strongly agree (5), and to provide any written feedback they had in a free text box.

7-day food diary

Whilst there is no “gold-standard” reference method for assessing the timings of energy consumption, 7-day food diaries identify differences in food intake across the week, and timing and pattern of daily food intake (Stephen 2007). Additionally, they overcome the limitations of single 24-h dietary recalls and food frequency questionnaires and have been used by studies to characterise meal patterns (Leech et al. 2015b). Participants were provided with 7-day food diaries and asked to record every food or drink that they consumed over seven consecutive days. Weighing of food was not required, but portion size estimates were encouraged based on standard household measures such as cups and spoons. A pamphlet was provided that contained pictures to aid in estimation of portion sizes. Participants were asked to indicate clock times of eating occasions based on the start time of eating, and to label each eating event as a main meal (i.e., breakfast, lunch, and dinner/supper) or snack (i.e., food or drink consumed outside of main meals). All eating occasions were asked to be recorded, including non-caloric drinks (such as water or black tea). Cooking methods and brands of food/drink were encouraged to be recorded.

14-day sleep diary

Sleep diaries of at least one week are the gold standard for subjective sleep assessment (Buysse et al. 2006) and have been used as continuous assessment methods to determine the validity of other questionnaires concerning sleep-wake behaviour in healthy adults and shift workers (Juda et al. 2013; Monk et al. 2003). Participants were provided with sleep diaries to record every sleep episode, including major sleeps and naps. Participants noted start date and time of sleep (i.e., the time that participant starts attempting to sleep), time taken to fall asleep, end date and time of sleep, and whether an alarm clock was used to wake up. Mid-sleep time on free days corrected for sleep debt (MSF_SC) was used to determine an estimation of chronotype based on the Munich Chronotype Questionnaire, where use of alarm clock to wake on free days precludes it from being used to calculate chronotype (Roenneberg et al. 2007).

14-day work diary

Participants noted down the start date and time, and end date and time of each work episode. Shift workers additionally indicated if the work episode was a morning, afternoon, night shift, or split shift. The work diary allows for information collected from the food and sleep diaries to be categorised according to their relevant work/school/shift day or work/school/shift-free day types for analysis against data collected by the CNQ.

Preparing the data for analysis

Data from sleep diaries were entered into Excel and cleaned; times were converted to decimal hours. Sleep episodes were coded by day type (e.g., morning shift or free day). For each sleep episode, based on sleep and wake times, the following variables were calculated: sleep duration, mid-sleep time, and chronotype (Table S1). These variables were similarly extracted and calculated from REDCap-exported Excel files.

Data from food diaries were entered into Foodworks 10 Professional (2019), cleaned, and all eating occasions of ≤210 kJ were removed (Leech et al. 2015a). Each day’s entry was coded by day type. The following data were extracted from Foodworks into Excel: time of first eating occasion, last eating occasion, largest eating occasion (based on energy), times of main meals, and times of snacks. They allowed calculation of: duration of eating window; meal, snack, and total frequency; and meal and snack regularity (Table S1). With the exception of meal and snack regularity, these variables were similarly extracted and calculated from REDCap-exported Excel files. Refer to Supplementary Information for a detailed step-by-step guide of how data was managed.

Table 1 is a summary of outcomes and final variables of interest derived from sleep diaries, food diaries, and REDCAP that were used in analyses of test-retest, convergent validity, and Bland Altman analyses.

Table 1. Data sources and variables included in analyses. A shift worker may perform one or more of: morning shift, afternoon shift, and night shift.

	Sleep			Temporal Patterns of Eating
Population	Context	REDCap Questions	Variables	Context	REDCap Questions	Variables
General Population	Work/school days Work/school-free days	Sleep time Wake time	Sleep time (hh:mm) Wake time (hh:mm) Sleep duration (hh:mm) Mid-sleep time (hh:mm) Chronotype MSFSC (hh:mm)	Work/school days Work/school-free days	Regularity of main meals and snacks Times of all eating occasions including: Main meals Snacks Largest eating occasion(s)	Meal regularity (%) Snack regularity (%) Meal frequency (count/24 h) Snack frequency (count/24 h) Total frequency (count/24 h) Duration of eating window (hh:mm) Time of first eating occasion (hh:mm) Time of last eating occasion (hh:mm) Time of largest eating occasion (based on energy) (hh:mm) Time of main meal 1 Time of main meal 2 Time of main meal 3
Shift-workers	Between two: Morning shifts Afternoon shifts Night shifts Between two work-free days after: Morning shifts Afternoon shifts Night shifts			On the day of: a morning shift an afternoon shift a night shift a split shift a work-free day after a work-free day

Data analysis

Test-retest reliability and convergent validity were conducted following the COSMIN Study Design checklist for Patient-reported outcome measurement instruments (Mokkink et al. 2019).

Test-retest reliability was calculated to examine the degree to which values provided on the CNQ were stable over a designated timeframe (Mokkink et al. 2019). Intraclass correlation coefficients (ICCs) were calculated for continuous variables (model: two-way mixed effects model, type: average measurement, definition: absolute agreement) and weighted Kappa were calculated for categorical values (Portney and Watkins 2015), along with their 95% confidence intervals. ICCs described the extent of correlation and the agreement between the measurements (Koo and Li 2016), and reliability for values ≥ 0.75 was interpreted as excellent, 0.40–0.75 as fair to good, and < 0.40 as poor (Fleiss 1999). Strength of agreement for Kappa values ≥ 0.81 was interpreted as almost perfect; 0.61–0.80 as substantial; 0.41–0.60 as moderate, 0.21–0.40 as fair, and 0.00–0.20 as slight (Landis and Koch 1977).

Convergent validity of CNQ variables was examined through correlations with variables from the food and sleep diaries that capture the same constructs (Portney and Watkins 2015). Spearman’s rho with 95% confidence intervals (CI) (Caruso & Cliff method) was calculated, and effect sizes < 0.30 were interpreted as weak, 0.30–0.49 as moderate, and > 0.50 as large (Hays 1981; Howell 1997). Bland-Altman plots were used to quantify the agreement and variability, calculating the differences (and 95% CI) between analogous CNQ and food/sleep diary variables relative to their averages.

Descriptive statistics are presented as means and standard deviation for the study population overall, and separately for the general and shift work populations. Consistent with the approach to MCTQ analyses (Roenneberg 2015) and with findings from our scoping review (Phoi et al. 2021), chronotype cut-offs (“morning,” “intermediate,” or “evening” chronotype) were identified by splitting the MSF_SC distribution of the study population (i.e. across all participants) into thirds.

Descriptive statistics (means and standard deviations) are presented to describe participant response in the feedback form for the CNQ (questions about question clarity, ease of understanding, and format). Paired samples t-tests were used to compare responses at baseline and after 2-weeks.

The COSMIN Study Design checklist for Patient-reported outcome measurement instruments (Mokkink et al. 2019) recommended a sample size of ≥ 100 individuals for testing of construct validity and reliability. Data availability was maximised by employing pairwise deletion for each variable. The sample size used in the analysis of each variable is listed alongside results.

Results

226 individuals underwent screening, 43 were ineligible and 67 declined to participate, leaving 116 participants enrolled; a total of 105 participants completed both visits (Figure 1). Participant demographics of completers are reported in Table 2 with age ranging between 18–83 years and BMI between 17.3–44.0 kg/m². 74% of participants were female, and 69% were of North-West European descent. The majority (59%) of people from the general population worked. Among the general population, 47% of participants were relative “morning” types compared to 20% of the shiftwork group. Conversely, 47% of shift workers were relative “evening” types compared to 26% of the general population. Among the shift workers, the most common shift type was night shift (37%), followed by morning (34%), and afternoon (28%) shift (Table 2). As only two individuals performed split shift, their free-day data were included in analysis, but not data on split shifts. Chronotype cut-offs for our total study population were morning (MSF_SC <02:45 h), intermediate (02:45–03:40 h) and late chronotypes (MSF_SC ≥03:41 h).

Table 2. Demographic details at baseline. *A shift worker may perform one or more of: morning shift, afternoon shift, and night shift. ^#Categories for ethnicity were derived from the Australian standard classification of cultural and ethnic groups 2019 (Statistics 2019). ^{^}Calculated based on data from the CNQ by splitting MSF_SC distribution of the study population in thirds; chronotype could not be calculated for all shift workers as insufficient information was provided about work schedule.

		Total	General Population		Shift workers
Sample size		105	58		47
Age		45.33 ± 16.4 years	47.1 ± 18.5 years		43.1 ± 13.4 years
Body Mass Index (BMI)		27.3 ± 5.6 kg/m^2	26.2 ± 5.5 kg/m^2		28.6 ± 5.4 kg/m^2
Gender	Male	27 (25.7%)	9		18
	Female	78 (74.3%)	49		29
Ethnicity^#	North-West European	72 (68.6%)	36		36
	Southern and Eastern European	9 (8.6%)	6		3
	South-East Asian	7 (6.7%)	5		2
	North-East Asian	5 (4.8%)	4		1
	Southern and Central Asian	8 (7.6%)	5		3
	Peoples of the Americas:	1 (1%)	1		0
	People of the Pacific Islands	2 (1.9%)	1		1
	Aboriginal or Torres Strait Islander	1 (1%)	0		1
Chronotype^^	Morning (<2:45 h)	33	27 (47%)		6 (20%)
	Intermediate (2:45–3:40 h)	26	16 (28%)		10 (33%)
	Evening(≥3:41 h)	29	15 (26%)		14 (47%)
Work commitments			Work	23	Morning shift*	33
			School	4	Afternoon shift*	27
			Work and School	11	Night shift*	36
			Retired/unemployed	20	Split shift	2

Chronotype, sleep, and temporal patterns of eating

Using the data from the CNQ or sleep diary, the average estimated chronotype (MSF_SC) was not significantly different for shift workers (CNQ: 03:29 h, sleep diary: 03:48 h, p = 0.318) or individuals from the general population (CNQ: 03:02 h, sleep diary: 03:02 h, p = 0.908). For the general population, sleep diaries yielded later sleep time (p = 0.006), wake time (p = 0.015), and mid-sleep time (p = 0.001) on workdays and shorter sleep duration on work-free days (p = 0.04) than the CNQ (Table 3). For shift workers, sleep duration was shorter on night shifts when measured using sleep diaries compared to the CNQ (p = 0.04) (Table 3).

Table 3. Sleep variables derived from CNQ compared to sleep diaries for General Population and for shift workers on morning, afternoon, and night shifts, and their following days off. Columns show mean values from the CNQ and Bland-Altman-calculated mean differences (Diff=sleep Diary-CNQ) and the range of the 95% confidence intervals (CI range=CI upper limit – CI lower limit). Bolded values show results of paired samples t-tests comparing CNQ and diary values that were significantly different, p < 0.05. Sample sizes less than 10 are marked with an ^.

				Shift workers
	General Population			Morning shift			Afternoon shift			Night shift
	CNQ	Diff	95%CI Range	CNQ	Diff	95%CI Range	CNQ	Diff	95%CI Range	CNQ	Diff	95%CI Range
Workday
Sleep time	22:32	00:24	−01:15–02:04	22:12	00:02	−01:41–01:45	01:05	−00:05^	−01:53–01:42	08:01	00:12	−02:18–02:44
Wake time	06:28	00:13	−00:50–01:18	05:25	00:10	−01:27–01:48	09:00	−00:43^	−02:45–01:18	14:53	−00:21	−04:15–03:32
Sleep duration	07:55	−00:10	−01:55–1:34	07:12	00:08	−02:03–02:19	07:55	−00:38^	−03:05–01:48	06:52	−00:33	−02:44–01:36
Mid-sleep time	02:30	00:19	−00:46–01:24	01:48	00:06	−01:10–01:23	05:02	−00:24^	−01:52–01:04	11:27	−00:04	−03:10–03:01
Free day
Sleep time	22:55	00:08	−01:23–01:40	23:00	−00:11^	−01:27–01:04	00:00	00:02^	−03:20–03:25	23:06	00:30	−02:24–03:26
Wake time	07:28	−00:08	−01:40–01:24	07:22	00:28^	−01:47–02:44	08:52	−00:38^	−03:00–01:44	08:04	00:24	−02:15–03:05
Sleep duration	08:32	−00:16	−01:56–01:23	08:22	00:40^	−02:40–04:00	08:52	−00:40^	−03:08–01:47	08:58	−00:06	−03:47–03:35
Mid-sleep time	03:12	00:00	−01:17–01:17	03:11	00:08^	−00:36–00:53	04:26	−00:18^	−02:57–02:21	03:35	00:27	−01:38–02:34

Compared to food diaries, CNQ responses resulted in significantly greater meal and snack regularity values on night shifts and across the week for the general population (snack regularity greater only on free days) and lower total frequency on morning shifts. The CNQ also captured a significantly later time of largest eating occasion across the week for the general population and on shift-free days, later time of main meal 2 on workdays of the general population and on night shifts, earlier time of main meal 1 on afternoon shifts and first eating occasion on shift-free days, longer duration of eating window on afternoon shifts and shift-free days, and shorter duration of eating window on night shifts (Tables 4 and 5).

Table 4. Temporal patterns of eating variables derived from CNQ compared to food diaries for General Population. Columns show mean values from the CNQ, Bland-Altman-calculated mean differences (Diff = Food Diary-CNQ), and the range of the 95% confidence intervals (95%CI range = CI upper limit – CI lower limit). Bolded values show results of paired samples t-tests comparing CNQ and diary values that were significantly different, p < 0.05. n/a: Bland Altman analyses and 95% CI not calculated for regularity as it was a categorical variable.

		Workday			Free day
		CNQ	Diff	95%CI Range	CNQ	Diff	95%CI Range
Regularity (%)	Meal	75.68	−8.89	n/a	65.63	1.48	n/a
	Snack	44.44	−14.25	n/a	42.05	−14.36	n/a
Frequency (n)	Meal	2.57	0.05	−0.90–1.00	2.72	−0.05	−0.92–0.82
	Snack	2.14	−0.21	−3.07–2.66	1.91	0.21	−2.37–2.79
	Total	4.70	−0.15	−2.91–2.60	4.64	0.15	−2.41–2.72
Duration (hh:mm)	Eating window	11:13	−00:13	−03:26–03:00	10:38	−00:19	−04:40–04:01
Eating Occasions (hh:mm)	First	08:18	00:13	−02:47–03:14	08:49	00:15	−02:50–03:21
	Last	19:32	00:02	−02:25–02:30	19:32	−00:03	−03:48–03:41
	Largest	17:48	-01:30	−06:32–03:32	17:24	-01:34	−07:44–04:35
	Meal 1	09:41	−00:51	−06:15–04:32	08:51	00:00	−02:26–02:25
	Meal 2	15:00	-01:10	−06:28–04:08	14:01	−00:37	−05:42–04:28
	Meal 3	18:47	00:06	−01:05–01:17	18:36	00:05	−00:59–01:09

Table 5. Temporal patterns of eating variables derived from CNQ compared to food diaries for shift workers on morning, afternoon, and night shifts, and days off. Columns show mean values from the CNQ, Bland-Altman-calculated mean differences (Diff = Food Diary-CNQ), and the range of the 95% confidence intervals (95%CI range = CI upper limit – CI lower limit). Bolded values show results of paired samples t-tests comparing CNQ and diary values that were significantly different, p < 0.05. Sample sizes less than 10 are marked with an ^. n/a: Bland Altman analyses and 95% CI not calculated for regularity as it was a categorical variable.

		Morning shift			Afternoon shift			Night shift			Free day
		CNQ	Diff	95%CI Range	CNQ	Diff	95%CI Range	CNQ	Diff	95%CI Range	CNQ	Diff	95%CI Range
Regularity (%)	Meal	55.56	17.72	n/a	52.08	−2.67	n/a	70.65	−22.52	n/a	66.30	−10.57	n/a
	Snack	36.11	−8.33	n/a	37.50	−16.67	n/a	46.74	−17.17	n/a	36.36	−12.55	n/a
Frequency (n)	Meal	2.33	0.24	−1.40–1.87	2.33	−0.28	−1.60–1.04	2.09	0.40	−1.49–2.3	2.74	−0.26	−1.64–1.12
	Snack	2.28	0.74	−2.76–4.24	2.25	−0.37	−2.65–1.92	2.09	0.51	−2.72–3.73	1.74	0.38	−2.87–3.64
	Total	4.61	0.98	−1.97–3.92	4.58	−0.65	−3.26–1.96	4.17	0.73	−2.97–4.43	4.48	0.12	−3.04–3.29
Duration (hh:mm)	Eating Window	12:42	00:00	−05:16–5:16	12:01	-02:15	−08:20–03:50	14:03	02:02	−04:23–08:29	11:08	-01:31	−04:24–01:21
Eating Occasions (hh:mm)	First	07:54	−00:04	−04:55–04:45	10:18	01:34	−04:31–07:40	14:39	−01:37	−10:36–07:21	08:35	01:02	−01:53–03:57
	Last	20:36	−00:05	−04:17–04:07	22:20	−00:40	−04:58–03:37	04:43	00:25	−06:57–07:47	19:44	−00:24	−03:16–02:27
	Largest	18:06	−01:06	−08:48–06:34	16:47	−00:20	−06:57–06:16	19:23	00:33	−10:06–11:13	18:07	-02:31	08:18–03:16
	Meal 1	09:35	00:11	−05:01–05:24	09:47	02:25^	−03:11–08:03	15:11	−02:02	−14:13–10:08	09:20	00:21	−03:18–04:00
	Meal 2	15:36	−00:21	−03:54–03:12	15:36	01:14^	−05:13–07:41	00:00	-05:12	−19:13–08:48	14:39	−00:51	−06:02–04:20
	Meal 3	18:41	−00:03*	−01:35–01:28	19:45	−00:04^	−00:43–00:34	23:12	01:24^	−03:29–06:18	18:32	00:23	−01:11–01:59

Test re-test reliability

Sleep variables (sleep time, wake time, sleep duration, and mid-sleep time) for the general population and shift work population on all work/school/shift and respective free days showed good to excellent reliability (Table S2 and Figure 2). Chronotype also had excellent reliability for both groups, with high ICC values (General population: 0.888, 95% CI: 0.811–0.934; Shift work population: 0.887, 95% CI: 0.732–0.952).

Figure 2. (a) Test-retest reliability of sleep variables represented on the left as a box (ICC) and whiskers (upper and lower confidence intervals) plot and on the right as heat map, ranging from dark green (excellent reliability) to red (poor reliability). (b) Convergent validity of sleep variables represented on the left as a box (r_s) and whiskers (upper and lower confidence intervals) plot and on the right as heat map, ranging from dark green (large effect size) to red (small effect size). Sample sizes less than 10 are marked with a ^. GP: general population; SW: shift worker; W: workday.

In terms of temporal aspects of eating (Table S3 and Figure 3), meal and snack regularity showed fair to moderate reliability across all day types in the general and shift work population (p < 0.05). Meal, snack, and total frequency; time of first, last, and largest eating occasion; and duration of eating window all showed good to excellent reliability for the general population and shift workers on morning shift, afternoon shift, and free days (p < 0.05), with the exception of time of last eating occasion on morning shift, which did not reach statistical significance. On night shifts, only meal frequency and time of first eating occasion had good reliability. Times of main meal 1, 2, and 3 all showed good to excellent reliability across all day types in the general and shift work population (p < 0.05), although results were not statistically significant for times of main meal 3 on afternoon and night shifts (noting the small sample sizes).

Figure 3. (a) Test-retest reliability of temporal patterns of eating variables represented on the left as a box (Kappa for regularity and ICC for frequency, duration, and time) and whiskers (upper and lower confidence intervals) plot and on the right as heat map, ranging from dark green (excellent reliability) to red (poor reliability). (b) Convergent validity of temporal patterns of eating variables represented on the left as a box (r_s) and whiskers (upper and lower confidence intervals) plot and on the right as heat map, ranging from dark green (large effect size) to red (small effect size). Sample sizes less than 10 are marked with a ^. GP: general population; SW: shift worker; W: workday; FD: free day; M: morning shift; A: afternoon shift; N: night shift; EW: eating window; EO: eating occasion.

Convergent validity

Convergent validity of chronotype calculated from CNQ and sleep diaries shows good validity (general population: r_s = 0.662, p < 0.001, shift worker: r_s = 0.655, p = 0.015).

Convergent validity between the CNQ and sleep diaries is presented in Table S4 and Figure 2. For both the general and shift work population, sleep variables (sleep time, wake time, sleep duration, mid-sleep time) show good validity with large effect sizes on morning shifts, night shifts, and free days after night shifts (p < 0.05 for all except for sleep time on free days after night shift). Sleep variables for afternoon shifts and free days after morning and afternoon shifts had low to large effect sizes but were not statistically significant due to low sample sizes (n = 4–6).

Convergent validity between the CNQ and food diary is presented in Table S5 and Figure 3. Meal and snack regularity had a medium effect size for the general population and was statistically significant on work/school-free days (p < 0.05) but was not statistically significant on work/school days. Amongst shift workers, meal and snack regularity had small effect sizes across all day types and were not statistically significant.

Meal, snack, and total frequency showed good validity for the general population (p < 0.05), except for total frequency on work/school days which had poor validity (p = 0.244). For shift workers, only total frequency on morning shift had a statistically significant large effect size (p < 0.001). Otherwise, meal, snack, and total frequency had medium to large effect sizes on morning and afternoon shifts (except snack frequency on morning shifts with a small effect size), and small to medium effect sizes on night shifts and free days, but none were statistically significant.

Correlations between CNQ and food diary of time of first and last eating occasions had a medium to large effect size across the week for the general population (p < 0.05). For shift workers, only time of last eating occasion on night shift had a statistically significant large effect size for (p = 0.011); times of first and last eating occasions across the other shifts had low to medium effect sizes that were not statistically significant. Duration of eating window had a low effect size on free days of the general population (p = 0.381), but medium to large effect sizes across all other day types that were statistically significant (p < 0.05) except on afternoon shifts (p = 0.135). For the general population, times of main meal 1, 2, and 3 had medium to large effect sizes across the week (p < 0.05) apart from time of main meal 2 on free days that had a small effect size (p = 0.26). Amongst shift workers, times of main meals 1, 2, and 3 generally had medium to large effect sizes, that were statistically significant only on morning and shift-free days (p < 0.05) and insignificant on afternoon and night shifts. Time of largest eating occasion was poorly correlated across both the general and shift work populations except for night shifts, which had a medium effect size (p = 0.035).

Face validity

After the first completion of the CNQ at baseline, 43 participants provided feedback to the questionnaire: 12 were positive; 16 were neutral (reflecting on variations in their lifestyle, or general comments about questionnaire or the study); 8 were suggestions; and 7 expressed difficulties around clarity of questions/instructions, use of 24-h clock, and formatting. When the CNQ was completed for a second time after two weeks, 39 participants provided feedback: 13 were positive; 19 were neutral; 7 were suggestions. The questionnaire was rated (out of 5) to be clear (4.28 ± 0.753), understandable (4.46 ± 0.665), and have a good format (4.32 ± 0.809); results were not statistically different at two weeks (p = 0.505, 0.801, 0.586 respectively).

Discussion

The CNQ is the first questionnaire designed to capture temporal patterns of eating, chronotype, and sleep and wake times. It has shown fair to excellent test-retest reliability for the general population, while greater variability has been identified in temporal eating patterns for shift workers on night shift. Convergent validity for chronotype and sleep was strong across the study population except for sleep variables on certain shift-related days. For all participants, duration of eating window and times of main meals generally showed moderate to strong validation metrics, but time of largest eating occasion failed to reach this level. Regularity, frequency and times of first and last eating occasions generally showed moderate to strong validation metrics for the general population but not shift workers.

Convergent validity

Convergent validity for sleep and chronotype was generally acceptable except for sleep on certain shift-related days, due to limited sample sizes (n = 4–6). It is likely that validity would have been achieved with a larger sample size, as questions were derived from the validated MCTQ^Shift (Juda et al. 2013).

To our knowledge, this is the first questionnaire that attempted to validate regularity and frequency of food consumption. Meal and snack regularity showed moderate validity for the general population. Regularity in food intake times has been determined through various methods previously (McHill et al. 2020; Popp et al. 2021; Pot et al. 2014; Wang et al. 2022). Our method (Supplementary C) yields a specific percentage of time that meals fall within an hour of each other based on reported fluctuations in circadian rhythms if mealtimes are consumed >1 hr apart from day to day, which are less influenced by snack times (Manoogian et al. 2022) that tend to be smaller in portion (Leech et al. 2015b). Hence, when investigating relationships with health outcomes, emphasis should be placed on meal regularity rather than snack regularity. Similarly, in the general population, meal and snack frequency demonstrated moderate to strong validity, but not total frequency. While eating frequency is a crucial element of chrononutrition (Almoosawi et al. 2016), meal but not snack frequency was positively related to micronutrient intake and overall diet quality (Leech et al. 2016; Murakami and Livingstone 2016). Compared to meals, snacks are typically smaller in portions (Leech et al. 2015b), providing only ~ 10% of total energy intake (Potter et al. 2018); hence have disparate effects on cardiometabolic outcomes. Because of differences in energy loading, implications on diet quality and importantly, cardiometabolic health outcomes, distinguishing meal from snack frequency should be superior to total frequency.

Amongst the general population, times of first and last eating occasions had moderate to strong validation metrics for work and free days but duration of eating window showed poor validity on free days, consistent with a similar validation study (Gioia et al. 2022). This may be explained by greater variability in the time of last eating occasion on free days (Gill and Panda 2015). Furthermore, time of main meal 3 (typically last main meal of the day) had strong validity on free days while our food diary data (unreported) showed snacks occurred more often after main meal 3 than reported in the CNQ, suggesting our findings could be due to additional snacks after main meal 3. Hence, interpreting duration of eating window on free days should be conducted with the awareness that it could be a longer duration than reported. Amongst shift workers, duration of eating window had a larger effect size than times of first or last eating occasions. Hence, the CNQ can be used in future studies with shift workers to identify relationships between eating windows and health outcomes or assess compliance in time-restricted eating studies.

The time of largest eating occasion generally showed poor convergent validity except on night shifts, as previous studies have reported (Chakradeo et al. 2021; Gioia et al. 2022; Veronda et al. 2020). The concept of the largest eating occasion as captured by the CNQ was based on overall food volume which can be skewed by low energy dense foods and factors such as serving plate size or packaging (Benton 2015). More importantly, some participants indicated lunch and dinner as equally large in the questionnaire, a similar challenge to another validation study (Gioia et al. 2022). As lunch and dinner was observed to have an energy differential of 12%, differences may be indistinguishable by the lay person (Kant 2018). Our findings show mealtimes (particularly main meals 2 and 3, typically lunch and dinner) may better represent caloric load than largest mealtime.

Test-retest reliability

Our results showed good to excellent test-retest reliability for sleep, chronotype, and temporal patterns of eating variables (in the general population), similar to previous validation studies (Chakradeo et al. 2021; Reis et al. 2020; Veronda et al. 2020; Wang et al. 2023). Being the first chrononutrition questionnaire for shift workers, test-retest reliability was generally fair to excellent, except on night shifts. This is consistent with intra-individual variability in food intake patterns found between night shifts (Shaw et al. 2019) and should be considered in future data collected from the CNQ.

Comments on shift work data

We found poor convergent validity or loss of statistical significance particularly in shift workers for sleep variables on afternoon shifts and free days after morning and afternoon shifts, and eating variables including regularity, frequency, and times of first and last eating occasions. These findings may be explained by the nature of night shift work and various factors limiting data availability from shift workers and translation of results obtained.

Firstly, poor convergent validity on night shifts particularly may be due to variability in wake window. Sleep diaries revealed wake time before the first night shift tends to be in the morning while wake time before the second night shift onwards is in the evening, as sleep cycles adjust to night work. Thus, a recommended change to the CNQ would be instructing participants to respond based on the 2^nd night shift and onwards, reflecting true eating habits around a night shift (Recommended change 2).

Sample sizes within each shift were reduced as not all 56 shift workers performed all shift types, and sleep and food diary data with only one entry per shift type were excluded. Furthermore, identification of sleep/wake patterns and chronotype were based on the MCTQ, which has specific criteria that limited data usability. For instance, from sleep diaries, wake times on free days were eligible only without alarm clock use and sleep/wake data were eligible only if they occurred between two of the same shifts, which not all shift workers followed. Lastly, 7-day food diaries may have been insufficient to capture representative data from rotating shift workers. This would have particularly affected the validation of meal and snack regularity, as determining regularity of food intake on a given shift type was constrained with only two or three days of food data per shift. Should further validation of the CNQ amongst shift workers be conducted, including individuals performing limited shift types would generate more days of data per shift type. Otherwise, 14-day food diaries or multiple targeted 24-h recalls may better suit rotating shift workers.

Being the first chrononutrition questionnaire for shift workers, there is no comparison for reference. While it is postulated that the lack of statistical significance in various temporal eating aspects amongst shift workers is likely due to limited sample sizes for the above reasons, there is a chance that our findings are reflective of genuine variations in consumption patterns, affected by time and food availability, with day-to-day variability that are industry-specific (Gupta et al. 2019). Therefore, validating this questionnaire across diverse shift work industries with unique shift schedules may ascertain if our results were limited by sample size or actually reflect “chaos” typical of shift work.

Strengths and limitations

Strengths include use of prospective food and sleep diaries that reduced recall bias. Averages of sleep and food diary data from at least two days per day type were included as representative data of habitual intake. Eating occasions were specifically defined as containing ≥210 kJ/50 kcal and separated by >15 minutes (Leech et al. 2015a), similar to previous studies (Gibney et al. 1997; Hartline-Grafton et al. 2010; Ma 2003; Murakami and Livingstone 2016) and endorsed by AHA recommendations (St-Onge et al. 2017). Lastly, our questionnaire was validated amongst a multi-ethnic population representing multicultural Australia and is the first questionnaire created not only for the general population but also shift workers, capable of identifying all relevant aspects of temporal eating endorsed by experts in the field alongside chronotype that may be used by researchers, clinicians and in workplace settings.

Notably, there were several limitations to the study. Firstly, the use of sleep diaries could have introduced recall bias, and actigraphy would have served as a more objective measure of sleep (Fekedulegn et al. 2020). Secondly, inadequate prompting in the CNQ led to missing information about shift schedule and chronotype not being able to be calculated for some shift workers, which forms a recommended change to the CNQ (Recommended change 1). Thirdly, the high proportion of females prevents generalisability of findings. Lastly, rounding errors or mistakes could have resulted from participants’ unfamiliarity with 24 h time, which are common in collecting data on time (Recommended change 3) (Dashti et al. 2021; Gioia et al. 2022; Jackson et al. 2020).

Key points

Table

Recommended changes to the CNQ

Shift workers should be asked to specify the pattern of shift types and shift-free days they have within each shift cycle (based on their last shift cycle) in order to have sufficient information to calculate chronotype Temporal patterns of eating on a night shift should be captured within wake time before a night shift and sleep time after a night shift from the 2^nd night shift onwards. Include a 24-h conversion list to support participants in filling in the CNQ.

Conclusion

The CNQ is the first tool created to capture temporal patterns of eating in relation to sleep variables and chronotype. Convergent validity and reliability metrics supports its use in the general population, and in identifying chronotype, sleep, and some temporal eating aspects in the shift work population. The tool was rated as easy to use in our population of day workers, shift workers, university students, and retirees. The CNQ accurately and reliably captures chronotype and sleep parameters in both the general and shift working populations. In terms of temporal patterns of eating, across the study population, it showed moderate to strong reliability except on night shifts (perhaps due to disrupted patterns of eating), and accurate at capturing duration of eating window and times of main meals, but not time of largest eating occasion (as it tends to alternate between lunch and dinner). The other temporal aspects of eating such as meal and snack regularity and frequency, and times of first and last eating occasions were more accurately captured amongst the general population than shift workers, likely attributable to the heterogeneity of shift workers in this study.

In summary, this study validates the CNQ for the general population and shift workers; although given that it is the first of such a questionnaire for shift workers, further validation specifically of times of first and last eating occasion, and regularity and frequency of food intake in a larger sample size of homogenous shift workers will increase confidence of its use in future shift work research. In using this tool, responses should be interpreted in the context of the study population, accounting for possible external factors, such as exam period of university students or industry of shift work, which could affect timing of food intake differently. In the short term, the CNQ will be useful for quantifying temporal eating patterns and impacts on health outcomes in epidemiological studies, and in the long-term, could inform development of national dietary guidelines around optimal meal timings.

Author contribution

Yan Yin Phoi: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration; Writing – original draft; Writing – review & editing. Maxine P. Bonham: Conceptualization, Methodology, Supervision, Validation, Writing – review & editing. Michelle Rogers: Conceptualization, Methodology, Supervision, Validation, Writing – review & editing. Jill Dorrian: Conceptualization, Methodology, Supervision, Validation, Visualization, Writing – review & editing. Alison M. Coates: Conceptualization, Methodology, Resources; Supervision, Validation, Writing – review & editing.

Acknowledgments

We would like to acknowledge Jaslie Wu, who assisted in conducting clinic visits and data entry in this study as part of her Honours Project.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are available from the corresponding author, YYP, upon reasonable request.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/07420528.2024.2342937.

Additional information

Funding

This project received funding from the Australian Government Research Training Program (RTP) fee offset scholarship (funding Yan Yin Phoi).