Predicting the actigraphy-based acrophase using the Morningness–Eveningness Questionnaire (MEQ) in college students of North Italy

ABSTRACT

Actigraphy is the reference objective method to measure circadian rhythmicity. One simpler subjective approach to assess the circadian typology is the Morningness–Eveningness Questionnaire (MEQ) by Horne and Ostberg. In this study, we compared the MEQ score against the actigraphy-based circadian parameters MESOR, amplitude and acrophase in a sample of 54 students of the University of Milan in Northern Italy. MEQ and the acrophase resulted strongly and inversely associated (r = −0.84, p < 0.0001), and their relationship exhibited a clear-cut linear trend. We thus used linear regression to develop an equation enabling us to predict the value of the acrophase from the MEQ score. The parameters of the regression model were precisely estimated, with the slope of the regression line being significantly different from 0 (p < 0.0001). The best-fit linear equation was: acrophase (min) = 1238.7–5.49·MEQ, indicating that each additional point in the MEQ score corresponded to a shortening of the acrophase of approximately 5 min. The coefficient of determination, R², was 0.70. The residuals were evenly distributed and did not show any systematic pattern, thus indicating that the linear model yielded a good, balanced prediction of the acrophase throughout the range of the MEQ score. In particular, the model was able to accurately predict the mean values of the acrophase in the three chronotypes (Morning-, Neither-, and Evening-types) in which the study subjects were categorized. Both the confidence and prediction limits associated to the regression line were calculated, thus providing an assessment of the uncertainty associated with the prediction of the model. In particular, the size of the two-sided prediction limits for the acrophase was about ±100 min in the midrange of the MEQ score. Finally, k-fold cross-validation showed that both the model’s predictive ability on new data and the model’s stability to changes in the data set used for parameter estimation were good. In conclusion, the actigraphy-based acrophase can be predicted using the MEQ score in a population of college students of North Italy.

KEYWORDS:

• Chronotype
• Morningness-Eveningness Questionnaire (MEQ)
• circadian rhythm
• actigraphy
• activity levels

Declaration of interest

The authors report no conflicts of interest.

Appendix

The purpose of this appendix is to describe the linear regression diagnostics, that is, how the underlying assumptions of the linear regression model between the acrophase and the MEQ score were evaluated.

Precision of parameter estimates

Precision was represented in terms of standard error, confidence interval and coefficient of variation for each parameter. The intercept and slope estimated from the sample data (a, b) will differ from the “true” population values (α, β) due to sampling variation. The standard error (SE) is the standard deviation of the sampling distribution of the parameter. As such, it is a measure of how uncertain the parameter is: the higher the SE, the lower the precision of the parameter estimate. SE can be interpreted intuitively as an indication of how much the parameter value can be changed before a significant degradation of the quality of fit occurs. The SE values were reported by the regression software in conjunction with the best-fit estimates a and b.

The SE value is also the basis for the calculation of the confidence interval (CI) of the parameter. The lower and upper 95% confidence limits for the parameter can be derived by either subtracting from or adding to the parameter estimate approximately twice the value of SE (the exact multiplier of SE is the critical value of a t distribution with n–2 degrees of freedom). Under the assumptions of linear regression, there is a 95% chance that β lies within the 95% confidence interval of b. Similarly, there is a 95% chance that α lies within the 95% confidence interval of a.

The SE value can also be used to derive the percent coefficient of variation (CV). CV is the fractional standard error, that is, the standard error normalized to the parameter value:

(A1)

Low CV values indicate good precision, whereas CV values exceeding 100% raise suspicion about the model adequacy.

Statistical inferences concerning the parameters

Statistical inferences concerning α and β were conducted testing whether or not α = 0 and β = 0. In particular, by testing the slope of the regression line against 0, we were able to assess whether the MEQ score is a statistically significant predictor of the acrophase. Inferences concerning α and β were carried out using a t-test with n–2 degrees of freedom (the values of the t_score and the associated p-values were supplied by the statistical software).

Inference about parameter β was also performed using the analysis of variance (ANOVA). The analysis of variance approach to linear regression partitions the sum of squares and the degrees of freedom (df) associated to the Y variable (the acrophase). The breakdowns of the total sum of squares and associated degrees of freedom are displayed in the form of an ANOVA table supplied by the statistical software. The ANOVA table reports a test for the null hypothesis β = 0 using an F-ratio with (1, n–2) degrees of freedom, together with the associated p-value. This F-test is equivalent to the above-mentioned t-test (the F-ratio is the square of the t_score). The ANOVA table also provides the mean square error (MSE), that is, the sample estimate of the unknown variance (σ²). The mathematical expression for MSE is as follows:

(A2)

where resi_i is the residual associated to the i-th observation (in this study, the residual is the difference between the experimentally observed acrophase and the model-predicted acrophase). The number (n–2) at the denominator is the number of degrees of freedom associated to the residual error (n is the number of observations, and 2 is the number of parameters defining the best-fit line).

Confidence and prediction intervals around the regression line

Provided that the null hypothesis β = 0 is rejected, the best-fit regression equation can be used to make predictions. A point estimate of the acrophase can be obtained from the regression equation by plugging in the measured value of the MEQ score. Such point estimate is accompanied by a confidence interval and a prediction interval. The confidence interval measures the uncertainty about the mean value of the acrophase for a given value of the MEQ score (Kutner et al., 2004). It has a confidence level (95% was used in the present study) and has a two-sided range with a lower and upper boud. The expression of the confidence interval is given by:

(A3)

where t^n–2_95% is the 95% critical value of a t distribution with n–2 degrees of freedom and is the MEQ sample mean.

Like the confidence interval, the prediction interval had a confidence level and has a two-sided range. Unlike the confidence interval, the prediction interval predicts the spread for individual observations rather than the mean. Indeed, it measures the uncertainty associated to a new, individual observation of the acrophase for a given level of the MEQ score (Kutner et al., 2004). The expression of the prediction interval is given by:

(A4)

Intuitively, there is greater uncertainty in predicting an individual acrophase value than in predicting the mean acrophase value because the averaging that is implied in the mean reduces the variability. Therefore, as expected, for a given level of MEQ, the prediction interval for a new observation of the acrophase is wider than the confidence interval of the mean of the acrophase (by comparing Equations (A4) and (A5), one can see that the two limits are identical except for the addition of +1 under the radical of the prediction limit).

Assessing the quality of the regression

The model fit represents the ability of the model to come close to the observed data. To visually determine how the linear model fitted the experimental data, the parameter estimates a and b were used to draw the regression line superimposed to the scatterplot diagram. To provide an additional, visual insight into the degree of accuracy and precision of the model prediction across the whole range of the acrophase, we also showed the scatterdiagram between the actual measured acrophase and the model prediction based on MEQ.

To provide numerical indices of the model fit, we calculated the coefficient of determination, the root mean square error and the mean absolute error. The coefficient of determination, R², which coincides with the square of Pearson’s correlation index, is a measure of how close the data are to the fitted regression line. R² expresses the proportion of the variation of the acrophase around its mean that is explained by the regression line. This index varies between 0 and 1, and the more it approaches 1, the better the data fits. The root mean square error (RMSE) is the square root of MSE. It estimates the standard deviation of the distances between the regression line and the experimental data and is expressed in the same units as the original acrophase data (i.e. min). It is usually interpreted as the degree of scatter, or dispersion exhibited by the data points around the regression line. The mean absolute error (MAE) is another popular index to measure how close the model predictions are to the observed data. MAE is given by:

(A5)

MAE is expressed in the same units as the original acrophase data (i.e. min).

Appropriateness of the model: Analysis of the residuals

Evaluation of the model performance can be enhanced by plotting the residuals against the corresponding predicted acrophase values. Such residual plot provides an overall visual idea of whether the model adequately described the data and is useful to uncover systematic patters in the model predictions. If the model is appropriate for the data, the observed residuals should reflect the properties assumed for the random term, ε (Equation (2)). In particular, we considered the use of residuals for graphically examining whether a nonlinear trend was appreciable, whether outliers or influential values were present, whether the error terms were normally distributed and with constant variance.

If there is a nonlinear term in the data, the plot of the residuals is expected to show non-random patterns around the zero line. Outliers can be identified because they are far away from the zero line. Influential values, that is, values that have high leverage because they both have elevated distance from the regression line and are relatively far from the center of the data, may have marked influence on the regression results in the sense that, if they are removed, the parameters of the regression line change considerably. They can be spotted by visual inspection of the residual plot and, more formally, by calculating Cook’s distance (Cook & Weisberg, 1982). A data point having a large Cook’s distance indicates that the data point strongly influences the regression results. As a practical, operational guideline, Cook and Weisberg suggested that any distance greater that 1 should be closely scrutinized. Non-constant variance is demonstrated by a systematic change in the spread of the residuals as the level of the predictor variable or the fitted variable changes. The residuals were further evaluated by plotting their distribution by means of a histogram and a boxplot. These graphical approaches are particularly useful to detect asymmetries and outliers. In order to assess the normality of the residuals, the normal Quantile-Quantile (Q-Q) plot was derived. This plot is a graphical device that allows one to check the validity of the normality assumption about the distribution of the residuals. The basic idea of the Q-Q plot is to display the ordered residual against their theoretically expected values under normality. If the distribution of the residuals is close to normal, the sample and theoretical quantiles will match and their scatterplot will lie close to a straight line. In addition to assessing the approximate linearity of the points in a normal Q-Q plot, the Shapiro-Wilk test was conducted on the residuals to assess normality (a p-value less than the pre-specified significance level suggests rejection of the normality hypothesis). The assumption of constant variance of the residuals was examined using the Breusch-Pagan test (Fox & Weinsberg, 2010). This test, which is carried out by regressing of the squared residuals against the independent variable, yields a test statistic that approximately follows a χ² distribution with one degree of freedom. The value of the statistic test greater than the critical value of the χ² with one degree of freedom indicates non-constant variance.

Model validation by the k-fold cross-validation approach

The above-mentioned approaches to evaluate the model performance hinge on the same data used to identify the model (i.e., the training and the test data sets are the same). However, one would also like to ascertain how well the model can make new predictions on cases it has not already seen. To address this issue, we resorted to k-fold cross-validation (Maindonald & Braun, 2010). This technique uses the available data to mimic the process of generalizing to new data. It hinges on the idea of picking a small integer k (usually between 5 and 10) and divide the data at random into k equally sized mutually exclusive subsets (the subsets are commonly called folds). For each fold i (with i = 1, 2,…to k), the data belonging to the fold are removed from the data set, while the rest of the data are used as training set. Once the model has been identified from the training set, its performance is evaluated on the held-back data that act as the surrogate of a “new” testing set. Eventually, all the data in the dataset are used for both training and testing, and the performances are averaged across the folds. Cross-validation is often used to rank the performance of different candidate models. In this study, we subjected to cross-validation the acrophase vs. MEQ linear model described by Equation (2) with the purpose of checking whether the model was “stable,” that is, if the model could be applied to different samples from the same population without losing its predictive ability and without changing its best-fit parameters too much across samples. We used k = 6 because this allowed having a reasonably balanced amount of data in the training set (45) and in the testing set (9). The regression lines associated with the six folds were plotted on a single diagram that also displayed the data belonging to each fold. In addition, the results from the six folds were averaged to produce summary performance indices.