Undergraduate research internships: Veterinary students’ experiences and the relation with internship quality

Abstract

Background and aims: The learning environment of undergraduate research internships has received little attention, compared to postgraduate research training. This study investigates students’ experiences with research internships, particularly the quality of supervision, development of research skills, the intellectual and social climate, infrastructure support, and the clarity of goals and the relationship between the experiences and the quality of students’ research reports and their overall satisfaction with internships.

Method: A questionnaire (23 items, a 5-point Likert scale) was administered to 101 Year five veterinary students after completion of a research internship. Multiple linear regression analyses were conducted with quality of supervision, development of research skills, climate, infrastructure and clarity of goals as independent variables and the quality of students’ research reports and students’ overall satisfaction as dependent variables.

Results: The response rate was 79.2%. Students’ experiences are generally positive. Students’ experiences with the intellectual and social climate are significantly correlated with the quality of research reports whilst the quality of supervision is significantly correlated with both the quality of research reports and students’ overall satisfaction with the internship.

Conclusion: Both the quality of supervision and the climate are found to be crucial factors in students’ research learning and satisfaction with the internship.

Introduction

The current decline in the number of new physician-scientists (Rosenberg 1999; O’Rourke 2004) underlines the urgent need to encourage students to pursue an academic career. Several studies found evidence that medical student research programs enhance medical students’ interest in research and academic careers (Solomon et al. 2003; Houlden et al. 2004; Willenbring et al. 2008). Hoping to reap the potential benefits of undergraduate research experiences, many universities offer research internships enabling students to actively participate in research (Zier & Stagnaro-Green 2001; Houlden et al. 2004). Actively engaging in research activities has been shown to be more beneficial to student learning than listening to lectures on research (Zier & Stagnaro-Green 2001; Pretorius et al. 2003; Gafney 2005). During research internships, students typically engage in activities and acquire knowledge related to library and critical appraisal skills (principles and methods of), empirical research, statistics and reporting of research results.

There is a sharp contrast between the learning environment of the regular undergraduate curriculum and that of a research internship. Learning in undergraduate programmes is well structured and all students are exposed to the same curriculum, study materials and assessments. Research internships, by contrast, differ in subject, site and supervisors and require the use of new skills for practical (lab) work, dealing with new situations and uncertainties, developing one's own ideas and engaging in planning and time management (Gafney 2005). As with all educational activities, undergraduate research experiences should be evaluated to ascertain whether they are effective and if and where improvement is needed. Good evaluation instruments are indispensable and should be based on a comprehensive picture of all aspects that determine the quality of the learning environment. Unfortunately, there are no evaluation instruments readily available, which are designed for undergraduate research experiences since, so far, most studies have focused on postgraduate education (Ainley 2001; Marsh et al. 2002; Kiley & Mullins 2003; Pretorius et al. 2003; Ginns et al. 2004,).

One of these studies resulted in a conceptual framework consisting of factors influencing the postgraduate research experience and its success or lack of it: ‘student characteristics’ (gender, prior research experience, intellectual capacities), ‘environment’ (research culture, research education, supervisors’ assessment strategies, student/supervisor relationship), ‘processes’ (students’ discipline, motivation and expectations), ‘supervisor/mentor characteristics’ (supervisors’ expectations/experiences) and ‘outcomes’ (such as the quality of theses, completion time) (Kiley & Mullins 2003). A similar approach was used based on the Postgraduate Research Experience Questionnaire (PREQ) in evaluating PhD students’ experiences regarding supervision, clarity of goals, social and intellectual climate, infrastructure support as well as outcomes comprising generic skill development and overall satisfaction (Ginns et al. 2004). The latter outcome proved to be associated not only with the quality of supervision but also with other factors of the learning environment, such as intellectual climate, infrastructure and the frequency of student–supervisor contacts (Ginns et al. 2004). We used these two frameworks in exploring experiences and learning outcomes in undergraduate research internships, despite being aware of the potentially substantive differences between postgraduate and undergraduate research experiences in setting, duration, assessment and end products. However, we did not think that the factors determining the quality of undergraduate and postgraduate research experiences would be essentially different although some discrepancies in their importance and effects might emerge.

The setting of our study of undergraduate research experiences was the Faculty of Veterinary Medicine (FVMU), Utrecht University, the Netherlands where students attend a 6-year undergraduate programme including a compulsory 3-month research internship in Year 5. FVMU is the only veterinary faculty in the Netherlands. Students can apply for a research internship in one of the departments of FVMU, at a university in another country or at a veterinary research institute in or outside the Netherlands. They are also free to choose their area of research in basic, clinical or social/behavioural sciences. Most students are assigned two supervisors who coach and assess their work. The actual research work consists of conducting an empirical study culminating in an oral presentation and a written report structured like a paper for a scientific journal with sections on introduction/background, methods, results and discussion and conclusions. At the start of the internship, student and supervisors sign a research contract containing the student's research proposal, a brief description of the work planned, the student's motivation for undertaking this particular research, how the objectives are to be achieved, what kind of facilities are needed and a time schedule. The internship is assessed by three marks: one for practical work, one for the report, and one for the presentation, each counting for 1/3 of the end mark. The supervisors do not currently use dedicated instruments to assess these aspects of the students’ internship (Jaarsma et al. 2007).

In order to evaluate the research experiences of undergraduate students we addressed the following research questions:

1. How do undergraduate students experience a research internship, more specifically the quality of supervision, research skill development, social and intellectual climate, infrastructure support and clarity of goals?

2. How do these factors relate to the quality of students’ research reports and their overall satisfaction with the internship?

Table

Practice points

Students’ experiences with undergraduate research internships are generally positive. The perceived quality of the supervisor and the department's working climate play a prominent role in undergraduate research experiences. Supervision and intellectual and social climate are correlated with the quality of research reports and students' overall satisfaction. This questionnaire, derived from the Postgraduate Research Experience Questionnaire, seems to be valid and reliable for measuring undergraduate students’ research internship experiences. Studies of factors not addressed in this study, such as students’ motivation, and in-depth investigation of aspects influencing the intellectual and social climate of internships, are needed.

Methods

Participants and procedures

We collected data from veterinary students in Year 5 between October 2006 and June 2007. All of the 101 students completing their research internship in that period were invited to participate in the study. The aims of the study were explained to them in a letter, which also stated that participation was voluntary and data would be treated confidentially and were impossible to link to individual students. Those who consented to participate were asked to fill out a questionnaire about the internship after completing their research internships and receiving their internship marks given to them by their supervisors. The questionnaires were distributed by e-mail and as a paper hand-out. Non-responders received one email reminder at which time the questionnaire was also sent to them by post. Ethics committees at Dutch universities are not required to give approval to studies of the type reported in this article.

Measurement of students’ research internship experiences

The Australian Council for Educational Research (ACER) and the Graduate Careers Council of Australia (GCCA) designed the PREQ to evaluate the quality of postgraduate research and research masters in a variety of disciplines including veterinary science (Ainley 2001). Marsh et al. (2002) found solid psychometric properties for the PREQ at individual student level, with factor analysis revealing six underlying dimensions (28 items) of the quality of students’ experiences: 1 quality of the supervisor, 2 research skill development, 3 intellectual and social climate, 4 infrastructure and organization, 5 clarity of goals and expectations and 6 the way thesis examination was performed. One overall satisfaction item was included in the PREQ (Ainley 2001).

We pilot-tested the applicability of the PREQ to the FVMU undergraduate veterinary research internship among six veterinary students. Discussion resulted in removal of two items: ‘A good seminar programme for postgraduate students was provided’ and ‘There was appropriate financial support provided for research activities’. Additionally, the word ‘postgraduate’ was consistently replaced by ‘undergraduate’ and ‘thesis’ was replaced by ‘research report’. To assess whether the 6-factor PREQ model fitted our data, we conducted confirmatory factor analysis using Multiple Group Method1 (MGM) (Nunnally 1978; Struive 2007). Based on the item-rest correlations and on content analysis, three items were excluded and two items on ‘the way thesis examination was performed’ were moved to ‘quality of the supervisor’. This appears to be a logical step since the students in our study were assessed by their supervisor(s). Confirmatory factor analysis using MGM showed that the new 5-factor model fitted the data.

After these adjustments our validated questionnaire consisted of 23 statements (including one overall satisfaction item) inviting a response on a 5-point Likert scale (1 = ‘completely disagree’, 2 = ‘disagree’, 3 = ‘neither agree nor disagree’, 4 = ‘agree’, 5 = ‘completely agree’). Full details of the confirmatory factor analysis and its iterative process can be obtained from the first author. The items represented five underlying clusters: supervisor, skill development, climate, infrastructure and clarity. In addition, the two open answer questions from the PREQ on best aspects and what needs improvement were also included. The internal consistency of the factors varied between 0.66 and 0.84 indicating that the instrument was both valid and fairly to highly reliable. Table 1 shows the five factors and the corresponding items.

Table 1.  Scores (5-point scale) on factors and items related to the undergraduate research experience (means and standard deviations (SD) and alpha coefficients per factor)

Item	Factor	Mean	SD	α
	1. Supervisor items	3.87	0.65	0.84
1	Supervision was available when I needed it	3.66	1.14
2	My supervisor(s) made a real effort to understand difficulties I faced	4.03	0.83
3	My supervisor(s) provided additional information relevant to my topic	3.89	0.99
4	I was given good guidance in topic selection and refinement	3.78	0.89
5	My supervisor(s) provided helpful feedback on my progress	4.14	0.71
6	I received good guidance in my literature search	3.34	1.00
7	The examination process was fair	4.10	0.73
8	I was satisfied with the examination process	4.04	0.89
	2. Skill Development items	3.88	0.51	0.68
9	My research further developed my problem solving skills	3.95	0.79
10	I learned to develop my ideas and present them in my written work	4.07	0.65
11	Doing my own research helped me to developed my ability to plan my own work	3.74	0.71
12	As a result of my research I feel confident about tackling unfamiliar problems	3.75	0.72
	3. Climate items	3.64	0.91	0.85
13	The department provided opportunities for social interaction with other undergraduate students	3.59	1.19
14	I was integrated into the department's community	3.93	1.02
15	The department provided opportunities for me to be involved in the broader research culture	3.67	1.04
16	I used the research ambience in the department or faculty to stimulate my own work	3.45	0.96
	4. Infrastructure items	3.57	0.86	0.75
17	I had access to a suitable working space	3.55	1.18
18	I had good access to the technical support I needed	3.77	0.93
19	I was given good access to computing facilities and services	3.38	1.04
	5. Clarity items	3.94	0.58	0.66
20	I developed an understanding of the standard of work expected	3.93	0.67
21	I understood the required level for the research report	3.97	0.78
22	I understood the requirements of examination	3.92	0.90
23	Overall satisfaction item	4.20	0.67

Note: The percentage of observed variance is 61.58% for the 5-factor model.

Outcome measurement of the quality of students’ research internships

Two outcome measures were used to evaluate the quality of the undergraduate research internship: the overall quality of research reports and students’ general satisfaction with the quality of the internship. When examining the quality of students’ research reports in an earlier study, we found that it was best reflected in (independent) assessors’ ratings of ‘general impression of content’ on a scale from 1: very poor performance to 10: outstanding performance (Jaarsma et al. 2007). In the present study, two independent assessors used the same scale to assess the quality of research reports. The mean of their ratings was used as the first outcome measure. Inter-rater reliability was 0.77. The second outcome measure, students' overall satisfaction with the internship, was measured by item 23 of our developed questionnaire. (5-point Likert scale: 1 = ‘completely disagree’, 2 = ‘disagree’, 3 = ‘neither disagree/or agree’, 4 = ‘agree’, 5 = ‘completely agree’).

Analyses

Descriptive statistics (mean and SD) were computed for the five factors of the questionnaire. We used hierarchical (blockwise entry) multiple linear regression analysis2 to explore the relationships between the five factors (as independent variables) and the two outcome measures (as dependent variables). Some student characteristics, which we regarded as possible external variables, were controlled for by entering them into step 1 of the regression model. These variables were gender and grade point average (average test score during the first 4 years of veterinary training). The factors quality of supervision, intellectual and social climate, and infrastructure support were entered into step 2 in order of their predictive value based on results from research in postgraduate training (Ginns et al. 2004). All the independent variables were entered into step 3. In order to account for correlations between the independent variables, we present both zero order (representing simple Pearson correlation coefficients) and part (representing the unique relationship of each predictor with the outcome variables) correlations (Field 2005). The assumptions of the multiple regression model were checked (multicollinearity, Durbin–Watson, linearity and normal distribution of the residuals) (Field 2005) and found to have been met.

Since the measurement of the quality of the research reports in the present study (by use of a reliable and valid instrument, Jaarsma et al. 2007) is different from the measurement in the regular examination process conducted by the students’ supervisors, descriptive statistics were calculated for both measurements and the relationship between the scores were explored by calculating the corresponding bi-variate correlations (Pearson's product-moment correlation). The students that did not participate in the study (the so-called non-responders) were analysed for their regular examination (end)marks and compared to the students’ marks that did participate and who filled out the questionnaire (t-test). SPSS® 12.0.1 was used for all statistical analyses.

Results

Participants’ characteristics

We received 80 questionnaires aimed at seeking information on the research internship experience (response rate 79.2%). Four reports were judged as being literature reviews rather than research reports. These reports were difficult to score using the rating scale and were, therefore, excluded from the study. Two other questionnaires were also excluded because the students stated in the open comment area of the questionnaire that their supervisors had actually not assessed their research reports, but had just handed in the examination form, leaving the sample at 74 questionnaires and corresponding reports.

Of the participants, 60 (81.1%) are female and 14 (18.9%) are male, a distribution reflecting that in the FVMU student population. The internship settings are distributed as follows: 55 (74.3%) at FVMU, 18 (24.3%) at a research institute or university in another country and 2 (2.7%) at a research institute in the Netherlands.

Research internship experience

The factor scores vary between 3.57 (SD 0.86) for ‘infrastructure’ and 3.94 (SD 0.58) for ‘clarity’. The lowest and highest item scores are those for ‘I received good guidance in my literature search’ (3.34, SD 1.00) and ‘My supervisor(s) provided helpful feedback on my progress’ (4.14, SD 0.71), respectively. Both belonged to the ‘supervisor’ factor.

In total, 74 different students made 134 responses on the two open-ended questions.

The top four of responses on ‘What were the best aspects of the research internship?’ were:

1. Learning: to conduct research, write a report, deepen knowledge specific subject (28)

2. Ability to work independently/sense of ownership (18)

3. Supervision: enthusiastic, good relationship, great research group ambience (17)

4. Practical skills: clinical work, laboratory work, client contact (17).

On the question ‘What aspects of the research internship were in most need of improvement?’ the top four responses were:

1. Supervision: more time, delays in feedback, more structured and frequent meetings (18)

2. Infrastructure: better computer facilities and working place (12)

3. Organization in advance of the internship: offer of subjects, clarity of expectations (10)

4. Planning: duration of internship too short, too much practical work (9).

Relationship between undergraduate research experience and outcomes

‘General impression of content’ is rated as 7.03 (SD 1.06) on a 10-point scale. Students’ overall satisfaction with the internship is rated as 4.20 (SD 0.67) on a 5-point Likert scale.

The hierarchical multiple linear regression analyses show regression coefficients and correlations between the five factors and the two outcome measures (Table 2).

Table 2.  Regression analyses for the two dependent variables (quality of research reports and overall satisfaction with research internship) and the five independent variables (supervisor, skill development, climate, infrastructure, clarity of goals) and the possible external variables i.e. grade point averages (GPA) and gender. Unstandardized coefficients (B), standardized coefficients (β), standard errors and zero-order and part correlations are presented

Quality of research report^a						Overall satisfaction^b
	B	SE B	β	Zero-order correlations	Part correlations	B	SE B	β	Zero-order correlations	Part correlations
Step 1
Constant	2.620	2.282				3.008	1.506
GPA Years 1–4	0.689	0.321	0.253	0.265	0.252	0.166	0.212	0.096	0.095	0.095
Gender	−0.213	0.309	−0.081	−0.118	−0.083	0.009	0.204	0.005	−0.009	0.005
Step 2
Constant	1.151	2.134				1.915	1.272
GPA Years 1–4	0.572	0.301	0.210	0.265	0.229	0.007	0.180	0.004	0.095	0.005
Gender	−0.301	0.290	−0.115	−0.118	−0.128	−0.128	0.173	−0.077	−0.009	−0.092
Supervisor	0.491	0.227	0.303*	0.377	0.258	0.563	0.136	0.547**	0.567	0.458
Climate	0.295	0.146	0.258*	0.360	0.244	0.113	0.087	0.155	0.398	0.159
Infrastructure	−0.167	0.159	−0.137	0.139	−0.130	−0.068	0.095	−0.088	0.263	−0.089
Step 3
Constant	1.484	2.243				1.070	1.271
GPA Years 1–4	0.585	0.309	0.214	0.265	0.232	−0.017	0.175	−0.010	0.095	−0.013
Gender	−0.303	0.295	−0.115	−0.118	−0.128	−0.120	0.167	−0.072	−0.009	−0.090
Supervisor	0.530	0.248	0.328*	0.377	0.260	0.459	0.141	0.445**	0.567	0.380
Climate	0.316	0.158	0.276*	0.360	0.243	0.070	0.090	0.097	0.398	0.098
Infrastructure	−0.148	0.173	−0.121	0.139	−0.107	−0.123	0.098	−0.157	0.263	−0.155
Skill development	−0.120	0.274	−0.059	0.160	−0.055	0.264	0.155	0.204	0.449	0.209
Clarity	−0.062	0.270	−0.035	0.142	−0.029	0.187	0.153	0.165	0.450	0.152

Notes: ^aR² = 0.08 for Step 1; ΔR² = 0.18 for Step 2 (p < 0.005); ΔR² = 0.01 for Step 3 *p < 0.05, **p < 0.001.

^bR² = 0.09 for Step 1; ΔR² = 0.34 for Step 2 (p < 0.001); ΔR² = 0.07 for Step 3 (p < 0.05).

The model in step 2 showed the best fit to the data: there is a significant relationship between ‘overall quality of the research report’ on the one hand and on the other hand ‘quality of supervision’ (β 0.303) and ‘intellectual and social climate’ (β 0.258). This model explains 26% of the variance. ‘Quality of supervision’ (0.547) is also significantly related to ‘students’ overall satisfaction with the research internship’. Forty-two percent of the variance is explained by this model.

Agreement between supervisors’ report scores and global ratings by independent assessors

The average supervisors’ report score is 7.62 (SD 0.79) on a 10-point scale. No significant linear correlation was found for the supervisors’ report scores and the global ratings by the independent assessors.

On average, the non-responding students received lower end marks by their supervisors (mean 7.72, SD 0.77) than the participating students (mean 7.26, SD 0.70). This difference was significant (t = 2.40; df = 92; p < 0.05).

Conclusions and discussion

Our questionnaire study provides insight into how students experience undergraduate research internships and whether and how these experiences are related to the outcomes of the internship in terms of the quality of research reports and students’ overall satisfaction. The relatively high factor scores suggest that students’ experiences are generally positive. The relatively low scores on ‘guidance in literature searches’ and the ‘department's research ambience’ confirm Ainley's finding of low agreement among respondents on these PREQ items (Ainley 2001). The quality of supervision was mentioned as one of the best aspect of the internship as well as the issue most in need of improvement, which coincides with the findings of the correlations between the quality of students’ experiences and the two outcome measures (quality of research reports and overall satisfaction with the internship) suggesting that the success of research internships hinges on the quality of supervision. This key factor appears to be associated both with the quality of research reports and with overall satisfaction, whereas intellectual and social climate of the internship setting is only associated with the quality of research reports but not with overall satisfaction. This is in line with the relationship reported by Ginns and colleagues between PhD students’ satisfaction with the quality of supervision and the quality of the prevailing intellectual and social climate in the department (Ginns et al. 2004). However, our study did not replicate their finding that infrastructure support is a significant predictor of successful outcome of research experiences (Ginns et al. 2004).

Given the strong resemblance between learning during research internships and apprenticeship learning (Gafney 2005), it seems only to be expected that the perceived quality of the supervisor and the department's working climate play a prominent role in both undergraduate and postgraduate research experiences. There are also differences, however. A PhD project usually takes 3 years or more to complete, whereas most undergraduate research experiences do not extend beyond 3 months. This may explain why infrastructure support appears to feature less prominently in undergraduate experiences compared to postgraduate ones.

Our finding that supervision is key to the quality of undergraduate research internships implies that supervisors should be keenly aware of the impact of their role as well as that of the department's working climate. This is particularly relevant since students with more positive research experiences may be more inclined to pursue a career in research later on (Rosenberg 1999; Pretorius et al. 2003; Solomon et al. 2003; Houlden et al. 2004; O’Rourke 2004; Willenbring et al. 2008). The quality of supervision may be enhanced by offering training to supervisors to improve their guidance skills. Another recommendation aimed at enhancing the quality of research internships, based on the impact of social climate, is for departments to facilitate interactions between students and to have groups of students simultaneously undertake internships within one department.

The low correlations between the supervisors’ report scores and the global (independent) ratings may be explained by the low variance of both variables. Another explanation may be that the supervisors assessed different aspects to those measured by the independent rating as discussed more elaborate in an earlier study (Jaarsma et al. 2007).

The robustness of our findings is supported by the use of outcome measures based not only on students’ subjective experiences but also on two independent ratings of their research reports using a valid and reliable rating scale. Most other studies have relied exclusively on students’ self-reports as outcome measures (Ginns et al. 2004). This brings us to the first limitation of this study, the fact that the internship experiences were investigated solely through students’ self reports. A second limitation is that the questionnaire has been validated in a veterinary curriculum and not in a medical curriculum. Although these curricula have many commonalities, it is important that validation should be repeated in other settings. A third limitation is that each supervisor was rated by one student only, although several student ratings per supervisor would have been preferable. As a consequence it is not possible to rule out that the results are influenced by dependency among cases due to students sharing the same supervisor. Finally, although the response rate was high (79.2%), the difference in supervisors’ end marks between the participating students and the non-responding students needs to be considered. Since students were asked to fill out the questionnaire after receiving their end mark, we cannot entirely rule out the influence of the end mark on students’ perception of the research experience. However, Marsh and Roche (1997) have shown that student ratings are relatively unaffected by biases such as grading leniency.

It is a strength of this study that it has yielded a validated questionnaire for evaluating the quality of undergraduate research experiences, whereas so far there was only a validated questionnaire for postgraduate research experiences.

Finally, we want to discuss suggestions for further study of this topic. The 26–42% explained variance of the two models suggests that there may be other variables affecting undergraduate research learning. For example, students’ motivation for undertaking a research internship (Kiley & Mullins 2003). Further studies should examine in greater detail the factors affecting research experiences established in this study as well as try to identify other explanatory variables. Another research angle could be to investigate factors influencing the intellectual and social climate of internship settings, for instance by interviewing undergraduates and supervisors. It would also be of interest to further investigate differences between supervisors, which would require ratings by more than 1 student. This study leaves a gap in our understanding of learning during research internships because it did not measure students’ performance during internship. Learning outcomes of students’ activities, such as technical or laboratory skills, developing their own ideas, work planning and scientific presentation skills, could be explored to complete the picture of learning from research experiences during undergraduate training presented in this study. Other stakeholders, such as research supervisors and institutional research directors could be involved for more longitudinal research that follows-up career choices of these undergraduate students and investigates whether the internship experiences decreased for instance the threshold to pursue an academic career.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Notes

Additional information

Notes on contributors

Debbie A. D. C. Jaarsma

DEBBIE (A.D.C.) JAARSMA, DVM, PhD is a veterinarian and assistant professor at the Faculty of Veterinary Medicine, Utrecht University, The Netherlands.

Arno M. M. Muijtjens

ARNO (A.M.M.) MUIJTJENS, MSc, PhD, is a statistician and associate professor in the Department of Educational Development and Research, Maastricht University, The Netherlands.

Diana H. J. M. Dolmans

DIANA (D.H.J.M.) DOLMANS, MSc, PhD, is an educational psychologist and associate professor in the Department of Educational Development and Research, Maastricht University, The Netherlands.

EVA M. Schuurmans

EVA (E.M.) SCHUURMANS, BSc, is a veterinary student at the Faculty of Veterinary Medicine, Utrecht University, The Netherlands.

Peter Van Beukelen

PETER VAN BEUKELEN, DVM, PhD, is a professor in quality improvement in veterinary medical education at the Faculty of Veterinary Medicine, Utrecht University, The Netherlands.

Albert J. J. A. Scherpbier

ALBERT (A.J.J.A.) SCHERPBIER, MD, PhD, is a professor in quality improvement in medical education and scientific director of the Institute for Education, Faculty Health, Medicine and Life Sciences, Maastricht University, The Netherlands.

Notes

1.  MGM reports on the percentage of observed variance that is explained by the tested item grouping and on the item-rest correlations after correction for test length and self-correlation. Based on the correlations of items with the sum scores of the clusters of items, it is possible to analyse whether items are in ‘the right cluster’, i.e. show high correlations with the pre-assigned cluster and low correlations with other clusters.

2.  In hierarchical regression independent variables are based on past work and the researcher decides in which order to enter these variables into the model. As a general rule, known independent variables (from other research) should be entered into the model first in order of their importance in predicting the outcome. After known variables have been entered, the researcher can add any new variables into the model (Field 2005).