Development of an instrument to measure medical students’ perceptions of the assessment environment: initial validation

Figures & data

Table 1 Response rate for the survey questionnaires

Academic year	Enrolment	No. of respondents	Complete QA	% response
Phase 1	179	166	161	92.74
Phase 2	204	181	178	88.73
Phase 3A	208	162	156	79.80
Phase 3B	203	117	116	57.64
Total	794	626	611	78.84

Fig. 1.  The scree plot.

Table 2 Summary of exploratory factor analysis results for the 20-item AEQ (n=611)

		Rotated factor loadings
Item no.	Statement	F1	F2	F3	F4
8	I received feedback on my performance for continuous assessment.	0.58
17	I received feedback on my performance for final exams.	0.76
18	Feedback from assessors about my performance is adequate.	0.77
19	Feedback is given promptly after an assessment.	0.82
27	The form of feedback I received matches the purposes of the assessments.	0.58
28	Feedback from assessors about my performance is appropriate.	0.64
29	I receive on-going feedback on my progress.	0.71
9	The assessment system encourages me to reflect on my own performance.		0.51
34	I receive feedback on my work from a range of sources (e.g., teachers, peers)		0.53
35	The feedback I received helped me to improve my learning.		0.83
36	The assessment system supports my learning.		0.61
40	The feedback I received helped me to improve my grades.		0.74
6	A description of how individual assessments and exams contribute to the total score is made known to students.			0.61
7	I received information about what is expected of me in any exam/assessment.			0.64
15	Students receive clear information about assessment.			0.76
16	I understand the assessment criteria for all the tests/exams I took.			0.75
25	Assessment criteria are clearly defined.			0.58
1	Assessment in the MBBS programme is conducted fairly.				0.82
2	Students are adequately assessed.				0.79
4	Learning outcomes are appropriately assessed.				0.71
Eigenvalues		6.54	1.93	1.47	1.40
% of variance		19.34	14.12	13.19	10.07
Total variance					57.62

Note: F1=feedback mechanism, F2=learning and performance, F3=information on assessment, F4=assessment system/procedure.

Table 3 Internal consistency of the AEQ

Factor/subscale	No. of items	Item no.	Cronbach's alpha coefficient
Feedback mechanism	7	8, 17, 18, 19, 27, 28, 29	0.87
Learning and performance	5	9, 34, 35, 36, 40	0.76
Information on assessment	5	6, 7, 15, 16, 25	0.75
Assessment system/procedure	3	1, 2, 4	0.71
Overall AEQ	20		0.89

Appendix 1

The Assessment Environment Questionnaire (AEQ)

For each statement below, please CIRCLE your level of agreement.

No	Statement	Strongly disagree [1]	Disagree [2]	Agree [3]	Strongly agree [4]
1	Assessment in the MBBS program is conducted fairly.	[1]	[2]	[3]	[4]
2	Students are adequately assessed.	[1]	[2]	[3]	[4]
3	There is sufficient time gap(s) between exam papers.	[1]	[2]	[3]	[4]
4	Learning outcomes are appropriately assessed.	[1]	[2]	[3]	[4]
5	Students’ practical skills are appropriately assessed using OSPE.	[1]	[2]	[3]	[4]
6	A description of how individual assessments and exams contribute to the total score is made known to students.	[1]	[2]	[3]	[4]
7	I received information about what is expected of me in any exam/assessment.	[1]	[2]	[3]	[4]
8	I received feedback on my performance for continuous assessment.	[1]	[2]	[3]	[4]
9	The assessment system encourages me to reflect on my own performance.	[1]	[2]	[3]	[4]
10	Feedback is given to students irrespective of performance.	[1]	[2]	[3]	[4]
11	There are too many assessments in the MBBS program.	[1]	[2]	[3]	[4]
12	Assessments are set at an appropriate level of difficulty.	[1]	[2]	[3]	[4]
13	I do NOT have enough time to reflect on my learning.	[1]	[2]	[3]	[4]
14	There is a range of assessment methods/tools being used to assess a wide variety of knowledge, skills and behaviour.	[1]	[2]	[3]	[4]
15	Students receive clear information about assessment.	[1]	[2]	[3]	[4]
16	I understand the assessment criteria for all the tests/exams I took.	[1]	[2]	[3]	[4]
17	I received feedback on my performance for final exams.	[1]	[2]	[3]	[4]
18	Feedback from assessors about my performance is adequate.	[1]	[2]	[3]	[4]
19	Feedback is given promptly after an assessment.	[1]	[2]	[3]	[4]
20	There is a balanced mix of continuous assessments and final exams.	[1]	[2]	[3]	[4]
21	Topics that have more learning outcomes are tested more extensively.	[1]	[2]	[3]	[4]
22	The assessment system gives me an idea on where I stand in comparison to my peers.	[1]	[2]	[3]	[4]
23	Assessment format chosen for each learning outcome is appropriate (e.g., MCQs for testing knowledge and OSPE/OSCE for testing skills).	[1]	[2]	[3]	[4]
24	Students’ professional behaviour is appropriately assessed in CFCS debriefings.	[1]	[2]	[3]	[4]
25	Assessment criteria are clearly defined.	[1]	[2]	[3]	[4]
26	The pass/fail border is clearly defined and made known to students.	[1]	[2]	[3]	[4]
27	The form of feedback I received matches the purposes of the assessment.	[1]	[2]	[3]	[4]
28	Feedback from assessors about my performance is appropriate.	[1]	[2]	[3]	[4]
29	I receive on-going feedback on my progress.	[1]	[2]	[3]	[4]
30	The pass/fail border is fair.	[1]	[2]	[3]	[4]
31	Learning outcomes are assessed at appropriate points during the curriculum.	[1]	[2]	[3]	[4]
32	Students’ knowledge is appropriately assessed using MCQs, SAQs and Essays format questions.	[1]	[2]	[3]	[4]
33	Students are given oral briefings prior to an exam.	[1]	[2]	[3]	[4]
34	I receive feedback on my work from a range of sources (e.g., teachers, peers).	[1]	[2]	[3]	[4]
35	The feedback I received helped me to improve my learning.	[1]	[2]	[3]	[4]
36	The assessment system supports my learning.	[1]	[2]	[3]	[4]
37	I know the requirements for me to progress from one phase to another.	[1]	[2]	[3]	[4]
38	Students’ clinical skills are appropriately assessed using OSCE, long case, short case.	[1]	[2]	[3]	[4]
39	I know the requirements for me to graduate.	[1]	[2]	[3]	[4]
40	The feedback I received helped me to improve my grades.	[1]	[2]	[3]	[4]

Appendix 2. 

Rotated component matrix^a

		Factor/component
Item No.	Statement	1	2	3	4	5	6
1	Assessment in the MBBS program is conducted fairly.	0.022	−0.038	0.699	0.066	0.122	0.042
2	Students are adequately assessed.	0.178	0.096	0.743	0.148	0.061	−0.041
3	There is sufficient time gap(s) between exam papers.	0.016	0.004	0.391	0.117	−0.214	0.475
4	Learning outcomes are appropriately assessed.	0.117	0.160	0.688	0.174	−0.001	0.030
5	Students’ practical skills are appropriately assessed using OSPE.	0.187	0.255	0.503	0.078	0.196	0.029
6	A description of how individual assessments and exams contribute to the total score is made known to students.	0.144	0.039	0.154	0.597	0.049	−0.168
7	I received information about what is expected of me in any exam/assessment.	0.216	0.069	0.199	0.522	0.157	0.145
8	I received feedback on my performance for continuous assessment.	0.565	0.208	0.184	0.288	0.027	−0.139
9	The assessment system encourages me to reflect on my own performance.	0.246	0.536	0.117	0.384	0.047	0.020
10	Feedback is given to students irrespective of performance.	0.432	0.196	0.158	0.369	−0.019	−0.260
11	There are too many assessments in the MBBS program.	0.290	−0.164	0.058	0.005	−0.171	−0.538
12	Assessments are set at an appropriate level of difficulty.	−0.011	0.246	0.219	0.198	−0.033	0.083
13	I do NOT have enough time to reflect on my learning.	0.028	−0.104	−0.061	−0.129	−0.044	−0.728
14	There is a range of assessment methods/tools being used to assess a wide variety of knowledge, skills and behaviour.	0.296	0.313	0.169	0.310	−0.003	−0.017
15	Students receive clear information about assessment.	0.246	0.213	0.090	0.677	0.058	0.183
16	I understand the assessment criteria for all the tests/exams I took.	0.178	0.158	−0.030	0.682	0.125	0.219
17	I received feedback on my performance for final exams.	0.734	0.077	0.098	0.172	0.024	−0.071
18	Feedback from assessors about my performance is adequate.	0.764	0.048	0.072	0.214	0.065	−0.040
19	Feedback is given promptly after an assessment.	0.781	0.011	0.027	0.118	0.078	−0.127
20	There is a balanced mix of continuous assessments and final exams.	0.410	−0.144	0.306	0.108	0.352	0.198
21	Topics that have more learning outcomes are tested more extensively.	0.236	0.190	0.393	−0.064	0.386	0.179
22	The assessment system gives me an idea on where I stand in comparison to my peers.	0.175	0.378	−0.013	0.251	0.149	−0.058
23	Assessment format chosen for each learning outcome is appropriate (e.g., MCQs for testing knowledge and OSPE/OSCE for testing skills).	0.224	0.104	0.337	0.109	0.393	0.275
24	Students’ professional behaviour is appropriately assessed in CFCS debriefings.	0.416	0.234	0.144	0.245	−0.002	0.005
25	Assessment criteria are clearly defined.	0.264	0.178	0.121	0.581	0.202	0.071
26	The pass/fail border is clearly defined and made known to students.	0.059	0.155	0.009	0.307	0.530	−0.099
27	The form of feedback I received matches the purposes of the assessment.	0.577	0.301	0.179	0.114	0.129	0.039
28	Feedback from assessors about my performance is appropriate.	0.664	0.197	0.036	0.080	0.277	0.039
29	I receive on-going feedback on my progress.	0.728	0.132	0.056	0.078	0.105	−0.029
30	The pass/fail border is fair.	0.111	0.024	0.080	0.248	0.553	0.125
31	Learning outcomes are assessed at appropriate points during the curriculum.	0.286	0.037	0.285	0.080	0.441	0.143
32	Students’ knowledge is appropriately assessed using MCQs, SAQs and Essays format questions.	0.138	0.243	0.480	−0.027	0.459	0.029
33	Students are given oral briefings prior to an exam.	−0.095	0.317	0.168	0.250	0.193	0.053
34	I receive feedback on my work from a range of sources (e.g., teachers, peers).	0.376	0.512	0.056	0.042	0.025	0.149
35	The feedback I received helped me to improve my learning.	0.318	0.712	0.076	0.007	0.064	0.135
36	The assessment system supports my learning.	0.121	0.629	0.191	0.221	0.139	0.082
37	I know the requirements for me to progress from one phase to another.	0.091	0.439	0.039	0.342	0.331	−0.012
38	Students’ clinical skills are appropriately assessed using OSCE, long case, short case.	−0.015	0.304	0.352	0.118	0.409	−0.174
39	I know the requirements for me to graduate.	0.014	0.157	−0.011	−0.067	0.623	−0.043
40	The feedback I received helped me to improve my grades.	0.350	0.630	0.114	−0.120	0.180	0.154
	Extraction method: Principal component analysis.
	Rotation method: Varimax with Kaiser normalisation.^a

^aRotation converged in eight iterations.