Assessment could demonstrate learning gains, but what is required for it to do so?

Abstract

A ready source of data to investigate learning gain is that generated normally through student assessment. That such data cannot currently be used for this purpose needs explanation and the refreshment of assessment thinking to bring it in line with thinking about standards. This opinion piece discusses what is required for this to occur.

Keywords:

• Student assessment
• learning gain
• standards

There is much discussion about the use of various instruments to measure learning gain, but we already have one that most people unfamiliar with universities would imagine could easily do the job. It is called student assessment. We regularly judge students based on the work they produce. If learning gain is to be demonstrated anywhere, surely this is the first place to look. If those that mark assessments are not concerned with learning gain, what are they doing?

The conundrum at the heart of the learning gain debate is that the most readily available measure of learning gain is existing data from student assessments. But, we can’t use assessment as it is practiced currently because ironically it is not set up to inform us about student learning. If assessment was really a process of judging performance against learning outcomes within a framework of declared standards, then there would not be a problem. That there is significant debate about how learning gain should be measured reflects badly on the adequacy of assessment. I suggest that the difficulty is not that higher education assessment is technically inadequate, though that might also be true, but that it is rooted in an unclear notion of what assessment should seek to do and how it should seek to do it.

Driven most recently by the Organization for Economic Co-operation and Development, higher education globally has moved from an era of elite intake and stability of cohorts in which assessment was used to judge students against each other, to one that is slowly becoming outcomes-based, using explicit standards against which each student is judged. This move was needed because the old norm-based regime could not, in principle, ensure that academic standards were being met and that the standards exhibited in one discipline in one university were equivalent to those in the same discipline in another university or another country. Although the new regime has been accepted in principle in most institutions now, it has not yet translated into everyday assessment practice either at the level of the institution or the course (Boud, 2017).

While different countries and institutions are at different stages of development with respect to these ideas, the following are some common characteristics of assessment today:

•	Marks for a given unit or module are not indicators of each learning outcomes
•	Marks are aggregated across different learning outcomes, so that it is impossible to discern what outcomes final marks represent
•	Pass marks are typically set in arbitrary ways that are not justified in terms of threshold learning outcomes
•	Pass marks are determined within a disciplinary culture in relation to the internal features of the course unit and its tasks; no calibration of pass marks in relation to agreed standards is typically undertaken across units. Fifty per cent, say, is assumed to have a universal shared meaning and referent across different subject matter and different levels.
•	It is common for students who do well on some learning outcomes to be compensated for poor performance on others. This can allow them to pass the unit without attaining threshold level outcomes on all learning outcomes.
•	Re-sits provide marks which are hard to match with those in earlier tests. Should the first failing mark be used to judge learning gain or the second successful one?
•	The standards to reach a pass may be higher in later course units than earlier ones.

From this observation flows the difficulty of using existing assessment data in any measure of learning gain. Increases in marks from one unit to another or one year to another cannot be taken as a gain as there is no common metric or frame of reference. Of course, there cannot be a common metric unless there is a common set of standards and accompanying criteria across a course of study, not just within a unit. The most problematic feature of current marking practices is that it is not possible to associate any reported mark with what a student can or cannot do. The meaning of the mark is not described in terms of the standards to be reached as articulated in the stated learning outcomes. Outside its immediate context, it is not clear what meaning should be attached to a mark. Marks act as obscuring devices.

Where does this leave us? While it would be desirable to fix these problems for the purposes of learning gain studies, there is a much more urgent task for assessment reform: to make marking and grading fit for the key purpose of indicating what a student has achieved. That is, showing that each student has met threshold learning outcomes and have achieved beyond them in identifiable areas.

A reform process could usefully start with the following:

•	All assessments be made in relation to explicit program learning outcomes against specific standards and criteria.
•	Standards and criteria be consistent and be calibrated across all units for each outcome.
•	The same program learning outcomes be judged (at different levels) across different units and different years

For the purposes of identification of learning gains this implies that

•	Assessments need to be directly related to explicitly articulated course/program learning outcomes, not unit or module outcomes.
•	No aggregation of marks across different learning outcomes should be permitted.
•	Marking rubrics at different levels need overlap of items to enable standards to be tracked across a program. Similarly, tests over time may need some common items to control for ease or difficulty of the test.

If this were done, then we could produce for each student a track of learning achieved for each course/program learning outcome: an obvious measure of learning gain in terms of what a course purports to achieve, and perhaps a meaningful assessment.

Disclosure statement

No potential conflict of interest was reported by the author.