Learning modules: transparency in assessment in law courses

ABSTRACT

This article argues in favour of more transparency in the assessment of law courses in the modern law school context in which it is increasingly common for law students to share notes. In 2020, the author introduced three types of asynchronous exam-focused exercises to the Land Law course at the University of Auckland. First, guided exercises: students are guided through a series of questions to answer a problem in bite-sized chunks. Secondly, modelling exercises: students plan an answer to a past exam problem and use a video in which the lecturer plans an answer while narrating their thought processes to reflect on their own plan. Thirdly, example exercises: students review previous exam answers against the relevant rubrics to understand what distinguishes answers at each grade. The article investigates student attitudes about: which exam-focused exercise is the most useful component of a law course; whether students prefer if an exam-focused exercise helps them to understand relevant content or develop legal reasoning skills; which exam-focused exercise is more helpful for (a) understanding relevant content and (b) developing legal reasoning skills; and whether students prefer modelling by the lecturer in pre-recorded videos, the lecturer in in-person lectures or a tutor in in-person tutorials.

KEYWORDS:

• Problem-based learning
• blended learning
• flipping the classroom

1. Introduction

In 2020, the author introduced three asynchronous exam-focused activities (guided exercises, modelling exercises and example exercises)¹ to the Land Law course² at the University of Auckland Faculty of Law³ to increase transparency about how the exam for the course is marked.⁴ First, guided exercises, for each of which students are guided through a series of multiple-choice, short-answer and long-answer questions to answer a problem in bite-sized chunks, before discussing the problem and reviewing their answers in a post-activity tutorial. Each guided exercise is worth 1% of the final mark for the course, which is enough to incentivise most students to complete the exercise before attending the tutorial.⁵ Secondly, modelling exercises, for each of which students plan an answer to a past exam problem, watch a video of the lecturer planning an answer while narrating their thought processes, and use the lecturer’s plan and commentary to reflect on their own plan. Thirdly, example exercises, for each of which students are provided with a past exam problem on a topic testable in the current course and previous students’ exam answers to the problem written in exam conditions (examples) at each grade, which students review against the rubric to understand what distinguishes responses at each grade. The modelling and example exercises are for exam preparation purposes and are not worth any marks. Students access all the exercises through the Land Law course site on the Canvas online learning management system.

There were two key drivers for developing exam-focused exercises.⁶ First, the author found that Land Law students in the third part of the four-part LLB degree had difficulties applying the law in answers to problems and were unconfident about doing so. Secondly, the law student body at the University of Auckland created an online open-access Google Drive folder called the Notes Bank in which students upload their notes and examples, which other students can download and use as a basis for their own notes to prepare for assessments.

This article argues in favour of more transparency about how answers to problems are marked to help fill students’ knowledge and resource gaps. Parts 2 and 3 outline the drivers for developing the exam-focused exercises: to increase transparency about how answers to problems are marked; and to respond to the Notes Bank. Part 4 outlines the study methodology. The study investigated student attitudes about the exam-focused exercises, in particular: which exam-focused exercise is the most useful component of a law course; whether students prefer if an exam-focused exercise helps them to understand relevant content or develop legal reasoning skills; which exam-focused exercise is better for (a) understanding relevant content and (b) developing legal reasoning skills; and whether students prefer modelling to be by the lecturer in pre-recorded videos, the lecturer in in-person lectures or a tutor in in-person tutorials. Part 5 presents and discusses the results.

2. Transparency in assessment

Transparency in assessment refers to student awareness of the “purpose of the assessment”, the “assessment criteria” and “what is expected of them”.⁷ Law exams are usually assessed by a combination of essays and problems. The problems are law-specific and assess legal reasoning skills, which are important for practice as lawyers. The law degree is an academic and a professional qualification.⁸

Legal reasoning skills are not only applicable in law school. Legal reasoning skills form the basis for life-long learning in the legal profession.⁹ Legal practice requires not only a substantive knowledge of the law, but also knowledge of how to apply that law in solving legal problems.¹⁰ However those skills cannot be taught by passively listening to lectures and taking notes. In a society rapidly advancing in technology, employers are becoming more demanding about law graduates’ competencies.¹¹ Yet there has long existed a disjunct between employers’ expectations and what students are actually taught at university.¹² Rohan Havelock suggests the optimum way to develop those skills is by attempting to solve problems “with appropriate facilitation and guidance”.¹³ According to Bloom’s taxonomy of learning, the act of “doing” engages a higher level of legal application and analysis.¹⁴

Legal reasoning skills also benefit students who do not enter the legal profession. As of 2017, only around 57% of law graduates in New Zealand were admitted as barristers and solicitors. Of those admitted graduates, only around 60% enter practice as lawyers.¹⁵ Additionally, many of those transition into non-legal work during their career.¹⁶ A legal education provides other benefits, such as developing analytical,¹⁷ recall¹⁸ and collaborative skills.¹⁹ So, while this article sometimes focuses on the vocational benefits of legal reasoning skills, it is recognised that these skills can have many different benefits for graduates in whatever career they pursue.

In light of the long-term importance of legal reasoning skills, it may be appropriate for institutions to re-evaluate how much emphasis is placed on developing legal reasoning skills relative to understanding relevant content.

Part 2 reviews the literature on each exercise the author developed to increase transparency about how answers to problems are marked: guided exercises; modelling exercises; and example exercises.

2.1. Guided exercises

A key skill developed in legal education is applying law to facts.²⁰ Law schools are increasingly adopting some form of problem-based learning (PBL) to ensure that students learn not only what the law is but how to apply it to reach solutions of their own.²¹ In Land Law at the University of Auckland, the mid-course legal opinion (15%) and final exam (70%) include problem questions.²²

In each learning module, students complete a guided exercise, for each of which they are guided through a series of multiple-choice, short-answer and long-answer questions to answer a problem in bite-sized chunks.²³ A sample problem on title, interests and priorities is provided in Appendix A and the guided exercise for that problem is featured in Appendix B. After submitting the guided exercise, students attend a tutorial on the same problem where they can clarify their understanding of the relevant content, discuss the problem, and collaborate with their tutor and classmates to develop a sound answer. The guided exercise is designed to help students become more experienced and confident in applying law to facts on their own and understand what will be expected of them when they answer similar problem-based questions in the opinion and exam.

The literature includes many examples of courses that set a problem-based exercise before a tutorial, akin to a guided exercise before a post-exercise tutorial. York Law School students are placed into student law firms where they collaborate to identify and analyse issues arising from a realistic legal problem.²⁴ Students are then assigned a specific legal issue to address in self-directed study.²⁵ Students reconvene with their peers the following week to discuss their findings, with the guidance and support of a tutor, to deepen discussion.²⁶

Similarly, Maastricht Law School students are provided with a unitbook with a series of problems, and some guidance on potential issues and the relevant law. Students prepare in advance, before attending a tutorial in which they work in small groups to analyse the problems.²⁷ Students take turns to chair the tutorials and are responsible for researching and reading further to deepen their understanding.²⁸ The authors concluded that the approach could create a powerful learning environment, particularly when students engage with real-world problems. However, they also acknowledged that some students have difficulty adjusting to the problem-based learning approach, especially when they have limited prior knowledge.²⁹

Laura Hemker, Susanne Narciss and Claudia Prescher investigated a practice that used problem-based cases (realistic scenarios) in the training of student teachers.³⁰ Students discuss the cases in groups at tutorials, with teachers facilitating discussion and providing progressively less guidance as students work through the cases. The cases highlight “dilemma situations” typical of the professional teaching experience and draw on topics from multiple subjects to develop an interdisciplinary awareness of topics that student teachers study.³¹ The authors reported that students welcomed being able to work on real-world problems and expected positive outcomes from problem-based learning. At the same time, several students did not perceive their prior knowledge as adequate for problem-based learning and often students’ responses to the real-world problems were not real-world-ready solutions. Accordingly, the authors suggested that more structure and supervision might increase student satisfaction and learning.³²

Similarly, David Lopes Cardozo, Laurie Raymond and Benjamin White studied a compulsory second-year medical course that divided students into small groups of seven students.³³ Each week, students were assigned a responsibility (such as conducting research or chairing discussions) and met three times to review a different case (a medical scenario requiring a diagnosis and treatment plan). Students prepared for the teacher-facilitated student-led sessions in advance, reading the case and preparing their diagnosis, before working as a group to reach a diagnosis and formulate a treatment plan for the fictional patient.³⁴ The cases were designed to enhance understanding of relevant content and develop research skills, while practising the discipline-relevant clinical skills of diagnosis and formulating a treatment plan.³⁵ The authors reported that the students seemed comfortable with the structured approach, found it efficient, appreciated having defined goals, understood what was expected of them, were more motivated than before, were better prepared than before, and participated more due to an increased level of accountability.³⁶

Some commonalities emerge from this review of pre-tutorial exercises: each pre-tutorial exercise contributes to a student-led collaborative learning exercise, emphasises discipline-relevant practical skills (often with a focus on real-world problems), and sets up a plan for further research, learning and review. However, students with limited prior knowledge may have difficulty with or be unconfident about problem-based learning. In the Land Law guided exercise, students are expected not to develop a robust answer to the real-world problems on their own, but to prepare sufficiently to be able to participate in a tutor-facilitated tutorial³⁷ where they discuss and develop answers to consecutive bite-sized chunks of the problem, which may be challenged or endorsed by the tutor – enhancing their understanding of relevant content and developing legal reasoning skills in the process.

2.2. Modelling exercises

Modelling is an instructional strategy in which teachers demonstrate a new concept or approach to learning, and students learn by observing. The case for modelling is premised on its efficiency in helping students learn new skills and knowledge.³⁸ Social biologists have recently advanced the notion that humans evolved to transfer knowledge to, and receive knowledge from, one another.³⁹ Thus, human behaviour is learnt via observing others, using the information gained by observation as a guide for future actions.⁴⁰ Indeed, Abdullahi Salisu and Emmanuel Ransom claim that learning can be difficult (and potentially hazardous) when students have to rely solely on the effects of their own actions to inform them about what to do.⁴¹ The benefits of enhancing understanding and developing skills are especially pronounced when the person modelling verbalises their thought processes while engaging in the activity. Combining verbal and non-verbal modelling gains and holds attention, and helps to produce lasting improvements in cognitive skills.⁴² Teachers can effectively model a range of skills, including those helpful when responding to a legal problem, such as contemplating and discussing contestable issues that may not have a correct answer.

Scholars have emphasised the need to deploy modelling appropriately. In the context of modelling music performances, Warren Haston goes as far as claiming that “[w]hen used inappropriately, [modelling] can be a crutch that actually prevents students from learning”.⁴³ One aspect of correct modelling is ensuring that the teacher demonstrates the skill or knowledge in a way that students can analyse, interpret and recall.⁴⁴ Novice learners need considerable help; thus, a good model does not simply reproduce the desired skill or knowledge.⁴⁵ Where the difference between the teacher exhibiting the skill or knowledge and the learner is great, providing progressive steps and ensuring that students have sufficient time to assimilate information gradually are important.⁴⁶ If sufficient levels of instructional support are not offered in the course of modelling, negative effects on the students may materialise as they will often feel overloaded, distressed and demoralised.⁴⁷ John Hattie and Gregory CR Yates sum up the keys to success:⁴⁸

When a knowledgeable teacher provides clear modelling, and clear direct instructional language, the conditions for strong learning are set to positive. Providing, (a) attention is captured and (b) learner capacities are not exceeded [emphasis added], then powerful learning can occur in the classroom.

These findings are mirrored by the research of Nancy Maynes, Lynn Julien-Schultz and Cilla Dunn, which examined how teachers in a Canadian school implemented modelling in practice.⁴⁹ The study found that while teachers used modelling often enough, they practised it in a way that was inconsistent with the research literature’s conception by providing some students with support in a manner that deprived others.⁵⁰ Researchers noted that immediately following the modelling to the whole class, the teachers would work one-on-one, or in small groups, with the students who needed further assistance.⁵¹ While this was good proactive teaching, it was also problematic because the rest of the students were now engaging in unsupported practice.⁵² Those students had no access to immediate supervision or error correction, because the teacher’s attention was elsewhere.⁵³ Strong scaffolding for a few students meant no scaffolding for many.⁵⁴

The use of modelling as an instructional strategy has not been without its critics. Some scholars contend that providing highly explicit procedural explanations (via the use of verbal and non-verbal modelling) can interfere with a student’s development of adaptive, flexible knowledge of the sort that is valued in problem solving.⁵⁵ They contend that students will learn better, and with greater understanding, if they discover the information for themselves, rather than being told the information.⁵⁶ The basis of their reasoning is that discovering something by oneself entails a deep and thoughtful process that one is unlikely to forget.⁵⁷

Many scholars have, however, rejected this criticism of modelling. Hattie and Yates, for instance, claim that “[t]here is no contradiction between active information processing and learning from direct instruction”, and that “[t]he idea that secure knowledge emanates automatically from personal discovery is flawed”.⁵⁸ They counter that the additional load imposed by needing to explore and find things out for oneself can detract from our ability to assimilate the information uncovered.⁵⁹ Studies suggest personal discovery in a PBL setting can lead to the “redundancy effect”, which is where additional material provided to assist in learning actually interferes with it.⁶⁰ Rather, modelling imposes necessary limits on the scope of information students may study, facilitating a more effective and efficient learning process.⁶¹ Further, there is a link between students’ abilities to effectively research and interpret information and their pre-existing knowledge, suggesting that higher-achieving students tend to research and interpret information more effectively than their lower-achieving peers.⁶² Thus, the “individual discovery” of PBL can cut off students from factors that may orient the knowledge-building process on secure, socially shared foundations.⁶³ Ultimately, the absence of teachers’ guidance means that the conditions for learning are lacking, and impediments to knowledge assimilation are imposed, as students tend to revert to exercising whatever prior knowledge can be activated quickly.⁶⁴ Instead, teachers might utilise modelling as an instructional strategy, which can direct the high level of non-productive mental effort to more profitable forms of knowledge-building.⁶⁵

Overall, while many scholars raise concerns about the use of modelling as an instructional strategy, others have pushed back against those concerns and have argued that their views are outdated. The proper use of modelling may provide the key to better and longer lasting learning in the classroom by helping students to improve their cognitive skills. Implemented correctly, modelling can serve as a valuable tool to teach students not only knowledge, but also how to approach analysis of contestable issues.

2.3. Example exercises

The use of examples in legal education is hotly debated. While many applaud its ability to clarify assessment criteria and standards, others raise concerns that their use may lead to plagiarism and a reduction in students’ creative thinking. However, many scholars have countered the latter concerns by arguing that the case against examples can be addressed to maximise the net benefits to students.

The case for examples centres on the ability to clarify assessment criteria and standards before the student is to hand in the assessment. Contemporary education has moved towards the express articulation of such criteria and standards in order to provide a transparent measure of student performance and achievement.⁶⁶ This has often manifested itself in rubrics which describe the characteristics of a piece of work at each grade level.⁶⁷ However, these rubrics have largely been unsuccessful in communicating meaningful knowledge on assessment criteria and standards – in other words, what students have to do to get A, B or C.⁶⁸ Difficulties will always arise to some extent as verbal descriptions of standards are always somewhat vague or fuzzy.⁶⁹ D Royce Sadler notes:⁷⁰

Fuzzy standards are implied, [for instance], when it is said that a dissertation is highly original, that a solution to a problem is elegant, that a student’s understanding of entropy is thorough… In each of these cases, the standards are designated by linguistic terms … A fuzzy standard cannot, therefore, be defined into existence.

Examples can help translate this fuzzy and vague rubric into tangible and real standards. They show students what sort of writing corresponds with each grade, demonstrating the criteria and standards. The “picture” in the form of the example can be of much more value to a student than the “thousand words” of criteria and standards could ever be.⁷¹ Examples are highly desired by students, especially since feedback on assessments is usually only provided after assessments have been marked and is, thus, perceived by many students as being too little too late to be of much use to them.⁷²

This use of examples has thus been linked with a corresponding increase in student grades. Newlyn and Spencer, for instance, provided examples to more than 2500 students undertaking an introductory law unit at an Australian university and found a corresponding increase in student marks.⁷³ In particular, there was a large increase in the number of students achieving at the highest grade possible.⁷⁴

Nonetheless, some scholars have criticised the use of examples in legal education. First, the construction of examples is time-consuming and labour-intensive. Someone needs to acquire the examples and staff may need to further craft the answer to reflect the criteria. If the example is past student work, student consent will need to be obtained and there may be a formal process for ethical clearance. Transcribing student work may be necessary for clarity, and annotations might be added to explain why a student received a certain grade and give the example meaning. Overall, Newlyn notes that “the entire process is likely to take a matter of months before it can be finalised”.⁷⁵

Secondly, many contend that providing examples may suppress creative thinking – a skill vital in law – and encourage plagiarism. In particular, exemplars (model answers) may encourage students to think there is only one correct way to approach a particular question or that the exemplar represents an objectively best approach.⁷⁶ Consequently, there is a temptation for students to mimic examples without properly considering whether the approach in the example is appropriate for answering the problem at hand.⁷⁷ The use of examples may hinder the development of analytical and writing skills, which may undermine the ability to practise law in the real world, where applicable models and templates will not always exist.⁷⁸ In any case, students generally consider examples to be less helpful than resources that directly engage their critical thinking and legal reasoning skills.⁷⁹ Students would prefer to learn the process of arriving at the correct answer instead of a formula of what to do in a set of given facts.⁸⁰ Challenging students to acquire the information for themselves will likely create a more meaningful, memorable and enjoyable experience.⁸¹

While the concerns of these scholars are justified, many have noted that adjustments can be made to address their concerns while retaining the benefits of their use. First, many have emphasised the need to ensure that the example provided is not too close to the assessment to be completed by the student. This, in turn, could address issues of plagiarism. As Newlyn notes:⁸²

Consider the situation where an educator wished to teach the skill of essay writing in an English literature course. If that course examined the writings of Brontë or Austin, then the provided [example] could be from Dickens or Conrad.

However, Patricia Montana suggests that the example should nevertheless relate to a context familiar to students.⁸³ When students understand the law relating to the facts, they can better understand the structural choices made by the writer. Where they do not, they are unable to see how the writer chose their organisational structure.

Secondly, multiple examples of good work should be provided to counter any student perception that there is only one good way to approach a legal question.⁸⁴

Thirdly, students should be provided with a proper explanation as to the strengths and weaknesses of an example in relation to the criteria and standards.⁸⁵ The whole point of the example is to clarify the meaningless criteria and standards against which students are being assessed. Therefore, without an adequate explanation of how the example fares in relation to it, the benefits of providing the example are not maximised. As Montana notes:⁸⁶

Professors must treat [examples] as opportunities for the students to learn and practice important analytical and writing skills, not as [illustrations] of good legal writing. This requires that professors meaningfully integrate the models into their classroom instruction and dialogue; they cannot simply distribute them without discussion or refer to them only casually.

An example’s strengths and weaknesses might be indicated with margin annotations.⁸⁷ Alternatively, researchers have found that students who engaged in a workshop where they marked the example and discussed their reasoning for their mark found the example more useful than those who did not.⁸⁸ Whichever way it is done, the example should be properly engaged with to maximise its usefulness.

In summary, while many scholars raise legitimate concerns about the use of examples, many others have pointed out that these concerns can be adequately addressed while retaining the benefits of their use. Therefore, the proper use of examples may provide students with a valuable tool in their legal education.

3. The Notes Bank

Part 3 explores how the Notes Bank came into being, the reasons why many academics oppose such databases, and how the author learned to stop worrying and love the Notes Bank.⁸⁹

In 2017, two University of Auckland law students established the Notes Bank – an open-access Google Drive for students’ notes and examples.⁹⁰ The Notes Bank is managed by an elected student who advertises for students to submit their notes and examples for law courses, vets the submissions for quality (generally only accepting work that scored 75% or higher) and uploads the accepted submissions to the Notes Bank for other students to access and download.

The Notes Bank poses several issues. The notes vary in depth and style – some are designed for use in an open book exam and are, therefore, brief, while others verge on lecture transcripts supplemented by excerpts from relevant readings. Problematically, students can largely rely on these notes instead of creating their own. In doing so, students miss an opportunity to exercise the encoding function of notetaking (translating lecture content into one’s own words and organising that material), which tends to lead to better academic results.⁹¹ Instead, reliance on other students’ notes incentivises students to skip lectures or attend lectures as passive observers.

Reliance on the Notes Bank for open book exams impedes and subverts the development of proper study and exam technique. Students forfeit the process of condensing their lecture notes into exam notes as part of their revision and blindly adopt the exam notes of a previous year’s student instead.⁹² In doing so, students fail to engage the storage function of notetaking, which is the process of reviewing notes by consolidating, re-reading and self-testing – which helps to maximise information retention.⁹³ These students may also expose themselves to error,⁹⁴ given the variation in lecturers, course content and relevant law (legislation, case law and policy) from year to year.

Frequent reference to the Notes Bank may also affect students’ academic self-efficacy – that is, their belief that they can successfully achieve at a designated level on an academic task. Instead of generating their own ideas and hypotheses, students benchmark their work against the Notes Bank’s student-curated peer-produced notes and examples. In a study conducted in an Israeli teachers’ college where the students had access to peer-produced study materials like those on the Notes Bank, results indicated that 50% of undergraduates believed they needed those materials to succeed.⁹⁵ Students who exalt the knowledge of previous students⁹⁶ featured on the Notes Bank may have low self-efficacy. This is reinforced by the way the Notes Bank publishes the overall law grade given to the student by whom the notes were written.

Further, the Notes Bank raises concerns about the quality of students graduating with a law degree. For students with performance-avoidance goals – that is, a desire to avoid performing worse than their peers – the Notes Bank provides a way to avoid exam failure without putting in much effort.⁹⁷ Studies suggest that those with performance-avoidance goals and a low grade point average (GPA) are more likely to rely on shared peer-produced notes and examples.⁹⁸ As such, the Notes Bank increases the risk that some students will graduate with a law degree and enter a profession for which they do not possess the required skills.

Finally, the Notes Bank may facilitate more frequent academic misconduct. This has been a particular issue during and following the COVID-19 lockdowns, with many exams being taken on exam invigilation software and assessments checked by a similarity detection service.⁹⁹ In the first semester of 2024, one half of law exams at the University of Auckland are planned as remote online non-invigilated exams, and another quarter are planned as in-person invigilated exams where students could copy and paste.¹⁰⁰ The remaining quarter of exams are piloting in-person invigilated exams with a lockdown browser.¹⁰¹ Apart from the latter format, where software locks the screen, students are able to copy and paste information from the Notes Bank directly into exam answers. While exam invigilation software and similarity detection services can monitor plagiarism, the temptation to copy and paste from the Notes Bank may be greater in the high-pressure exam environment. Research shows that, while knowledge-sharing between students can be beneficial when formally facilitated in a course by the lecturer, there is a greater risk for students to engage in academic misconduct when it occurs in an informal and unmanaged environment.¹⁰² Although software that locks students’ screens responds to plagiarism concerns (as well as concerns about generative artificial intelligence use), it may be problematic for open book exams.¹⁰³

Today’s law students face many pressures and may have poor wellbeing. A 2019 report on the experiences of fifth year law students in New Zealand found that 63% of respondents reported a mental health disorder.¹⁰⁴ Similarly, in a study of law students in the United States, United Kingdom and Australia, Graham Ferris found that law students’ mental health was generally poor.¹⁰⁵ Law students at the University of Auckland likely fit this general trend, particularly following two years of pandemic disruption.¹⁰⁶ In that time, the cost of living has also increased in New Zealand, forcing many students to take up part-time or full-time employment.¹⁰⁷

While there are many concerns about the Notes Bank, it is unlikely to go away. Even if the university were to try to shut it down, the sharing would likely move to another platform – potentially a platform that is less commonly known and accessible by all students, which would be less equitable. Generative artificial intelligence is already posing similar issues,¹⁰⁸ which are only likely to increase as the technology develops.

Rather than resist transparency, the author wanted the Land Law course to be more transparent about how students would be assessed. The author decided to explore ways in which teachers can provide students more guidance and transparency, with hopes of alleviating the level of reliance that students place on unofficial sources like the Notes Bank for assessment preparation.

Increased transparency has several potential benefits. In light of increasing pressures on students, transparent assessment could reduce student stress by improving their learning and wellbeing, and increasing their trust in teaching staff. Students who trust that their teachers are providing them with the tools and advice to succeed stand to apply those tools more effectively.¹⁰⁹ Anecdotally, many students are sceptical of their lecturer’s advice that developing their legal reasoning skills is the most useful preparation for the exam, instead spending most of their preparation time understanding the relevant content as much as possible – relying on the Notes Bank’s notes and examples uncritically. The author hypothesises that students who trust their lecturer will tend to invest more time into developing their legal reasoning skills and so score higher in assessments, which should improve their wellbeing.

4. Methodology

Part 4 outlines the study methodology. In 2020 and 2021, students were surveyed at the start of the course (SOC) and the end of the course but before exams (EOC).¹¹⁰ The SOC surveys investigated students’ expectations, while the EOC surveys investigated students’ reflections. The surveys included five-point Likert scales (strongly agree = 5, agree = 4, neither agree nor disagree = 3, disagree = 2, and strongly disagree = 1), open-ended boxes for students to elaborate, and other open-ended questions. Students’ responses to the open-ended questions were coded and grouped by their themes and sentiments, and summarised in Part 5 to supplement the findings from students’ responses to the Likert-scale questions.

The nature of the data limits the choices of summary statistics suitable for describing the data. The article reports medians and quartiles (25th, 50th and 75th percentiles) to describe the central tendency and distribution of the relevant data. Quartiles and medians are visualised in boxplots in Parts 5.1, 5.3 and 5.4. Means and standard deviations are not suitable summary statistics for the ordinal data because the intervals between adjacent options are not necessarily equal.¹¹¹ The ordinal nature of the data also limits the choices for inferential statistics suitable for generalising the sample findings. The analysis involves three inferential statistical tests.

The Friedman test and the post-hoc Wilcoxon signed-rank test were used for the analysis in Parts 5.1 and 5.3 to compare students’ responses within each of the four surveys. The Friedman test was used as a non-parametric alternative to the Repeated-Measures ANOVA. The Friedman test detects differences in multiple related groups without assuming the normal distribution of data. The survey data are ordinal and positively skewed, so non-parametric tests are valid. P-values lower than 0.05 are deemed to be statistically significant. Where a statistically significant difference was identified, post-hoc pairwise comparisons were conducted to identify which pair of groups originated the statistically significant difference. Post-hoc pairwise comparisons were conducted using the Wilcoxon signed-rank test, which compares paired measurements and determines whether there is a statistically significant difference between them. In most cases, no post-hoc pairwise comparison was conducted as the difference between the variables was too small.

Spearman’s rank-order correlation was used for the analysis in Part 5.2 to determine the relationships between the responses to a statement about an exercise’s usefulness and the responses to the other statements about the exercise. The correlation coefficient value ranges from −1 to 1. The closer the correlation coefficient is to 1, the stronger the correlation; and the closer it is to 0, the weaker the correlation. If the value is negative, it is in an inverse relationship with the usefulness variable. All values listed in the tables section 5.2 (Tables 1, 2 and 3) are statistically significant, with a p-value less than 0.001. No factors are negatively correlated with usefulness. Correlation does not indicate causation. However, correlation is a good measure of how much two variables are related to each other. A correlation greater than 0.75 is generally considered to be a strong correlation between two variables.

Table 1. The reported numbers illustrate whether students’ responses to the statement “I think [the exercise is] a useful component of this law course” correlate more with the usefulness for improving understanding of relevant content or the usefulness for improving legal reasoning skills. The highest value of each row, which is bolded and underscored, indicates a stronger correlation with the statement for each setting.

		Content	Skills
Guided exercises	SOC 2020	0.868	0.815
	EOC 2020	0.772	0.775
	SOC 2021	0.734	0.618
	EOC 2021	0.574	0.655
Modelling exercises	SOC 2020	0.779	0.778
	EOC 2020	0.773	0.754
	SOC 2021	0.561	0.612
	EOC 2021	0.529	0.602
Example exercises	SOC 2020	0.832	0.796
	EOC 2020	0.735	0.761
	SOC 2021	0.762	0.854
	EOC 2021	0.791	0.828

Table 2. The reported numbers illustrate which exam-focused exercise tends to correlate the most with students’ responses to the statement: “I think [the exercise is] a useful component of this law course”. The highest value of each row, which is bolded and underscored, indicates a stronger correlation with the statement for each setting.

		Guided exercises	Example exercises	Modelling exercises
Content	2020 SOC	0.868	0.832	0.779
	2020 EOC	0.772	0.735	0.773
	2021 SOC	0.734	0.762	0.561
	2021 EOC	0.574	0.791	0.529
Skills	2020 SOC	0.815	0.796	0.778
	2020 EOC	0.775	0.761	0.754
	2021 SOC	0.618	0.854	0.612
	2021 EOC	0.655	0.828	0.602

Table 3. The reported numbers illustrate whether correlation of students’ responses to the statement “I think [the exercise is] a useful component of this law course” with the usefulness for improving understanding of relevant content and the usefulness for improving legal reasoning skills changed at EOC. The highest value of each row, which is bolded and underscored, indicates a stronger correlation with the statement for each setting.

			Content	Skills
SOC	Guided exercises	2020	0.868	0.815
		2021	0.734	0.618
	Modelling exercises	2020	0.779	0.778
		2021	0.561	0.612
	Example exercises	2020	0.832	0.796
		2021	0.762	0.845
EOC	Guided exercises	2020	0.772	0.775
		2021	0.574	0.655
	Modelling exercises	2020	0.773	0.754
		2021	0.529	0.602
	Example exercises	2020	0.735	0.761
		2021	0.791	0.828

5. Results and discussion

The study investigates student attitudes about the exam-focused exercises, in particular: which exam-focused exercise is the most useful component of a law course; whether students prefer if an exam-focused exercise helps them to understand relevant content or develop legal reasoning skills; which exam-focused exercise is better for (a) understanding relevant content and (b) developing legal reasoning skills; and whether students prefer modelling to be by the lecturer in pre-recorded videos, the lecturer in in-person lectures or a tutor in in-person tutorials. Part 5 presents and discusses the results.

At SOC 2020, 59 out of 361 (16.34%) students participated. At EOC 2020, 117 out of 361 (32.41%) students participated. At SOC 2021, 103 out of 435 (23.68%) students participated. At EOC 2021, 136 out of 435 (31.26%) students participated.¹¹²

5.1. Do students consider guided exercises, modelling exercises or example exercises to be more useful components of a law course?

The author was interested in whether students considered guided exercises, modelling exercises or example exercises to be more useful components of a law course. Students generally considered that each exam-focused exercise was useful as a component of a law course (Figure 1).

Figure 1. The boxplots compare students’ perceptions about the usefulness of each exam-focused exercise at SOC and EOC in 2020 and 2021.

5.1.1. SOC

In 2020, at SOC, the medians for the guided exercises and example exercises (medians = 5 and 5, respectively) are observed to be greater than that for the modelling exercises (median = 4). In 2021, at SOC, the median for the guided exercises (median = 5) is observed to be greater than those for the modelling exercises and example exercises (medians = 4 and 4, respectively).

In 2020, at SOC, only one or two students did not expect the guided exercises and example exercises to be useful.¹¹³ By contrast, a quarter of students did not believe that the modelling exercises would be useful. It appears that the modelling exercises were expected to be a less useful component of the course. The statistical tests confirm this. The result of the Friedman test indicates that the observed difference is not due to chance alone (chi-squared = 10.396, df = 2, p = 0.006). Further, the pairwise comparisons between the three components conducted using the Wilcoxon signed-rank test indicate that the modelling exercises were expected to be less useful, compared with the guided exercises and example exercises (p = 0.013 and p = 0.016, respectively).

In the open-ended responses, some students expected the guided exercises would provide a good opportunity to practise skills and receive feedback; and expected the example exercises would elucidate the length, structure and depth of good answers, help to clarify marking expectations, and provide an opportunity to check their understanding of content. However, one student suggested the example exercises would be more beneficial if they included annotations identifying the strengths and weaknesses of the answers.¹¹⁴ Students were generally less supportive of the modelling videos: one expected they would not have time to engage with the modelling exercises given the guided exercises would take priority as they are awarded marks; one indicated the modelling exercises seemed to provide content that was surplus to what students needed to know (the teachers may have insufficiently explained their utility);¹¹⁵ and one suggested that the availability of modelling exercises might give the impression that tutorials were redundant (which would not be the case given tutorials engaged with the guided exercises). However, some students expected the modelling exercises would be useful, particularly for providing clarity about the standard of work required in the exam.¹¹⁶ Further, another student commended the teachers for developing the modelling exercises which they believed demonstrated effort and care for their students.

In 2021, at SOC, despite the difference in medians, a Friedman test showed no statistically significant difference in the usefulness of the three exam-focused exercises, meaning that the observed difference is due to chance alone (chi-squared = 5.610, df = 2, p = 0.061). In the open-ended responses, at SOC, students appreciated the teachers’ effort in creating and providing the exercises, noted their innovativeness, and recommended their provision in other law courses. Some students believed the guided exercises would be helpful for revising course content, and supporting them to consider and draft an answer to the problem before the tutorial, which would help them to prepare for the tutorial.¹¹⁷ In none of the SOC or EOC surveys did students report that their prior knowledge was inadequate to work through the guided exercises¹¹⁸ or that more guidance would benefit their learning.¹¹⁹ However, some students were concerned that the time and effort spent completing the guided exercises would be disproportionate to the marks awarded and would take priority over engagement with the modelling exercises and example exercises.¹²⁰ Some students considered modelling and example exercises useful because they would help students develop legal reasoning skills and do so at their own pace.

5.1.2. EOC

In 2020 and 2021, at EOC, students did not seem to consider one component to be more useful than the others: the median agreement levels for the relevant survey questions are the same (median = 5) and the distributional shapes of their attitudes are almost identical (Figure 1). Specifically, three-quarters of the respondents agreed or strongly agreed that each of the three exam-focused exercises was useful (lower-quarter = 4) and the remainder had no strong negative or positive feelings towards each exercise (minimum = 3). The results of the Friedman test for EOC 2020 and EOC 2021 confirmed that there is no statistically significant difference between the perceived usefulness of the three exercises at the end of the course (EOC 2020: chi-squared = 3.657, df = 2, p = 0.161; EOC 2021: chi-squared = 2.923, df = 2, p = 0.232).

Students’ open-ended responses indicate more favourable attitudes towards the modelling exercises at EOC than at SOC. Some students commented it is more helpful to go through exam answers with commentary and discussion (as in the modelling exercises) rather than simply being provided them (as in the example exercises),¹²¹ and that the commentary is better at helping students to develop skills in answering exam questions¹²² and explore multiple valid approaches to a good answer. These comments indicate that students come to appreciate the value in receiving expert guidance as to the process of how to answer a question. By contrast, example exercises provide more limited guidance and require students to piece together an approach to answering questions themselves. Example exercises require students to evaluate several examples independently to gauge those multiple valid approaches, which students were less confident doing on their own – for example, because the examples may contain student errors under exam conditions.¹²³ Further, some students mentioned that exposure to examples may complicate their ability to develop novel approaches to answering exam questions.¹²⁴

Ultimately, more favourable attitudes towards modelling exercises and less favourable attitudes towards example exercises meant that students at EOC no longer considered example exercises more useful than modelling exercises. As indicated, this attitude change likely comes from the greater degree of guidance students receive from modelling exercises. Nonetheless, students found example exercises useful altogether (median = 5) and many commented that they were useful, particularly because they were provided and organised by the course teachers, rather than an unofficial source, like the Notes Bank.

The literature endorses various features of the example exercises. For example, the examples exercises feature previous students’ answers on topics that are testable in the current course, meaning that students should understand the relevant law and be able to understand the writers’ choices,¹²⁵ but the issues focused on differ from those tested in the current course.¹²⁶ Further, the example exercises provide students with dozens of answers at the A, B, C and D grades and, thus, multiple ways of answering the problem, demonstrating there is not only one good way to approach a legal question.¹²⁷ Nonetheless, the literature provides ideas about how to improve the example exercises. First, the teacher could annotate the examples to indicate strengths and weaknesses, perhaps focusing on what the student could have done to score a higher grade, and upload the annotated versions to the learning management system.¹²⁸ Secondly, it could be beneficial for students to mark examples in the tutorial and discuss their reasoning for the mark provided.¹²⁹ Finally, some students identified that the students’ handwriting could be difficult to read at times; this is, of course, something that markers need to contend with, and the importance of writing legibly is perhaps a lesson in itself! With the increased use of online typed exams, typed example exercises, which should be easier for students to read, are likely to become standard.

5.2. Do students prefer if an exam-focused exercise helps them to understand relevant content or develop legal reasoning skills?

Students generally considered that each exam-focused exercise helped them to understand relevant content and develop legal reasoning skills (Figures 2 and 3).¹³⁰ The author was interested in whether students preferred if an exam-focused exercise helped them to understand relevant content or develop legal reasoning skills. Students were asked at SOC and EOC how helpful they believed guided exercises, modelling exercises and example exercises would be/were for enhancing their understanding of relevant content and developing legal reasoning skills. Spearman’s rank-order correlation was used to analyse how students’ responses to those questions correlated with their responses to the statement: “I think [the exercise is] a useful component of this law course”.

Figure 2. The boxplots compare students’ perceptions towards the usefulness of each exam-focused exercise for understanding relevant content at SOC and EOC in 2020 and 2021.

Figure 3. The boxplots compare students’ perceptions towards the usefulness of each exam-focused exercise for developing legal reasoning skills at SOC and EOC in 2020 and 2021.

High usefulness ranks tended to coincide more with high ranks for developing legal reasoning skills, rather than with high ranks for understanding relevant content (Table 1), meaning that students considered exercises that helped them to develop legal reasoning skills to be more useful for a law course than exercises that helped them to enhance their understanding of the relevant content. This coincidence is particularly so for example exercises because their high usefulness ranks coincide more with high ranks for developing legal reasoning skills than with high ranks for understanding relevant content in three of the four surveys (Table 1). Across all four surveys, most respondents who elaborated on their attitudes to example exercises explained that they were particularly helpful for providing examples of how to write exam answers and understanding markers’ expectations of exam answers.

Example exercises seemed to be particularly helpful for understanding relevant content and developing legal reasoning skills, compared with the other two exercises (Table 2). In two out of four surveys, the correlation of the perceived usefulness of an exercise with its usefulness for understanding relevant content is the strongest when the exercise is an example exercise. Similarly, in two out of four surveys, the correlation of the perceived usefulness of an exercise with its usefulness for developing legal reasoning skills is the strongest for example exercises. After all, example exercises provide more realistic expectations of exam answers than modelling exercises and, thus, are valuable and unique as a source of learning material.

While the main purpose of the guided exercises is to develop legal reasoning skills, many students value the component more when they develop a better understanding of the relevant content. It makes sense that many students at the SOC place greater emphasis on understanding relevant content, beginning at a position of no content knowledge. At the start of a semester, teaching focuses more on substantive content and the application of that content tends to have greater relevance closer to exams.¹³¹ Students may also consider understanding content to be more valuable because of the strong relationship between the way that students learn and the nature of assessments.¹³² Erik Driessen and Cees Van Der Vleuten suggest that for many students where a course contains a high-stakes final exam, “the examination programme becomes the actual curriculum”, thus fostering a content-driven approach to learning.¹³³ While most students would likely want to possess strong legal reasoning skills, many enter law school expecting those skills to be explicitly taught to them as part of the core curriculum.¹³⁴ Thus, it is understandable why many students initially value exercises that enhance understanding of relevant content over exercises that develop legal reasoning skills. Regardless of student attitudes, given the emphasis in the literature on developing skills for practice as lawyers, it is important that lecturers provide exercises to help students develop their legal reasoning skills.

This speaks to a broader tension. A balance must be struck between acknowledging students’ preferences and teaching them the requisite skills. Cathy Driscoll and David Wicks caution that “if students are able to negotiate curriculum and evaluation based on their perceived needs as customers, then universities may possibly erode the quality to which students were attracted in the first place”.¹³⁵ Increasingly, tertiary institutions are becoming consumeristic, with teaching staff experiencing pressure to design courses in ways that satisfy students at the expense of academic rigour.¹³⁶ The implications of this are far reaching: it compromises the professionalism of teachers; it fosters an approach to learning that is degree-focused, rather than laying the foundation for life-long learning; and it defers to students’ individual priorities over what the informed teacher believes are their longer-term best interests.¹³⁷ Students’ responses need to be interpreted in this context – to yield completely to students’ preferences would risk compromising the quality of their legal education and competence in legal practice.¹³⁸

Samuel Messick notes that for an assessment to be considered a “worthwhile educational experience serving to motivate and direct learning”, it needs to be meaningful to the students.¹³⁹ At SOC, students believed that understanding relevant content was more meaningful to their legal education than developing legal reasoning skills. Stronger correlations were found between the usefulness of understanding relevant content and the overall usefulness ranks than between the usefulness of developing legal reasoning skills and the overall useful ranks at SOC (Table 3). However, at EOC, the opposite was observed: students believed that developing legal reasoning skills was more meaningful to their legal education than understanding relevant content. This could indicate that the closer students are to the exam, the more they value developing legal reasoning skills, which bear the most on good exam performance.

Lecturers might make students more positive about attempting guided exercises by emphasising that the guided exercises enhance understanding of relevant content. When an exercise makes students believe they are improving their understanding of relevant content while developing legal reasoning skills, they may become more motivated to engage with the exercise. Alternatively, lecturers might directly address misconceptions about legal education by explaining the value of developing legal reasoning skills. Ultimately, lecturers should be transparent about how assessments are marked, which, in the case of problem questions, should emphasise the criticality of strong legal reasoning skills to attain a high grade.

Lecturers should also clearly communicate to students the course’s learning outcomes, and organise activities and assessments that engender for students the feeling that they are achieving those outcomes. With a clearer understanding of the course’s learning outcomes and how their performance will be assessed, students are more likely to consider enhancing their understanding of relevant content and developing legal reasoning skills to each be meaningful pursuits.¹⁴⁰

5.3. Do students consider guided exercises, modelling exercises or example exercises to be more helpful for (a) understanding relevant content and (b) developing legal reasoning skills?

Next, the author investigated whether students considered guided exercises, modelling exercises or example exercises to be more useful for (a) understanding relevant content and (b) developing legal reasoning skills. The Friedman test indicated no significant difference between the exercises for either understanding relevant content or developing legal reasoning skills, except for EOC 2021.

5.3.1. Understanding relevant content

Students were asked at SOC and EOC how helpful they considered guided exercises, modelling exercises and example exercises to be for enhancing their understanding of relevant content. The median scores for the three responses were compared to investigate what exercise students considered to be most helpful for enhancing their understanding of relevant content.

In 2020, at SOC, the median of modelling exercises is observed to be smaller than those of guided exercises and example exercises (medians = 4, 5 and 5, respectively) (Figure 2), suggesting that students expected guided exercises and example exercises to be more useful than modelling exercises for understanding relevant content. However, the observed difference is not statistically significant (chi-squared = 1.14, df = 2, p = 0.566). Therefore, we cannot say that any of the three exercises were expected to be more useful than the others for understanding relevant content.

In 2020, at EOC, students did not seem to consider any one of the exercises to be more useful than another for understanding relevant content. The medians of the three exercises are the same (median = 5) and the distributional shapes are almost identical (Figure 2). The results of a Friedman test confirmed that there was, indeed, no statistically significant difference (chi-squared = 1.571, df = 2, p = 0.456). Similarly, in 2021, at SOC, despite having observed differences in the medians of the three exercises (medians = 5, 4 and 4 for guided exercises, modelling exercises and example exercises, respectively), the differences were not statistically significant (chi-squared = 4.248, df = 2, p = 0.120).

In 2021, at EOC, the results of a Friedman test showed statistically significant differences between the three exercises (chi-squared = 8.829, df = 2, p = 0.0121). However, post-hoc pairwise comparisons do not indicate statistically significant differences within any pair of exercises: between guided exercises and modelling exercises p = 0.064; between guided exercises and example exercises p = 1.000; and between modelling exercises and example exercises p = 0.052. Therefore, students did not consider any one of the exercises to be more useful than the others for understanding relevant content.

Students generally considered that each exam-focused exercise helped them to understand relevant content (Figure 2). At SOC, students mentioned that they expected each exercise to help them enhance their understanding of relevant content. Some students mentioned they expected the guided exercises, in particular, to help them understand relevant content, monitor whether they have understood each topic, receive early, frequent feedback, and be accountable for keeping up with the course. One student also believed that the guided exercises would be an “effective way of gaining a richer understanding of a topic as they involve a [great] degree of reasoning”. Students generally considered modelling exercises helpful when they attempted the question beforehand. However, they found the videos long and, at times, difficult to digest; it is unclear from the responses, but this might be because students are not focused on exams early in the course and, so, find exam preparation disengaging at that time. One student mentioned that example exercises would be useful as they would give students an idea about what they are expected to write within a given timeframe,¹⁴¹ what aspects to focus on and the best structure for an exam answer.¹⁴² However, another student foresaw that it could be difficult to understand past student exam answers without the marker’s annotations.¹⁴³

At EOC, students considered guided exercises helpful for understanding relevant content. One student commented that guided exercises “help a lot with my understanding. They consolidate what we have learned and help to explain what areas of the law hold more weight, and generally how to apply them”. As to modelling exercises, a student commented that modelling exercises provided “a perfect balance of different types of learning”¹⁴⁴ and another commented that they added variety to the course in a way that complemented lectures and tutorials. Further, modelling exercises facilitate a more effective and efficient learning process,¹⁴⁵ while levelling the playing field for high and low achieving students.¹⁴⁶ One student considered example exercises helpful for learning how to distil large amounts of course content into an exam answer.¹⁴⁷

5.3.2. Legal reasoning skills

Students were asked at SOC and EOC how helpful they considered guided exercises, modelling exercises and example exercises to be for developing legal reasoning skills. The median scores for the three questions were compared to investigate what exercise students considered to be most helpful.

In 2020, at the SOC, the medians and the distributional shapes of the three exercises are slightly different (Figure 3). However, the observed differences are not statistically different (chi-squared = 0.747, df = 2, p = 0.688). In 2020, at EOC, there is no observed difference in the medians or the distributional shape of the three exercises (Figure 3). The results of the Friedman test showed that there is no statistically significant difference (chi-squared = 3.187, df = 2, p = 0.203). Similarly, in 2021, at SOC, there is no observed or statistically significant difference between the expected usefulness of the three exercises for developing students’ legal reasoning skills (chi-squared = 4.439, df = 2, p = 0.109).

However, in 2021, at EOC, students considered guided exercises less useful for developing reasoning skills than the other two exercise types. The median of guided exercises is smaller than those of modelling exercises and example exercises (medians = 4, 5 and 5, respectively). A Friedman test indicated statistically significant differences existed between the three exercises (chi-squared = 11.446, df = 2, p = 0.003). The results of post-hoc pairwise comparisons showed that it is statistically significant that guided exercises were considered less useful for developing legal reasoning skills than modelling exercises (p = 0.001), although there were no statistically significant differences between example exercises and the other types of exercises (guided exercise: p = 1.000; modelling exercises: p = 0.052).

In the open-ended responses, students generally considered that each exam-focused exercise would help or had helped them to develop legal reasoning skills (Figure 3). At SOC, a student commented that breaking the guided exercise down into each IRAC step was logical and made it easier to work through the exercise.¹⁴⁸ Students found modelling exercises helpful for developing legal application skills, and one student recommended that similar exercises be developed in other courses. Students did not reflect that the availability of examples might lead to an overreliance on them or negative long-term consequences for their professional performance.¹⁴⁹ Rather, students considered example exercises helpful for developing strategies for answering problems under time pressure.¹⁵⁰

At EOC, students identified the guided exercises as particularly helpful in developing the skill of applying law to facts by providing a scaffolded opportunity to practise the skill step by step, and receive feedback in the tutorial on the approach they took. Students found modelling exercises helpful in similar ways, although they generally considered them to be more detailed and less engaging. Some students commented that example exercises were limited in helping students to develop legal skills because students are left to work out the best approach to an answer by surveying multiple other students’ answers without the lecturer’s guidance,¹⁵¹ which creates an additional load on students.¹⁵²

5.4. Do students prefer modelling by the lecturer in pre-recorded videos, the lecturer in lectures or a tutor in tutorials?

The author was also interested in what students considered to be the best forum for modelling. In the SOC and EOC surveys, students were asked: whether they prefer lecturers modelling legal reasoning skills in online videos rather than lecturers modelling those skills in lectures; whether they prefer lecturers modelling legal reasoning skills in online videos rather than tutors modelling those skills in tutorials; and whether they prefer lecturers modelling legal reasoning skills in lectures rather than tutors modelling those skills in tutorials. The median scores for these questions were compared to investigate which forum students preferred for modelling.

In 2021, students generally preferred modelling by lecturers over modelling by tutors. At both SOC and EOC, half of the class preferred modelling by lecturers in either online videos or lectures over modelling by tutors in tutorials (median = 4 for the two “Videos over Tutorials” and two “Lectures over Tutorials” boxplots) (Figure 4). Only a quarter of the class preferred the opposite (lower quartile = 3 for the two “Videos over Tutorials” and two “Lectures over Tutorials” boxplots). So, the 2021 cohort preferred modelling by lecturers, whether through an online or in-person forum, over modelling by tutors.

Figure 4. The boxplots compare the central tendency and the distribution of students’ preference of the forum for modelling at SOC and EOC in 2020 and 2021.

However, the cohort seemed to prefer an online forum over an in-person one, provided the lecturers conducted the modelling. Half of the class preferred modelling by lecturers in online videos over modelling in lectures (median = 4 for the two “Videos over Lectures” boxplots) and a quarter of the class showed no preference (lower quartile = 3 for the two “Videos over Lectures” boxplots). So, students preferred an asynchronised, online forum for modelling by lecturers.¹⁵³

In 2020, students’ preferences were less consistent throughout the year. At SOC, half of the class preferred lecturers modelling in lectures over tutors modelling in tutorials, and a quarter of the class had no preference between modelling by lecturers or tutors (median = upper quartile = 4, lower quartile = 3). When asked to compare modelling in online videos with modelling in lectures and modelling in tutorials, respectively, only a quarter of the class indicated a preference for modelling in online videos (upper quartile = 4 for the “Videos over Lectures” and “Videos over Tutorials” boxplots). In the open-ended responses, one student suggested that modelling in pre-recorded videos especially may provide overly prescriptive approaches to answering problems, which the literature suggests can interfere with students’ development of problem solving.¹⁵⁴ Half of the class preferred modelling in lectures or tutorials (median = 3 for the “Videos over Lectures” and “Videos over Tutorials” boxplots). So, students preferred an in-person, synchronous forum over an online, asynchronous forum.

Students’ preferences shifted at EOC. Students generally indicated a preference for modelling by lecturers over modelling by tutorials, whether through an online forum or not (median = 4 for the “Videos over Tutorials” and “Lectures over Tutorials” boxplots). The literature suggests that students may experience difficulty with modelling when the difference between the teacher exhibiting the skill or knowledge and the learner is great.¹⁵⁵ The students seemed to prefer modelling by the model with the greater skill or knowledge – the lecturer as opposed to a tutor. Such a preference may suggest that the lecturers provided “progressive steps”¹⁵⁶ and ensured that students had “sufficient time to assimilate information gradually”, which is supported by no students indicating they felt “overloaded”, “distressed” or “demoralised” by the modelling process.¹⁵⁷ Whereas at SOC students preferred in-person modelling, at EOC, at least half of the students did not express a preference for a synchronous, in-person forum when asked explicitly to compare modelling by lecturers in online videos and lectures (lower quartile = median = 3).¹⁵⁸ In 2020, at EOC, it seems that the factor that ultimately determined students’ preference is who conducted the modelling.

While modelling in asynchronous videos might not be able to provide instructional support that is as tailored as modelling in a synchronous forum, modelling in videos seems to play a useful role in combination with synchronous teaching.¹⁵⁹ One student emphasised that modelling may be valuable in each forum (lectures, tutorials and videos), with each serving a different purpose – for example, modelling in lectures for a high-level sense of a good approach to an exam problem, tutorials for students to ask questions and receive immediate feedback, and videos for a detailed investigation of the exam-answering process.¹⁶⁰ So, modelling exercises are probably most effective when thoughtfully and purposefully integrated in the broader course design.¹⁶¹

6. Conclusion

The introduction of guided exercises, modelling exercises and example exercises to the Land Law course has proved useful, not only in enhancing students’ understanding of relevant content and helping them to develop their legal reasoning skills, but also in effectively rebuffing concerns about increasing transparency in assessment.

Students’ preferences for guided, modelling or example exercises are not clear-cut. At SOC and EOC, in both years, students considered guided exercises, modelling exercises and example exercises to each be useful components of a law course. In 2020, at SOC, students expected to find both example exercises and guided exercises more useful than modelling exercises – the difference was statistically significant. When reflecting on the usefulness of each exercise at EOC, students did not consider one to be better than the others – no statistically significant difference was found. In 2021, students did not expect or consider any one of the exercises to be more useful than the others at either SOC or EOC – no statistically significant difference was found.

The results indicate that students prefer an example exercise if it helps them to understand relevant content and develop legal reasoning skills. First, students considered exercises that helped them to develop legal reasoning skills to be more useful for a law course than exercises that helped them to enhance their understanding of the relevant content. Secondly, students considered example exercises more useful for understanding relevant content and developing legal reasoning skills than guided or modelling exercises. Hence, students may appreciate time and effort invested in developing example exercises more than investment in other exam-focused exercises. Thirdly, the closer students are to the exam, the more they value developing legal reasoning skills.

It is unclear whether students considered guided exercises, modelling exercises or example exercises to be more useful for (a) understanding relevant content and (b) developing legal reasoning skills. At SOC and EOC, in both years, students considered guided exercises, modelling exercises and example exercises to each be helpful for understanding relevant content and developing legal reasoning skills. There were some interesting results from 2021. First, for understanding relevant content, the Friedman test showed no significant difference between the three exercises at SOC but a significant difference between the three exercises at EOC, although a Wilcoxon signed-rank test showed no significant difference between modelling exercises and example exercises. Secondly, for developing legal reasoning skills, the Friedman test shows no significant difference between the three exercises at SOC but a significant difference between the three exercises at EOC – modelling exercises were considered more useful for developing legal reasoning skills than guided exercises (statistically significant), but there were no statistically significant differences between example exercises and the other two types of exercises.

Further, students indicated preference among three modelling forums. Students preferred lecturer modelling over tutor modelling – students valued the expert perspective. Importantly, students preferred lecturers modelling in online modelling exercises rather than in lectures, suggesting that modelling exercises are viable for demonstrating how to answer a problem and increasing transparency in assessment without needing to dedicate lecture time to doing so.

The article concludes that the exam-focused exercises are positive developments in the modern law school context in which it is increasingly common for law students to share notes. Databases with peer-produced notes and examples, such as the Notes Bank, are unlikely to disappear in the near future. By increasing transparency as to the purpose of assessments, what is expected of students and how assessment criteria are applied, students may become less reliant on peer-produced notes. Further, exam-focused exercises, like those studied in this article, which furnish students with materials produced, curated and regulated by the teacher, should reduce student reliance on non-teacher-regulated databases, like the Notes Bank, particularly given the value evidently placed by students on the expert perspective.

Increasing transparency in assessment does not deprive students from engaging in notetaking’s encoding and storage functions. However, it can mitigate other issues arising from reliance on peer-produced notes, such as low self-efficacy, student incompetence and academic misconduct. Indeed, exam-focused exercises may enhance students’ understanding of relevant content and their development of legal reasoning skills in ways that are efficient and meaningful to them.

The author acknowledges that this study was conducted before the widespread adoption of generative artificial intelligence. However, its findings are likely to become more relevant in this context. The increased use of generative artificial intelligence raises many of the same concerns as the Notes Bank does, and only provides more reason for teachers to increase transparency. The author invites other teachers to consider increasing transparency in assessment in their courses, particularly by incorporating exercises that engage, encourage and support students to develop their legal reasoning skills.

The author appreciates that implementing exam-focused exercises requires a time investment on the part of teaching staff.¹⁶² However, provided the course content is relatively stable, and the exercises are based on core areas of the course as opposed to peripheral issues that might change year to year, it is likely that all or most of the exercises can be used for multiple years with minor adjustments. The author sought student volunteers at the end of 2019 to donate their scripts to the project, before scanning the scripts, dividing them into questions and categorising those questions by grade, and then uploading them to the learning management system organised by grade. That process took a few full days of work. However, all the exercises the author developed in 2020 for the first iteration of the learning blocks continue to be used in 2024. Thus, the time spent developing exam-focused exercises can be confined to the first round of implementation, creating a lasting resource for future teaching.

Disclosure statement

No potential conflict of interest was reported by the author.

Notes

¹ The activities were introduced as part of a broader course assessment redesign. See Jayden Houghton, “Learning Modules: Problem-Based Learning, Blended Learning and Flipping the Classroom” (2023) 3 The Law Teacher 271. The activities feature in a learning modules programme that runs throughout the course. Briefly, the programme comprises five learning modules that students complete throughout the year. Each learning module focuses on a course topic and has four components. First, students complete a pre-tutorial activity (guided exercise) in which they are guided through a series of multiple-choice, short-answer and long-answer questions to answer a problem in bite-sized chunks. Secondly, students attend an in-person tutor-facilitated tutorial on the same problem, in which the students discuss the problem and review their answers. Thirdly, students take a post-tutorial quiz on key ideas relating to the problem. Finally, students are provided with exam-oriented review (EOR) exercises relating to the topic, specifically modelling exercises and example exercises, which they can use to prepare for exam questions on that topic.

² The Land Law course is a stage three 400-student required course in the undergraduate LLB degree. The LLB degree at the University of Auckland has four stages. In stage one, students study Law and Society, Legal Method and Legal Foundations, as well as electives from another degree programme. In stage two, students study Criminal Law, Public Law, the Law of Torts and the Law of Contract, as well as Legal Research, Writing and Communication. In stage three, students study Land Law, Equity, and Jurisprudence, as well as law electives. In stage four, students study Legal Ethics, as well as Advanced Legal Research, Writing and Communication, and law electives. Each stage equates to about a year of full-time study for students studying an LLB degree only.

³ To be admitted and enrolled as a barrister and solicitor in New Zealand, persons require a legal qualification approved by the New Zealand Council of Legal Education (NZCLE). The Bachelor of Laws (LLB) is currently the only approved legal qualification available at New Zealand law schools. Property Law is a required course for the LLB degree and is usually taught as separate Land Law and Equity courses, in which case each course is required. Overseas legal qualifications can be recognised for admission, but persons with an overseas legal qualification need to demonstrate competence in the required courses for the LLB degree and may be required to sit a New Zealand Law and Practice Examination for one or more required courses. New Zealand Council of Legal Education, Professional Examinations in Law Regulations 2008 (2017); and New Zealand Council of Legal Education, “New Zealand Law and Practice Examination: Information for the July 2023 Examinations” (2023). See also Lawyers and Conveyancers Act 2006.

⁴ The Land Law course must be assessed in part by exam. Land Law is a core course in the LLB degree that all law students must pass in order to complete their LLB degree and get admitted to the Bar. The New Zealand Council of Legal Education provides a prescription for the course. The prescription requires that the exam counts for at least 50% of the final grade. Before 2020, the prescription required that the exam counted for at least 60% of the final grade. The activities were developed in 2019.

⁵ Houghton (n 1) 287.

⁶ The drivers might not have been given airtime without university-wide policy changes requiring adjustments to the assessment regime in many courses, including the Land Law course. See ibid 278.

⁷ See Anders Jonsson, “Rubrics as a Way of Providing Transparency in Assessment” (2014) 39 Assessment & Evaluation in Higher Education 840, 840.

⁸ Peter K Rofes, “The Responsible Law School” (1991) 74 Marq LR 119, 120.

⁹ Anne Haarala-Muhonen and others, “Law Students’ Descriptions of Legal Reasoning” (2022) 56 The Law Teacher 471, 475.

¹⁰ Rohan Havelock, “Law Studies and Active Learning: Friends Not Foes?” (2013) 47 The Law Teacher 382, 401.

¹¹ Francina Cantatore and others, “A Comparative Study into Legal Education and Graduate Employability Skills in Law Students through Pro Bono Law Clinics” (2021) 55 The Law Teacher 314, 314.

¹² Cantatore and others (n 11) 315.

¹³ Havelock (n 10) 385.

¹⁴ At 385, fn 16. See BS Bloom and others, Taxonomy of Educational Objectives: The Classification of Educational Goal – Handbook I: Cognitive Domain (David McKay 1956). See also David R Krathwohl, “A Revision of Bloom’s Taxonomy: An Overview” (2002) 41 Theory into Practice 212.

¹⁵ Geoff Adlam, “Snapshot of the Profession 2019” (New Zealand Law Society 2019) 32. It is likely that the proportion of practitioners to graduates in New Zealand has decreased since 2017 given dramatic increases in the law student intake without any discernible expansion of the legal profession. See, for example, Vaimoana Tapaleao, “Auckland University’s Law School to Take More Students, But Staff Want Justice” (New Zealand Herald, 3 December 2016) <www.nzherald.co.nz/nz/auckland-universitys-law-school-to-take-more-students-but-staff-want-justice/DQQIXAZ4GS74JRYABHOG5MMSEM> accessed 18 May 2024.

¹⁶ In 2016, 44.1% of junior lawyers in New Zealand considered it unlikely or uncertain that they would continue to practise law in 10 years’ time. Josh Pemberton, “First Steps: The Experiences and Retention of New Zealand’s Junior Lawyers” (New Zealand Law Foundation 2016) 3.

¹⁷ ibid 13–14.

¹⁸ Stefan H Krieger, “The Development of Legal Reasoning Skills in Law Students: An Empirical Study” (2006) 56 J Legal Educ 332, 342–45.

¹⁹ See Elisabeth Peden and Joellen Riley, “Law Graduates’ Skills – A Pilot Study into Employers’ Perspectives” (2005) 15 Legal Education Review 87.

²⁰ In New Zealand, the core law courses required for admission to the legal profession need to assess legal reasoning skills, specifically the skill of applying law to facts. University of Auckland LLB students learn legal reasoning in terms of “applying law to facts”. The author is confident that most respondents would commonly understand what “applying law to facts” requires of them in practice, even if they cannot articulate that as a general, abstract instruction. See Dawn Jones and Lynn Ellison, “When Is a Word Not Just a Word? An Investigation into the Dissonance and Synergy between Intention and Understanding of the Language of Feedback in Legal Education” (2021) 55 The Law Teacher 155, 162.

²¹ See Jos HC Moust, “The Problem-Based Education Approach at the Maastricht Law School” (1998) 32 The Law Teacher 5, 8. See also Julie Tausend, “Flipping the Law Classroom to Ease Student Anxiety” (EdTech Focus on Higher Education, 6 December 2013) <www.edtechmagazine.com/higher/article/2013/12/flipping-law-classroom-ease-student-anxiety> accessed 29 February 2024; Tenielle Fordyce-Ruff, “Teaching Active Listening: Flipping Roles in Client Interviewing Exercises” (2014) 22 Perspectives: Teaching Legal Research and Writing 131; and Debra D Burke, “Scale-Up! Classroom Design and Use Can Facilitate Learning” (2015) 49 The Law Teacher 189.

²² The remaining 15% comprises 5% for the pre-tutorial guided exercises (1% each) and 10% for the post-tutorial quizzes (2% each). See Houghton (n 1) 279.

²³ Technically, the guided exercise is a quiz (graded survey) on Canvas. The guided exercises are not marked per se but audited – that is, the teacher flicks through each submission and summarily awards the mark to each student who made what looks like a good and honest attempt at the assessment. The emphasis is less on correctness but rather on demonstrating engagement with the problem and preparedness to engage in the tutorial. In practice, the mark is awarded if an intelligible and relevant answer has been provided for every question or all but one question. The submissions for a class of 400 students can be audited in two to three hours.

²⁴ The PBL approach at York Law School is an exemplar of PBL embedded in undergraduate and postgraduate legal curricula: “Study with York Law School” (York Law School) <www.york.ac.uk/law/study> accessed 29 February 2024.

²⁵ John Bennett, “LLM Student Guide to PBL” (University of York, September 2012) 7.

²⁶ ibid 9.

²⁷ Moust (n 21) 16–17.

²⁸ ibid 16.

²⁹ See ibid 34–35.

³⁰ Laura Hemker, Susanne Narciss and Claudia Prescher, “Design and Evaluation of a Problem-Based Learning Environment for Teacher Training” (2017) 11(2) Interdisciplinary Journal of Problem-Based Learning, Article 10, 2.

³¹ ibid 3.

³² See ibid 6.

³³ David Lopes Cardozo, Laurie Raymond and Benjamin White, “A Structured PBL Tutorial Involving Small Teams for Teaching the Human Nervous System” (2012) 34 Medical Teacher e763.

³⁴ ibid 765.

³⁵ See ibid 765.

³⁶ See ibid 768–69.

³⁷ Compare Moust (n 21) 16–17; Hemker, Narciss and Prescher (n 30) 3; and Cardozo, Raymond and White (n 33) e763–66. Contrast Bennett (n 25) 9.

³⁸ Abdullahi Salisu and Emmanuel N Ransom, “The Role of Modeling Towards Impacting Quality Education” (2014) 32 International Letters of Social and Humanistic Sciences 54, 54.

³⁹ John Hattie and Gregory CR Yates, “Acquiring Complex Skills through Social Modelling and Explicit Teaching” in Visible Learning and the Science of How We Learn (Routledge 2014) 72, 80.

⁴⁰ Salisu and Ransom (n 38) 54–55.

⁴¹ ibid 54.

⁴² ibid 55.

⁴³ Warren Haston, “Teacher Modeling as an Effective Teaching Strategy” (2007) 93(4) Music Educators Journal 26, 26.

⁴⁴ Hattie and Yates (n 39) 73.

⁴⁵ ibid 73.

⁴⁶ ibid 73.

⁴⁷ ibid 73.

⁴⁸ ibid 78.

⁴⁹ Nancy Maynes, Lynn Julien-Schultz and Cilla Dunn, “Modeling and the Gradual Release of Responsibility: What Does It Look Like in the Classroom?” (2010) 19(2) Brock Education 65.

⁵⁰ ibid 73. Compare Dolores Durkin, “What Classroom Observations Reveal about Reading Comprehension” (1979) 14 Reading Research Quarterly 481.

⁵¹ Maynes, Julien-Schultz and Dunn (n 49) 73.

⁵² ibid 73.

⁵³ ibid 73.

⁵⁴ ibid 73.

⁵⁵ Contrast Hattie and Yates (n 39) 77. See also, for example, two studies illustrating strategic instruction through modelling: Susan De La Paz and Mark K Felton, “Reading and Writing from Multiple Source Documents in History: Effects of Strategy Instruction with Low to Average High School Writers” (2010) 35 Contemporary Educational Psychology 174; and David F Feldon and others, “Translating Expertise into Effective Instruction: The Impacts of Cognitive Task Analysis (CTA) on Lab Report Quality and Student Retention in the Biological Sciences” (2010) 47 JRST 1165.

⁵⁶ Hattie and Yates (n 39) 77.

⁵⁷ ibid 77.

⁵⁸ ibid 77. See, for example, De La Paz and Felton (n 55); and Feldon and others (n 55).

⁵⁹ Hattie and Yates (n 39) 78. See also Paul A Kirschner, John Sweller and Richard E Clark, “Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching” (2010) 41 Educational Psychologist 75.

⁶⁰ Lisette Wijnia and others, “Is There a Role for Direct Instruction in Problem-Based Learning? Comparing Student-Constructed Versus Integrated Model Answers” (2014) 34 Learning and Instruction 22, 28.

⁶¹ ibid 28.

⁶² ibid 24.

⁶³ Hattie and Yates (n 39) 79. See also Carel P van Schaik and Judith M Burkart, “Social Learning and Evolution: The Cultural Intelligence Hypothesis” (2011) 366 Phil Trans R Soc B 1008.

⁶⁴ Hattie and Yates (n 39) 78.

⁶⁵ ibid 78.

⁶⁶ John Juriansz and David Newlyn, “Improving Performance in Contractual Problem Solving: Clarifying Criteria and Standards Through the Use of Exemplars” (2009) 2 Journal of the Australasian Law Teachers Association 185, 185.

⁶⁷ At 187. See, for example, “Auckland Law School Handbook 2019” (University of University, Faculty of Law 2019) 38–40.

⁶⁸ Juriansz and Newlyn (n 66) 187. The literature cites issues with teachers clearly and precisely articulating marking criteria and standards, and with students accurately decoding and understanding them. See Berry O’Donovan, Margaret Price and Chris Rust, “Know What I Mean? Enhancing Student Understanding of Assessment Standards and Criteria” (2004) 9 Teaching in Higher Education 325, 327; and Berry O’Donovan, Margaret Price and Chris Rust, “Developing Student Understanding of Assessment Standards: A Nested Hierarchy of Approaches” (2008) 13 Teaching in Higher Education 205.

⁶⁹ Juriansz and Newlyn (n 66) 187.

⁷⁰ D Royce Sadler, “Specifying and Promulgating Achievement Standards” (1987) 13 Oxford Review of Education 191, 202.

⁷¹ David Newlyn and Liesel Spencer, “Using Exemplars in an Interdisciplinary Law Unit: Listening to the Students’ Voices” (2009) 2 Journal of the Australasian Law Teachers Association 121, 121.

⁷² David Newlyn, “Providing Exemplars in the Learning Environment: The Case for and Against” (2013) 1 Universal Journal of Educational Research 26, 27.

⁷³ David Newlyn and Liesel Spencer, “Improving Student Performance in Interdisciplinary Law Unit Assessment by Providing Annotated Exemplars” (2010) 3(1) Journal of the Australasian Law Teachers Association 67, 74.

⁷⁴ ibid 74.

⁷⁵ Newlyn (n 72) 30.

⁷⁶ ibid 30.

⁷⁷ Patricia Grande Montana, “Meeting Students’ Demand for Models of Good Legal Writing” (2010) 18 Perspectives: Teaching Legal Research and Writing 154, 154.

⁷⁸ ibid 155.

⁷⁹ Elizabeth Ruiz Frost, “Feedback Distortion: The Shortcomings of Model Answers as Formative Feedback” (2016) 65 Journal of Legal Education 938, 958–59.

⁸⁰ Karen Handley and Benita Cox, “Beyond Model Answers: Learners’ Perceptions of Self-Assessment Materials in e-Learning Applications” (2007) 15 ALT-J 21, 32.

⁸¹ Hattie and Yates (n 39) 77.

⁸² Newlyn (n 72) 30 (diaeresis added).

⁸³ Montana (n 77) 156.

⁸⁴ See Carol McCrehan Parker, “Writing Throughout the Curriculum: Why Law Schools Need It and How to Achieve It” (1997) 76 Neb L Rev 561, 583.

⁸⁵ ibid 583–84.

⁸⁶ Montana (n 77) 156.

⁸⁷ ibid 156.

⁸⁸ Graham D Hendry, Nikki Bromberger and Susan Armstrong, “Constructive Guidance and Feedback for Learning: The Usefulness of Exemplars, Marking Sheets and Different Types of Feedback in a First Year Law Subject” (2011) 36 Assessment & Evaluation in Higher Education 1, 8.

⁸⁹ Or: substitute a more subtle film reference.

⁹⁰ “Auckland Law School Notes Drive Users’ Manual” (October 2017) at 1.

⁹¹ Kayla Morehead and others, “Note-Taking Habits of 21st Century College Students: Implications for Student Learning, Memory, and Achievement” (2019) 27 Memory 807, 808.

⁹² Edith Bouton, Smadar Bar Tal and Christa SC Asterhan, “Students, Social Network Technology and Learning in Higher Education: Visions of Collaborative Knowledge Construction vs. the Reality of Knowledge Sharing” (2021) 49 The Internet and Higher Education, Article 100787, 9.

⁹³ Kenneth A Kiewra, “A Review of Note-Taking: The Encoding-Storage Paradigm and Beyond” (1989) 1 Educational Psychology Review 147, 148. See Morehead and others (n 91) 808.

⁹⁴ Sharmila Gamlath and Therese Wilson, “Dimensions Student-to-Student Knowledge Sharing in Universities” (2022) 20 Knowledge Management Research and Practice 542, 549.

⁹⁵ Bouton, Bar Tal and Asterhan (n 92) 4.

⁹⁶ The Notes Bank encourages and reinforces this by publishing the student’s mark for the assessment when an example is uploaded and the student’s grade for the course when notes are uploaded. Students have the option of submitting notes and examples anonymously.

⁹⁷ Compare Bouton, Bar Tal and Asterhan (n 92) 9.

⁹⁸ See ibid 9.

⁹⁹ See, for example, “Transition to Digital Assessment, Grading and Proctoring” (Inspera) <www.inspera.com> accessed 29 February 2024; and “Empower Students to Do Their Best, Original Work” (Turnitin) <www.turnitin.com> accessed 29 February 2024.

¹⁰⁰ “University of Auckland Semester 1 2024 Examination Timetable” (University of Auckland 2024) 6–7.

¹⁰¹ “Exam Mode E – In-Person Invigilated Exam on Computer through Inspera Integrity Browser (IIB)” (University of Auckland) <www.auckland.ac.nz> accessed 18 May 2024.

¹⁰² Gamlath and Wilson (n 94) 549.

¹⁰³ Requiring students to print copious notes for locked-screen computer-based exams will be increasingly problematic as institutions increasingly commit to sustainability.

¹⁰⁴ Lynne Taylor and others, “The Making of Lawyers: Expectations and Experiences of Fifth Year New Zealand Law Students and Recent New Zealand Law Graduates” (Ako Aotearoa University of Canterbury, September 2019) 47: 37% were likely to be well, 49% were likely to have a mild/moderate mental health disorder and 14% were likely to have a severe mental health disorder.

¹⁰⁵ Graham Ferris, “Law-Students Wellbeing and Vulnerability” (2022) 56 The Law Teacher 5, 5. See Kennon M Sheldon and Lawrence S Krieger, “Does Legal Education Have Undermining Effects on Law Students? Evaluating Changes in Motivation, Values, and Well-Being” (2004) 22 Behavioural Sciences & Law 261, 262. See generally G Andrew H Benjamin and others, “The Role of Legal Education in Producing Psychological Distress Among Law Students and Lawyers” (1986) 11 American Bar Res J 225.

¹⁰⁶ See Houghton (n 1) 281.

¹⁰⁷ Lynne Taylor and others, “The Making of Lawyers: Expectations and Experiences of First Year New Zealand Law Students” (Ako Aotearoa, University of Canterbury, May 2015) 29–30.

¹⁰⁸ See Geoffrey M Currie, “Academic Integrity and Artificial Intelligence: Is ChatGPT Hype, Hero or Heresy?” (2023) 53 Seminars in Nuclear Medicine 719.

¹⁰⁹ See, for example, Monika Platz, “Trust Between Teacher and Student in Academic Education at School” (2021) 55 Journal of Philosophy of Education 688.

¹¹⁰ The study was approved by the University of Auckland Human Participants Ethics Committee on 17 March 2020 for three years (reference number 024318).

¹¹¹ Numeric calculations involved in computing means and standard deviations are valid only for continuous variables using a ratio scale. For example, a mean of 4.5 is not defined because “agree and a half” has no meaning.

¹¹² The response rate at SOC 2021 was noticeably higher than that at SOC 2020, perhaps reflecting that some students had some knowledge of the learning modules from previous students.

¹¹³ These students are the two standalone data points (outliers) at the bottom of the boxplot for guided exercises and the one standalone data point at the bottom of the boxplot for example exercises.

¹¹⁴ Compare Montana (n 77) 156.

¹¹⁵ See Hattie and Yates (n 39) 73.

¹¹⁶ Compare Newlyn and Spencer (n 71) 124–25; and Newlyn (n 72) 27–28.

¹¹⁷ Compare Hattie and Yates (n 39) 77.

¹¹⁸ Contrast Hemker, Narciss and Prescher (n 30) 6; and Moust (n 21) 34–35.

¹¹⁹ Contrast Hemker, Narciss and Prescher (n 30) 6.

¹²⁰ In other words, the additional load imposed by guided exercises potentially detracted from other activities that might assist in the assimilation of information. See Hattie and Yates (n 39) 78.

¹²¹ See Salisu and Ransom (n 38) 54.

¹²² Compare Frost (n 79) 958–59.

¹²³ Contrast Parker (n 84) 583. See Newlyn (n 72) 30.

¹²⁴ Compare Handley and Cox (n 80) 32.

¹²⁵ Compare Montana (n 77) 156.

¹²⁶ Newlyn (n 72) 30.

¹²⁷ See Parker (n 84) 583.

¹²⁸ Compare Montana (n 77) 156.

¹²⁹ Hendry, Bromberger and Armstrong (n 88) 8.

¹³⁰ Cardozo, Raymond and White (n 33) 766.

¹³¹ Haarala-Muhonen and others (n 9) 481.

¹³² Erik Driessen and Cees Van Der Vleuten, “Matching Student Assessment to Problem-based Learning: Lessons from Experience in a Law Faculty” (2000) 22 Studies in Continuing Education 235, 236.

¹³³ ibid 236.

¹³⁴ See Haarala-Muhonen and others (n 9) 472. However, at the University of Auckland, legal reasoning skills are primarily explicitly taught in the core curriculum in early foundation courses.

¹³⁵ Cathy Driscoll and David Wicks, “The Customer-Driven Approach in Business Education: A Possible Danger?” (1998) 74 Journal of Education for Business 58, 59.

¹³⁶ Tawney Bennett, “The Effects of Student-Consumerism on Discipline Specific Teaching Practices: A Comparison of Education and Law” (2019) 45 Journal of Further and Higher Education 417, 421.

¹³⁷ ibid 427.

¹³⁸ Compare Haston (n 43) 26.

¹³⁹ Samuel Messick, “The Interplay of Evidence and Consequences in the Validation of Performance Assessments” (1994) 23(2) Educational Researcher 13, 16.

¹⁴⁰ ibid 16.

¹⁴¹ Compare Montana (n 77) 156.

¹⁴² See Newlyn and Spencer (n 71) 124–25; and Newlyn (n 72) 27–28.

¹⁴³ Compare Parker (n 84) 583.

¹⁴⁴ Compare Salisu and Ransom (n 38) 54.

¹⁴⁵ Wijnia and others (n 60) 28.

¹⁴⁶ ibid 24.

¹⁴⁷ See Montana (n 77) 154.

¹⁴⁸ IRAC stands for issue, relevant law (rules), application (applying law to facts) and conclusion.

¹⁴⁹ See Newlyn (n 72) 30; and Montana (n 77) 155.

¹⁵⁰ Compare Newlyn and Spencer (n 71) 124–25; and Newlyn (n 72) 27–28.

¹⁵¹ Compare Montana (n 77) 156.

¹⁵² Compare Hattie and Yates (n 39) 77.

¹⁵³ See Hattie and Yates (n 39) 73.

¹⁵⁴ See ibid 73.

¹⁵⁵ See ibid 73.

¹⁵⁶ Compare Cardozo, Raymond and White (n 33) e768–69.

¹⁵⁷ See Hattie and Yates (n 39) 73.

¹⁵⁸ See Maynes, Julien-Schultz and Dunn (n 49) 73.

¹⁵⁹ See Hattie and Yates (n 39) 73.

¹⁶⁰ Compare ibid 73.

¹⁶¹ See Havelock (n 10) 385.

¹⁶² Newlyn (n 72) 30.

Appendix A

Learning Module A

Problem

In 2018, Elizabeth purchased a small investment property in the Auckland suburbs. The purchase price was $1,000,000, of which she contributed $500,000. She borrowed the balance of $500,000 from her sister, Jane, at 4% annual interest and signed a mortgage agreement as security. The mortgage was not however registered, nor did Jane get her lawyer to lodge a caveat at that time. Jane just wanted to help Elizabeth, and was happy to ignore the interest, as long as the principal would be paid off in quarterly instalments over the next 5 years.

In mid-2019, Elizabeth decided to complete a small extension to the property, but, as she did not have a good credit rating, she was unable to borrow from a bank. Instead, she borrowed $300,000 from Pemberley Finance at 10% annual interest payable fortnightly for a term of 5 years. The principal was repayable at the end of the term. Elizabeth signed a mortgage agreement, but the mortgage was not registered on title. Because of a solicitor’s error, Pemberley Finance did not search the title. It did lodge a caveat to protect its interest immediately after having entered into the mortgage agreement with Elizabeth.

In 2021, Elizabeth’s café business went into receivership. She has not been able to pay the interest on Pemberley Finance’s mortgage since March 2021. Elizabeth has also not paid any instalments to Jane since early 2021. Jane became concerned and lodged a caveat to protect her interest in mid-2021. Jane says that Elizabeth will have to pay interest on the unpaid amount until the debt is repaid. Pemberley Finance became aware of the loan from Jane in late 2021 but did not immediately take steps to initiate a mortgagee sale as there was then sufficient value in the property to cover both debts.

It is early 2023. With penalty interest and legal fees, the debt to Pemberley Finance has now compounded to $400,000. Elizabeth also still owes Jane $400,000 plus some $50,000 of back interest. Furthermore, it has recently been discovered that the home is flood-prone, and the property is now valued at only $700,000.

Pemberley Finance wishes to commence the mortgagee sale process.

Who has priority to the property? Refer to relevant cases and statutory provisions in your answer.

Appendix B