Designing public financial management systems: exploring the use of chatbot-assisted case studies

IMPACT

The design of public financial management (PFM) systems requires the exploration of the problem space before solution options are generated. Case studies are often employed to teach the design of PFM systems, but conventional forms of delivery of case study materials fail to help develop the skills needed to explore the problem space. This article investigates the use of chatbot-assisted case studies as a way to stimulate students’ efforts to pose questions about features of the problem scenario.

ABSTRACT

This article describes the use of a chatbot-assisted case study in teaching accountancy students how to design a public financial management system. A case study on the redesign of invoice payment system at the government of Andalucía in the 1980s was delivered to 18 distance-learning postgraduate students via a chatbot that helped explore the problem space. Analysis of conversational interactions and students’ feedback provides some support for the use of chatbot-assisted case studies but also some limitations.

KEYWORDS:

• Accountancy teaching
• case study
• chatbot
• design
• distance learning
• natural language processing
• payment system
• public financial management

The capacity to design public financial management (PFM) systems is an important part of public sector accountants’ expertise. The design of PFM systems consists of configurations of accounting and finance institutions, rules and regulations that are intended to help attain the goals of relevant stakeholders, such as policy-makers, members of representative bodies, public managers, taxpayers and the general public (Cangiano et al., 2013). Designing effective PFM systems is crucial to deliver public value (Kioko et al., 2011). Public sector accountants are increasingly expected to display various competences and skills, in addition to proficiency in financial accounting, management accounting and auditing, such as those related to leading change and innovate (CIPFA, 2020). Yet, public sector accounting education devotes relatively little attention to cultivating design capacity. As a ‘meta-competence’ (Brown & McCartney, 1995), developing design capacity requires the acquisition of problem-framing and problem-solving skills that help bridge the gap between theoretical education and practical application.

A main challenge in the design of PFM systems—as in any design problem scenario generally (Goel & Pirolli, 1989; Goldschmidt, 1997)—is exploring the problem space. The problem space in a design problem scenario consists of features of what the problem is, what the needs, goals and requirements of relevant stakeholders are, and what constraints arise from the context (Hmelo-Silver et al., 2000; Maedche et al., 2019). Designers explore the problem space by searching for specific features of the problem scenario at hand that are relevant to explore the solution space—i.e. possible configurations that fulfil the goals and satisfy the requirements of relevant stakeholders. Also, because of limited time and resources, designers should develop the capacity to orient their search for relevant features of the problem scenario while ignoring irrelevant details. A main challenge in design, then, is how do designers know what to search for when exploring the problem space?

Case studies have been widely used to teach problem-framing and problem-solving skills (Lowensohn & Reck, 2005; Rivenbark, 2007). In teaching case studies, students are often exposed to problem scenarios where they are required to appraise present conditions or future tendencies, explain sources of dissatisfaction with current or expected performance, devise possible remedies, and decide future courses of action. Case studies are also used to develop the capacity to design PFM systems by asking students to devise design solutions to given features of problem scenarios, including details about the needs, goals and requirements of relevant stakeholders and constraints that arise from context conditions. Students are typically provided with all the relevant information about the problem scenario in the case study materials. The explicit provision of problem scenario features, however, does not stimulate students’ skills to actively search for relevant information. In a way that is similar to actual practitioner experience, students should face the challenge to gather accounts of the problem scenario, to make sense of partial, contradictory or ambiguous information, and to figure out what else they need to know about the problems space in order to explore the solution space (Ertmer & Koehler, 2014).

We investigated the potential for using chatbot-assisted case studies in teaching the design of PFM systems, with specific attention to the exploration of the problem space. A chatbot-assisted case study is a form of delivery of a case study where students interact with an automated conversational software that can impersonate a case study character (Io & Lee, 2017). Differently from a conventional text-based form of delivery of case study materials, a chatbot-assisted case study is intended to stimulate students’ skills to explore a problem space by requiring them to pose questions about features of the problem scenario. This study repurposed Barzelay’s (1988) case study on the redesign of the invoice payment system at the government (‘Junta’) of Andalucía in the 1980s to develop a web-based chatbot-assisted case study on the design of an invoice payment process. The chatbot-assisted case study was delivered to 18 postgraduate students on online courses at the Centre for Financial and Management Studies (CeFiMS) of SOAS University of London in July–August 2020. Analysis of conversational interactions and students’ feedback provides some evidence of potential benefits of the use of chatbot-assisted case studies to stimulate students’ capacity to design PFM systems but also of limitations that need to be carefully addressed. This article contributes to the research on training and education of accountants in the public sector (Stout et al., 2013; Thom, 2019; Heiling, 2020).

Teaching accounting students how to design PFM systems

By teaching accounting students how to design PFM systems they will become more knowledgeable about PFM sub-systems and processes, for example, bookkeeping, invoice processing, cash management, budgeting and preparation of financial statements. They will also be better versed in the functions, effects and limitations of PFM institutions, tools and practices. They need an understanding of the role of specific context conditions—crafting tailored design solutions to PFM problems, rather than adopting a one-size-fits-all solution. They will therefore learn to exercise judgement when considering the most appropriate interventions.

An important component of the capacity to design PFM systems is exploring the problem space (Kotovsky & Simon, 1990). Exploring the problem space entails, in part, collecting information about clients’ needs, existing context conditions, interactions between a system and its environment as well as between component parts of a system, and future tendencies. The most challenging part of problem space exploration is making sense of information, properly understanding the problem at hand, and focusing on possible solutions (van de Ven, 2007). Various studies have highlighted the importance of identifying the correct problem space (Keren, 1984; Maedche et al., 2019), as well teaching students how to navigate the problem space effectively (Appleton, 1995; McCall, 2012).

Acquiring the skills to explore the problem space results in students’ capacity to identify stakeholders, needs, goals and requirements (Maedche et al., 2019). When learning to design PFM systems, students develop the capacity to appraise whose needs should be prioritized among stakeholders (van de Ven, 2007), to devise solution conjectures that fulfil the intended goals (Cross, 2001) and to select configurations of PFM institutions, tools and practices that bring about the desired effects. Case studies have been long employed in teaching design skills and capabilities (Breslin & Buchanan, 2008; Koehler & Ertmer, 2016). Previous research has shown that teaching strategies where students actively explore the problem space (for example by posing questions) are more effective than those where students are passive recipients of information about the problem scenario (Appleton, 1995; Chin & Chia, 2004; Ertmer & Koehler, 2014; Phillips et al., 2018).

The growth of online and blended learning during the last years has created opportunities and challenges in terms of teaching PFM systems. The lack of face-to-face exchange between instructors and students and between students themselves in online teaching seriously hampered the use of case studies. On the other hand, online resources provide tools for asynchronous interaction and automated responses that help adjust the learning process to specific students’ time availability and learning style. The use of chatbot-assisted case studies could provide a means to stimulate students’ skills to explore the problem space by making them interact with a ‘virtual instructor’ that impersonates a case study character.

Using chatbots in teaching PFM systems design

Chatbots have increasingly attracted attention within education circles for their potential to administer and assist relatively large cohorts of students (Winkler & Soellner, 2018) and for personalized learning (Perez et al., 2020). Chatbots are computer programs that aim to simulate human behaviour by interacting with users through text messages (Bernardini et al., 2018). Chatbots are designed to carry out conversations with users through natural language as inputs and outputs (Brennan, 2006), typically building on a knowledge base that helps answer questions (Fryer & Carpenter, 2006). While transactional chatbots execute a user command as a single task, conversational chatbots are designed to carry out a conversation on a topic (Long et al., 2019). Improvements in natural language processing (NLP) has resulted in chatbots that are increasingly capable to imitate meaningful conversations, although users may sometimes dislike the awkward feeling (‘uncanny valley’) that arises from the close but far-from-perfect resemblance of chatbot exchanges with those with other humans (Ciechanowski et al., 2019).

As discussed above, exploring the problem space is an important skill that students of PFM systems design should develop. Rather than providing problem scenario features explicitly in case study material, a chatbot-assisted case study requires students to actively search for relevant information by questioning the chatbot, which can impersonate a case study character. In this way, students can develop the capacity to identify missing information, elicit the disclosure of relevant information through specific questions, and integrate new information into a more comprehensive understanding of the problem scenario. A further advantage is that students can interact with the chatbot at any time and, therefore, they can also experiment with different approaches to explore the problem space.

The use of chatbots in teaching has some weaknesses and limitations, however. Previous research showed that the role of the instructor and of class discussion are pivotal in the effectiveness of case study on students’ learning (Flynn & Klein, 2001; Gillies, 2011; Levin, 1995; Stepich et al., 2001). Without adequate background knowledge and stimuli, students may explore the problem space too superficially and provide simplistic or inadequate design solutions (Ertmer & Koehler, 2018; Heckman & Annabi, 2006; Hmelo-Silver & Barrows, 2006; Hmelo-Silver et al., 2002; Perez & Emery, 1995; Tan & Ng, 2006). Too much structure may discourage discussion, while too little instruction may leave students without clues and sense of direction (Mitchem et al., 2008). Chatbots, therefore, may assist case study teaching provided that they are embedded within a participative and inclusive teaching strategy.

Method

This exploratory study focused on the development of students’ capacity to design PFM systems, with specific attention to the stimulation of the skills to explore the problem space. Our research question was: what role can chatbots potentially play in the delivery of case studies that are intended to help students develop their capacity to explore the problem space for the design of PFM systems? The study followed the action research method, which fits conditions where the researchers are integral part of the researched social phenomenon (Baskerville, 1999; Hult & Lennung, 1980). The fieldwork consisted of:

• The design and implementation of a chatbot intended to assist students to explore the problem space of a selected case study.

• The delivery of the chatbot-assisted case study to a cohort of 18 distance learning postgraduate students.

• The collection of student interaction records with the chatbot and of their feedback to the use of the chatbot in the case study.

The chatbot developed for this study impersonated a character in Barzelay’s (1988) case study on the redesign of the invoice payment system at the Junta de Andalucía in the 1980s. The case was positioned within the development of PFM systems at the Junta, which was formed in 1982 in the context of a growing call for increased autonomy of Spanish sub-national governments from the central government in Madrid. As the Junta expanded activity into more areas, from agriculture and fishing to education and urban development, financial administration became burdened by a growing volume of tasks, including the payment of invoices. Upon arrival, invoices were sent by the Junta departments to the treasury division of the finance department, which issued validated transfer orders to banks. The invoice payment process triggered various sources of dissatisfaction, however. Vendors complained of delayed and missed payments, and inquiries overburdened the administration with repeated phone calls and visits to bureaus.

Within this context, students were required to play the role of a newly-appointed head of the treasury section, who had to fix the invoice payment process and make it work quickly, efficiently and in an accountable way. Students were required to attend a simulation of a meeting with the head of finance (whose role was played by the chatbot), who would explain how the payment process worked, provided that students posed the appropriate questions to elicit the necessary information. Students were required to offer a design solution to the problems of the invoice payment system.

The chatbot was designed by mapping the problem space, drawing from one of the author’s personal experience of facilitating the case study on several occasions in the past. The case problem space was mapped into six categories. Questions that students could ask, and relevant information that they should acquire in order to gain a comprehensive understanding of the problem scenario, were devised. A list of pairs of questions and answers was progressively enriched through two rounds of small-scale (three students) focus group discussions. The process resulted in the selection of 36 answers that students might obtain by asking a total number of 197 possible questions. The chatbot was implemented through Google’s Dialogflow. NLP capabilities of Dialogflow ensured that questions could be identified even if students posed them slightly differently from the expected phrasing. The chatbot was delivered through a web-based chat box.

Data collection

The chatbot-assisted case study was delivered to 18 postgraduate students (four female; 14 male) on distance learning courses at the Centre for Financial and Management Studies (CeFiMS) of SOAS University of London in July–August 2020, who were taking a 10-week module on PFM. The learning objective of the case study was to teach students how to design a component part of the PFM system, while taking into consideration multiple stakeholders’ needs, goals and requirements. The course convenor introduced the students to the design of PFM systems, including the functions of assessing a client’s needs, exploring alternative design features, and reconfiguring PFM processes. Students were told that the case study on the Junta of Andalucía provided a problem scenario requiring the redesign of the invoice payment system at the Andalucía government. Background knowledge included topics on the spending process in government, budget execution, and cash management, which had been covered in the previous two weeks of the course. All students had work experience in the public sector, including some exposure to financial management processes.

Students were instructed that, after reading an introduction to the case, they would play the role of the newly-appointed head of the treasury section, while the chatbot would play the role of the head of finance. Students were encouraged to explore the problem space by asking the chatbot questions about the performance of the existing invoice payment process, features of the existing PFM system, and reasons for the existing arrangements for invoicing and paying vendors via banks. Students were then invited to share comments between themselves and the instructor and then submit a proposed design solution to an online discussion forum, which was facilitated by the course convenor over a period of one week.

The chatbot session was attended by 13 students, although just three of them carried out relatively long conversations (i.e. with more than 10 messages sent to the chatbot) (see Table 1). The two researchers independently coded the exchanges between the students and the chatbot and then reconciled their views. Eleven students submitted design solutions to the case study. Proposed design solutions varied considerably. Two solutions (provided by students Nos 12 and 13, indicated as 12/male and s13/male in Table 1) approximated the proposal to redesign the invoice payment system that is offered as case study solution in Barzelay (1988), i.e. that transfer orders should be aggregated and sent to specific banks where vendors held their respective bank accounts. Other solutions provided relatively marginal or limited reconfigurations of the invoice payment process. Four students also provided feedback on the use of the chatbot.

Table 1. Number of messages sent to the chatbot.

Number of messages sent to the chatbot	Number of students	Student clue
1	3	s1/male, s2/male, s3/male
2	1	s4/female
3	2	s5/male, s6/male
4	2	s7/male, s8/male
6	1	s9/male
8	1	s10/female
14	1	s11/female
20	1	s12/male
21	1	s13/male
	13

Data analysis

Following previous studies (Ertmer & Koehler, 2014; Jalota et al., 2019; Følstad et al., 2018), the exchanges that the students had with the chatbot were coded according to the six categories in Table 2, and then requests and responses were analysed. Coding the exchanges was intended to appraise the extent to which the problem space was covered. The analysis showed that questions from the students addressed most of the problem space, although they paid relatively little attention to the reasons and explanations for the activities performed within the invoice payment process and to the timing and duration of activities. Lack of attention to these parts of the problem space could limit students’ understanding of the problem scenario and their search for possible solutions.

Table 2. Questions posed by category of areas of the problem space, with examples of questions asked by students and total number of questions.

Category No.	Category of the problem space	Examples of questions asked	No. of questions
(i)	General features of the problem scenario	What are sources of dissatisfaction with the payment process? What would you like to be done or improved?	9
(ii)	Identification of the entities involved in the problem scenario	Who are the parties involved in the payment process? What is the treasury division?	10
(iii)	Features of the invoice payment process	How does the payment process work? Who makes the transfer orders?	18
(iv)	Reasons and explanations for the activities performed in the invoice payment process	Why does the Junta of Andalucía hold accounts in several banks? Why did it take so long to make payments?	3
(v)	Modalities of execution of the activities	How are invoices matched with orders? How are payments tracked?	13
(vi)	Timing and duration of activities	How long on average does it take to pay invoices? How long does it take for the banks to make payments?	3

Request analysis showed that most of questions expressed the intent to acquire information about the procedural details of the invoice payment process and the needs of the finance department, which they could have understood as the main stakeholder. On occasion, students’ questions displayed an intent to explore the options in the solution space, for example ‘Can the finance department move all funds to a single bank?’ (s12/male). Students paid very little attention to the needs of stakeholders other than the finance department. They were also uninterested in exploring the rationales for the design choices that had been made for the existing invoice payment process. Questions were relatively simple (the average length of questions was 6.5 words), as already noted in previous research work (Hill et al., 2015). The most common terms were ‘how’ (27 occurrences, 13%), ‘payment process’ (26 occurrences, 13%), ‘invoices’ (nine occurrences, 4%), ‘transfer’ (seven occurrences, 3%), and ‘checked’ (six occurrences, 3%).

Response analysis showed that the students posed 12 questions which the chatbot did not answer (see Table 3). Lack of an answer to three of the questions might originate from the use of pronouns, which the chatbot may have not related to an antecedent noun. A couple of questions were not articulated in a complete sentence. Five questions were phrased in a way that the chatbot could not successfully analyse, but which were subsequently rephrased in a way that matched an intent. Another couple of questions called for hints about the solution, which the chatbot had not been designed to address.

Table 3. Questions unanswered by the chatbot by type of failure, with examples of questions asked by students and total number of questions.

Issue type No.	Type of issue	Examples of questions asked	No. of questions
(i)	Use of pronouns	Why is this done in this way? How can this be more efficient?	3
(ii)	Incomplete sentence	PO? Chiefs?	2
(iii)	Phrasing of the sentence	Who can make a transfer order? (The question was then rephrased as: Who makes the transfer orders?) Sources of dissatisfactory performance? (The question was then rephrased as: What are the sources of dissatisfaction?)	5
(iv)	Question outside the knowledge domain of the chatbot	How can the process be made more efficient? Why wasn’t the payment process fixed before?	2

The online discussion forum provided some evidence of students’ understanding of the problem scenario. Students’ comments displayed awareness of the features of the problem:

The payment process performs quite poorly. It takes about two weeks from when a payment order is received by the treasury division until a transfer order is sent to a bank. The treasury does not know, then, how long the inter-bank transfer process takes—but, judging from the complaints received from the vendors, the execution of transfer orders may take many days or even weeks (s13/male).

Students’ comments also showed a fair understanding of the activities carried out in the invoice payment process and hinted at possible areas of intervention:

The treasury division of the finance department checks the payment orders, it registers the arrival of the payment orders and examines the materials for completeness. Within the treasury division, the treasury section issues the valid transfer orders, which authorize a bank to transfer funds from the Junta's account to the account of a vendor. I believe receiving banks should be investigated and other ways of payments should be employed (s12/male).

Students’ comments also provided evidence of identification of plausible areas of the solution space that could be pursued, together with an explicit reference to sources of dissatisfaction in the existing invoice payment system, as in the following quotations:

Since the creation of the Junta, payments to vendors have been made through banks. The Junta holds accounts in several banks in the region. I would suggest the Junta make payments from a single bank to reduce the hectic that treasury goes through (s11/female).

The payment process is manual. There could be an automated system (depending on the technology that existed at the time). Eight employees for that amount of work do not seem enough (s10/male).

Payment orders don’t seem to have unique IDs, which creates confusion (s13/male).

Discussion

Feedback on the use of the chatbot in the case study was received from four students via an online forum. This feedback included some positive comments: ‘The use of chatbot is a good way of inviting us to think deeper about the case and ask further questions’ (s12/male) and that ‘Many answers were clear’ (s11/female). The feedback also included some issues concerning coverage of the problem space (‘Sometimes I could not get an answer to questions about motivation of employees or quality of the process’, s10/male, and ‘I find that the answers lacked numerical data’, s9/male), but the chatbot had not been designed to answer questions about features of the problem scenario that would be immaterial to tackling the case study problem.

Researchers’ reflections on the use of chatbot included both positive and critical considerations. On the one hand, the chatbot seemed to help stimulate some students to actively explore the problem space of the assigned case on the invoice payment process. By using the chatbot, some students could have learned ‘what to ask’ when searching for information on the problem scenario. Had the students been provided with more background knowledge and prompts to suggest how the problem space could be explored, then they might have made better use of the chatbot. Students might have gained a better understanding of the problem space if they had taken part in a live class discussion, rather than sharing their comments and proposed design solution through asynchronous means (a forum and message communications).

This exploratory study provides some evidence about the role that chatbots can potentially play in the delivery of case studies. Students’ use of the chatbot confirmed the potential of chatbot-assisted case studies to assist the exploration of a problem space. In our case study on the redesign of the invoice payment process in the Junta of Andalucía, the use of a chatbot allowed us to simulate a meeting where students recalled their knowledge about features of expenditure management (for example issuing transfer orders, validating transfer orders, and monitoring cash disbursement), formed a mental model of the existing invoice payment process, and figured out what information they needed in order to appreciate needs, goals and requirements of the main stakeholders (i.e. the department of finance). In so doing, the use of the chatbot elicited the development of problem-framing and problem-solving skills in ways that differed from the learning process that would be triggered by the use of text-based case study materials only.

Students’ use of the chatbot also highlighted the limitations of chatbot-assisted case studies. The modest level of engagement (both in terms of the number of students meaningfully interacting with the chatbot and in terms of the number of exchanged messages) suggests that students could be discouraged by the difficulties encountered in having meaningful conversations with the chatbot. At present, state-of-the-art open source NLP technology cannot handle complex conversations. Existing technical limitations may prevent the use of chatbots in the exploration of more intricate problem spaces requiring relatively sophisticated sentences in terms of grammatical structure and semantic content, for example anaphoric and co-referent usage (Sukthanker et al., 2020). It is possible that students were not familiar with the use of chatbots in teaching, or that their expectations towards chatbots are biased because of dissatisfactory past interactions with other conversational interfaces. Perhaps our students should have received more assistance or guidance to make sense of the chatbot within the overall structure and learning objective of the case study.

These limitations suggest some ways to further explore the potential of chatbot-assisted case studies. The use of chatbots in the delivery of case studies may benefit from improved capabilities of NLP technology in the future (for example OpenAI’s GPT-3). Diffusion of chatbots in commercial and public services may increase familiarity with conversational interfaces over time. Instructors may devise improved teaching strategies to integrate chatbots with features of students’ background knowledge, instructors’ guidance and class discussion. The task of exploring the problem scenario at hand might be better split into smaller assignments so that it could be explored in chunks with the assistance of chatbots that are designed around specific knowledge domains. For example, different chatbots might impersonate different characters of a case study that students can interrogate about specific concerns or activities or constraints before reporting to the instructor or to the class about what they learned about a particular component part of the problem scenario.

The study suggested some ways forward for designing an effective learning experience for students by using a chatbot-assisted case study:

• First, the chatbot should be embedded within a case study narrative that provides enough background information and clues to make sense of chatbot responses.

• Second, the instructor should define the knowledge domain of the chatbot, which includes answers to possible questions about the case scenario, and a related list of intents that the students may express. Heuristically, the knowledge domain may be conveniently structured around answers to questions about what happened, why events happened, when events happened, how events happened, and where those events happened.

• Third, students should be provided with instructions about the chatbot interaction explaining what information they can get from the chatbot, what they the chatbot cannot answer, and when they should stop interaction with the chatbot before moving on to next task.

Conclusions

Teaching PFM is about making sure students understand the normative and regulatory side to PFM, as well as the technical and quantitative aspects. As accounting professionals, PFM specialists also need to possess the capacity to design PFM systems while taking into consideration specific circumstances that arise from stakeholders’ needs, goals and requirements and context conditions. Teaching students to develop the capacity to design PFM systems can be challenging. Students need to know how to tackle problem-framing and problem-solving, including how to acquire relevant information about a client’s needs and requirements, features of existing PFM processes, constraints that arise from the context, and the options available. Case study teaching provides a way to develop such a capacity, although the cultivation of the skills to explore a problem space requires the stimulation of an inquisitive mind that may not be fully attainable through text-based case study materials. Instructors are important in order to increase students’ curiosity and capacity to notice missing information that would help clarify the features of the problem to solve. Current technologies provide additional tools for the delivery of case studies—chatbots, in particular, enable students to interactively engage with a case scenario remotely and on an individual basis.

Our study has provided some support to the argument about the potential of chatbot-assisted case studies to help students explore the problem space when dealing with a PFM system design challenge. The study had a few limitations. For example, shortcomings of the technology hampered the conversation between the students and the chatbot. Most students did not ask enough questions to explore the problem space. No evidence was collected regarding students’ skills development. We did not investigate how students can be engaged into meaningful exchanges with chatbots, what students do when they explore a problem space with the assistance of a chatbot, and whether chatbot-assisted case studies effectively stimulate students’ skills to explore the problem space.

More research is needed, therefore, to investigate the design, implementation and effectiveness of chatbots in education generally, and PFM in particular. Research on chatbot design could focus on what forms of chatbot interaction are more appropriate to attain specific learning objectives and how chatbots can assist or complement other forms of teaching. Research on chatbot implementation could tackle questions about what affects students’ engagement and what role instructors should play in chatbot-assisted teaching. Research on the effectiveness of chatbots could address issues concerning whether the use of chatbots helps increase students’ acquisition of knowledge, skills and capabilities, or helps relatively weaker students (who may benefit from multiple sources of stimuli or more frequent interactions), or makes learning faster or more cost-effective (especially when teaching large cohorts). A related set of issues, in this respect, also arises from the comparative analysis of different approaches and forms of teaching. Further research might compare the use of chatbot-assisted case study with a conventional text-based form of delivery of case study materials.

NLP technology is developing quite rapidly, so chatbots are likely to be able to carry out more meaningful conversations in the near future and therefore help deliver case studies by impersonating characters in problem scenarios. Chatbot-assisted case studies could then be used to stimulate the development of students’ skills to explore the problem space when they are given the challenge of designing improved PFM systems.

Acknowledgments

We would like to thank the Centre for Distance Education of the University of London for funding this research project.