Teaching Analytics Online: A Self-Study of Professional Practice

ABSTRACT

As the COVID-19 pandemic caused severe disruption to education enterprises throughout the world, the main response by educational institutions was to move to online learning environments. The purpose of this study was to understand better how instructors could improve online learning for a professional-level week-long short course in a highly technical area (data analytics), which had, pre-COVID, been a hands-on computer, laboratory-based learning experience. The authors used self-study of professional practice to elicit and understand the major issues and concerns of the transition to an online learning environment. Under the guidance of a colleague in teacher education, three course instructors recorded and analyzed their experiences and, through discussion and data analysis, gained a deeper understanding of three challenging dimensions: course adaption to the online environment, personnel considerations, and technological considerations. Through the reflection that self-study provided, several of these challenges were addressed in subsequent courses. Self-study provided the instructors with an opportunity to reflect on the transition of the course to an online environment. This, in turn, provided an opportunity for course improvement and, arguably more importantly, an opportunity for the instructors to better understand each other’s concerns and desires as it pertained to the course.

KEYWORDS:

• Self-study of professional practice
• engineering education
• online learning
• short course
• professional learning
• data analytics

The COVID-19 pandemic had a significant impact on the world. Beyond the loss of human life, it changed how people work and learn, with the United Nations Educational, Scientific and Cultural Organization (2021) estimating that up to 81.8% of all enrolled learners in the world were affected by the pandemic. Aliyyah et al. (2020) concluded that K-12 students were left substantially behind previous cohorts, and their ability to catch up was hindered. The pandemic forced many to engage in new forms of teaching, especially online (Jovanovic et al., 2022), and so, whilst the pandemic was disruptive, some have argued it could catalyze change within education (Campos-Remon et al., 2021, Darling-Hammond & Hyler, 2020).

Academic research has been conducted about the COVID-19 pandemic and its effect on the world. A search for the keyword ‘COVID’ in Google Scholar reveals over five million results; limiting this search to ‘COVID and Education’ still shows four million results. This begs the question, is there anything left to say about the effects of the COVID-19 pandemic in the context of education? If we limit our keyword search even further to ‘COVID and professional learning and data science’ (or data analytics), we find less than 300 manuscripts, with almost all about data analytics’ use to improve educational enterprise. For example, Ifenthaler (2021) wrote about using learning analytics for school management, giving examples of applications from the pandemic. Discussions on the actual teaching of data analytics are limited. We believe this is due to the unique nature of teaching data analytics because, unlike other technical subjects, it requires a holistic understanding of various technical areas (Leathrum et al., 2020). Similarly, research on the pandemic’s impact on professional learning has focused on teacher training courses (Darling-Hammond & Hyler, 2020, Guskey, 2021, Tucker & Quintero-Ares, 2021), not the teaching of technical subjects. Those that do discuss the impacts of teaching technical subjects do so from K-12 or higher education perspectives (Webb et al., 2021). Our research provides a unique three-fold take on the effects of the COVID-19 pandemic in an education environment because of the (1) educational material considered (data analytics), (2) the education level considered (professional learning), and (3) format of learning (one-week short courses). This article also demonstrates the benefit of self-study of professional practice for the engineering and broader non-teacher education communities.

This article documents the outcomes of a self-study of professional practice by a group of engineers seeking to understand the impacts of converting a data analytics short course from an in-person to an online learning environment during the COVID-19 pandemic. Specifically, we sought to answer the following research question: How do engineers understand and adapt to challenges associated with transitioning a short-term professional course online? The next section provides some background on data analytics, as well as a discussion relating to the impact of the COVID-19 pandemic to engineering and adult learning. We then provide some context to the course and the researchers. Afterward, we introduce self-study as the methodology used in this work, making its case for assessing rapid educational changes such as those encountered with COVID-19. Results are presented and a discussion is given on how the outcomes have been applied to decisions for subsequent courses, implications for others transitioning courses online, and the applicability of self-study to assessing rapid educational changes in non-teacher education disciplines in general.

Review of Literature

This section provides some background information on data analytics and the COVID-19 pandemic’s impact on engineering education.

What is Data Analytics?

Data analysis involves procedures for analyzing data, techniques for interpreting the resulting findings, and methods for improving the analysis process (Tukey, 1962). Data analytics is the holistic use of empirical data to gain insights and support decisions (Leathrum et al., 2020). It includes data analysis but also encompasses data mining, data management, data modeling, and machine learning. Data analytics allows people to better make sense of their data.

Teaching data analytics involves being able to provide hands-on exposure, and the ability to generate useful and interpretable data visualizations (Lynch et al., 2021, Weirich et al., 2018). Key topic areas within data analytics include exploratory data analysis, predictive modeling, computer-assisted data analysis, and programming. The combination of these topics has pushed the need for data analytics in practice beyond the boundaries of traditional spreadsheet-based analyses. As such, conducting modern data analytics requires versatile analysis environments and tools such as statistical programming languages or specialized software (Qasim et al., 2020). These tools tend to have a steeper learning curves but, as educational tools, provide better environments to learn advanced analytics concepts.

COVID-19 Pandemic’s Impact on Engineering

The COVID-19 pandemic has clearly had a significant impact on higher education. Engineering presents its own challenges, as highlighted in the American Society for Engineering Education’s report, COVID-19 and Engineering Education (American Society for Engineering Education, 2020). Many studies on the impact of the COVID-19 pandemic on engineering preparation have been performed; generally, these studies involve surveys of students and/or faculty across a range of institutions (e.g., Backer & Chierichetti, 2022, Hidalgo López et al., 2022). Others presented the results of applying different learning paradigms or introducing new technologies, either learning technology or software, to support learning outcomes (e.g., Felder et al., 2021). The consensus was that the COVID-19 pandemic highlighted issues with online engineering preparation but that faculty and students adapted quickly to a workable environment.

From the student perspective, Hidalgo López et al. (2022) found that while students preferred face-to-face instruction, they valued online material and hoped to have continued access to it after returning to face-to-face study. Backer and Chierichetti (2022) identified students’ mental health and time management as major issues. Prada (2021) reported that students believed online education was not sufficient to bridge the gap between theory and practice in engineering.

Technology usage was also a common theme among studies of COVID-19’s impact on engineering education. Oliva-Cordova et al. (2022) surveyed faculty to assess their satisfaction with the tools for online learning, with some reservations but general usability. Felder et al. (2021) highlighted the benefit of creating an online lab experience that fosters student interaction.

COVID-19 Pandemic and Adult Learners

The consensus is that adult education is less accessible and the learning environment more challenging (DeMartino, 2021, James & Thériault, 2020, Kwapong, 2022, Singh et al., 2021). The pandemic placed an economic burden on adults forcing them to put job opportunities – when available – over educational opportunities. When educational opportunities were available, the ‘learn from home’ environment posed more difficulties due to the stresses and distractions at home (Backer & Chierichetti, 2022). The pandemic created general stress, caused mental health issues, and made it more challenging to be productive. Children were also at home, creating more distractions and impacting time management.

Fortunately, the course environment discussed in this article avoided some of these issues. In particular, the employer promoted the courses, encouraging employees to take the courses as part of their normal workload. As such, there were no financial issues and conflicts with ongoing work beyond those that existed pre-pandemic. However, the ‘learn at home’ model, especially with children present, was an issue and is discussed in this article.

Context

The course used in this self-study is one of a series of week-long courses in data analytics developed for the United States Navy. The context of the work is explained from four perspectives: organizational, content, teaching, and technology. Each of these perspectives brought its own challenges to teaching the course in a modified format, given the COVID-19 pandemic.

Organization

The sponsoring organization for the project was the Naval Sea Systems Command (NAVSEA), which wanted to train its analysts in the emerging field of data analytics. This was driven by a need to better analyze the large quantities of data generated, collected, and available to NAVSEA; with the course having students from different branches of the Department of Defense (which brought its own challenges not discussed in this paper).

The sponsoring organization was a partner in the development of the course and was developed over years of interaction between the instructor team and the organization. Their main influence was on the material, the tools and programming language used in the course; for example, the sponsor did not want Python or SQL covered in the courses and so these elements were removed. After the development of each course in the series, a prototype course was conducted with the sponsor present. This gave the sponsor an opportunity to provide practical feedback which was usually in the form of rearranging elements of the course and requesting more practical examples over theory.

The project was originally designed to span six years, with eight week-long courses, each taught to approximately 20 in-person students, in a given fiscal year, at various locations on the east and west coasts of the USA. Each year, a new course was developed and prototyped as one of the week-long courses for the sponsoring organization, aka the ‘customer’. Originally, the intent was that each new course would build upon previous courses, allowing students to return to further develop their analytic skills. The courses each explored a different facet of data analytics. Course titles, in order, are An Introduction to Data Analytics, Predictive Analytics, Data Modeling, Data Management, Enterprise Architecture, and Modeling and Simulation. The context for this article is the third data analytics course: Data Modeling.

Due to the nature of how project work is handled at the host institution, Old Dominion University, conducting the courses was considered beyond the instructor’s normal teaching requirement. This required intense planning sessions to ensure the schedule was compatible with ‘customer’ and ‘instructor’ needs. Currently, most courses are taught in the summer when instructors do not have their regular teaching load. Pre-COVID, each course was taught with two instructors and one support staff, with one instructor acting in support while the other taught. Administrative support staff ensured that instructors were not distracted by non-teaching concerns or problems. This was especially important at off-site locations. This teaching model proved effective, as determined by the continuation of financial support by the customer.

Rapidly adapting to the pandemic forced a rethinking of administrative strategies. Recognizing that remote learning had more distractions for learners and instructors, it was decided that the teaching of a single course would be distributed across all instructors. This had the benefit of reducing the heavy load on instructors of home distractions whilst teaching. This approach generally kept each instructor in their comfort zone as they only taught, with a few exceptions, material they had developed.

Content

The data modeling course covered the following concepts: a review of the R statistical programming language, data wrangling, mechanistic models, exploratory data analysis, empirical model building, multivariate regression diagnostics, stepwise regression, and missing data. The lecture material covered the related theory as well as how to practically apply the concepts in the R programming language. Computer ‘laboratory’ time was dedicated to ensure students were able to apply methods on their own. For assessment, each student was expected to choose a dataset of interest and apply the techniques learnt to that dataset, presenting their approach and findings orally on the last day of the course.

Teaching

Each course took five days, organized as four 1.5-hour modules per day. Time was allocated for a practicum experience with students presenting their results at the end of the course. The described experience with the data modeling prototype course was used to evaluate the material and assess the ability to continue the courses in a remote format during the pandemic. The need to modify the learning experience began with the university restricting travel and the size of gatherings in the pandemic. Previously, prototypes were taught locally. Because of the pandemic, the prototype was taught in a hybrid format, with less than half the students taught in-person, and the rest remotely, generally from home. All instructors except one chose to teach remotely despite the presence of students on site. One instructor chose to teach one day in person and one day remotely to compare the experiences. Since the prototype, all courses have been taught fully remotely.

Technology

During the on-site courses, presenters had two large screens, one for slides and one for R code, as well as a whiteboard for questions. In-class assignments allowed instructors to provide direct assistance. In addition, half-hour breaks were provided between modules and an hour for lunch, allowing sufficient time to address more significant problems. Each student was expected to provide their own laptop with the open-source software tools, R and Rstudio, installed. Thus, the environment was set up to lecture students as well as allow them to engage in computer workshops throughout the week. In both cases, the setup was equivalent to the expectations of the instructors from their previous working experiences.

The Instructors

The course had a total of six instructors who came from various business units at Old Dominion University, a large public university in the eastern United States, specifically academic engineering departments. Three of these instructors (Andy, Jim, and Chris) took part in this self-study. These three instructors took on the roles of facilitator, instructor, and technical support specialist at different points in the course; hence, they were involved in all aspects of delivering the course. Specifically, Andy, an assistant professor, led the facilitation, Jim, an associate professor, the instruction and course development, and Chris, a lead project scientist, the technical support. The instructors had extensive online teaching experience at the graduate and professional levels, and all had taught in web-based and blended learning environments prior to the COVID-19 pandemic. The project was the first time the group had taught together.

All the instructors had been with the project since its inception in 2018. A mutual acquaintance introduced Brandon, a professor of teacher education and the second author, to the group and recommended his inclusion due to his expertise as a self-study researcher. Brandon served as critical friend for the three engineers. Brandon’s role was crucial because this was the first time the instructors involved had conducted a self-study. Though conducting a self-study can be intimidating for those who have not undertaken any reflective research before, the instructor team was confident of success, at the beginning, because they believed they had been effective in working together for several years and were prepared to face the challenge together.

Methods

Self-study of professional practice is an improvement-aimed research method framed through personally situated and critically collaborative inquiry (Samaras, 2011). It has roots in action research and the concept of critically reflective practice, originating in the field of teacher education in the early 1990s as self-study of teacher education practices. In recent years, researchers have extended self-study research beyond teacher education, with some favoring the term ‘self-study of professional practice’ as it is more encompassing of professional learning in higher education (Pithouse-Morgan & Samaras, 2015).

Ragoonaden et al. (2018), reflecting on their interdisciplinary explorations of practice as a teacher educator and nursing educators, found that self-study afforded them opportunities to engage collaboratively while improving individual and collective teaching practices, developing understandings of self, and making public empirically-grounded research outcomes related to teaching practice and faculty learning. They noted, ‘Campus-wide reflective communities of practice, situated into inquiry-based models, sustained with contributions from interdisciplinary colleagues, can provide spaces where pedagogical practices can be carefully planned, [and] continuously revised’ (p. 82). In engineering, Nilsson (2010, 2013) identified self-study research methods as one way to encourage engineering educators’ scholarship of teaching and learning. In her capacity as a critical friend, Nilsson supported six engineer educators in their learning of self-study to develop their pedagogical practices associated with machine engineering. Based on her work, Nilsson (2013) argued that ‘collaboration through collegial conversations and reflections might lead to new insights about the problematic nature of (engineering) teaching’ (p. 205).

Like Nilsson, Brandon served as a critical friend to a group of engineers (Andy, Chris, and Jim) in this self-study. Critical friendship is a common reflective and methodological tool used in self-study research. According to Swaffield (2008), a critical friend is:

[S]omeone who provides both support and challenge within a relationship … is a detached outsider who assists through questioning, reflecting back and providing another viewpoint, prompting honest reflection and reappraisal, a seeing anew that may be challenging and uncomfortable, yet enhancing. Critical friends are concerned with both the learning of the person or people they engage with directly, and the success of whatever project is the focus of the work. (p. 323)

In self-study, the use of critical friendship allows researchers to ‘act more wisely, prudently, and critically in the process of transforming education’ (Carr & Kemmis, 1986, p. 161). With higher education faculty often feeling isolated regarding their teaching practice and professional growth, critical friendship affords faculty the opportunity to connect with, and learn from, one another (Russell, 2022).

Data Collection

We first met as a research group in March 2021 to determine the study’s focus and data collection procedures. In that meeting, Andy, Chris, and Jim expressed their interest in better understanding their teaching of the course that had shifted online due to COVID-19. Brandon suggested self-study as an avenue to simultaneously understand prior practice and improve future practice, to which we collectively agreed. The instructors were initially skeptical because reflective research is not well-regarded within the engineering community. Andy believes is due to the focus on quantitative methods within engineering.

We acknowledged that as newcomers to self-study research, Brandon – in his role of critical friend and experienced self-study researcher – would provide Andy, Chris, and Jim with resources to learn about self-study, and we set aside time within the meetings to unpack that learning. As such, early meetings were divided between learning about self-study, providing Brandon with necessary background information on the project, and beginning to problematize and reflect upon the challenges and opportunities provided by the shift to online teaching. Having developed a working knowledge of self-study and the project, we shifted focus in later meetings to critically interrogating the teaching and learning that occurred in the data modeling course and generating new understandings of practice.

We met seven times between March and June 2021. Meetings were between 30 minutes and an hour in length. These meetings were audio-recorded and transcribed, totaling 50,225 words. In our one May meeting, we identified a series of reflective questions to which Andy, Chris, and Jim responded in writing. These questions gave them an opportunity to provide additional background on the project, consider their interactions with project personnel, anticipation for the course, project operations, course enactment, and post-event reflections. Initial responses were completed in early June. Andy, Chris, and Jim then engaged in an iterative dialogue in journal form continuing through September where they posed questions to one another and provided additional insights into their experiences with the course. The total word count for the three journals was 24,610 words. The informal nature of these meeting helped overcome some of Andy’s nervousness regarding self-study and provided a safe environment for discussion; the new method of interaction enabled Andy to begin to see the benefit of self-study.

In July 2021, we held one final meeting. This meeting was structured as a reflective interview with Brandon as moderator. Prior to the meeting, Brandon revisited transcripts from previous meetings and journal entries completed to that point. He generated a set of guiding questions that encouraged Andy, Chris, and Jim to reflect further on the data analytics project, their individual and collective responsibilities in the project, what they learned from teaching in the project, what they learned from engaging in the self-study, and how they might improve their teaching practice – and the data analytics courses specifically – moving forward. This reflective interview lasted an hour-and-a-half, was audio-recorded and transcribed, with a word count of 15,429. In total, the complete data set – transcripts from seven meetings and one reflective interview, and three journals – totaled 90,264 words.

Data Analysis

We shifted to data analysis upon completion of data collection, entering the analytic process inductively as we preferred to keep an open mind regarding data related to the transition of the course from face-to-face to an online format. To gain insight into the actions associated with transitioning the course, we used process coding as an appropriate first cycle approach to coding. Saldaña (2021) noted that process coding highlights the ‘actions intertwined with the dynamics of time, such as those things that emerge, change, occur in particular sequences, or become strategically implemented through time’ (p. 144). This first cycle of coding produced 182 process codes. Reducing for repeaters brought the number of codes down to 158.

We then sought to generate themes through a second cycle of analysis using focused coding. Saldaña (2021) states that focused coding, also known as selective or intermediate coding, allows researchers to ‘search for the most frequent or significant codes to develop the most salient categories in the data corpus’ (p. 303), and to engage in hierarchical ordering, through which categories and sub-categories emerge. This second cycle of analysis resulted in three categories. Each category consisted of two-to-four sub-categories. An overview of categories, sub-categories, and representative process codes can be found in Table 1.

Table 1. Categories, sub-categories, and sample process codes.

Category	Sub-Categories	Sample Process Codes
Adapting to Online Teaching: The Implications of COVID-19 on Course Design and Instruction	Adapting course to online environment Adapting course content to online environment Adapting course instruction for online environment Feeling the impact of project adaptation	Being forced into online environment Anticipating challenges in online transition Adapting online structure Seeing need for curricular redesign Struggling with instructional pace Identifying changes in teaching online
		Identifying the importance of communication
		Learning difficulties in online environment
Shifting the Course Online: Acknowledging the Personal Dynamic	Adapting course content to client needs Seeing need to adapt as instructors	Identifying client challenges Conflicting perspectives between client and institution
	Accommodating student needs	Teaching as organic and adaptive activity Identifying work/life issues for instructors
		Feelings of participant apprehension Perceiving lack of participant engagement
Adapting to the Technological Demands of an Online Course	Adapting to technology demands Contrasting face-to-face and online teaching	Feeling concerned about technology Expressing concern about transition to online
	Identifying and experiencing technology issues Implementing technology effectively	Experiencing challenges in hybrid environment Conflicting demands on online and F2F participants
		Experiencing technical challenges Experiencing student technology issues
		Identifying success with technology Using digital tools in online environment

To ensure trustworthiness and validity, LaBoskey (2004) argues that self-study researchers ‘must make visible our data, our methods for transforming the data into findings, and the linkages between data, findings, and interpretations’ (p. 853). We believe the descriptions provided above highlight our methods to transform data into findings, and readers will find a rich description of data in the following sections. But we also acknowledge other conceptions of how trustworthiness can be achieved in self-study, such as Mena and Russell’s (2017) recommendation of data triangulation and critical friendship; both of which were used in this study.

Findings

As we examined the identities and practices of Andy, Chris and Jim as engineers in relation to facilitating an online analytics course during COVID-19, three themes emerged from the data: the implications of the COVID-19 pandemic on course design and instruction; the impact shifting the course online had on clients, instructors and students; and adaptation to the technological demands of a newly online course. We unpack these three themes in this section.

Adapting to Online Teaching: The Implications of COVID-19 on Course Design and Instruction

In this theme, we highlight the transition to online instruction due to the COVID-19 pandemic, and how the pandemic necessitated an adaptation of the course from a face-to-face model to a hybrid model that included a mixture of face-to-face instruction and synchronous online learning. The project was initially designed to be taught in person at various locations across the United States. With the sudden onset of the COVID-19 pandemic in March 2020, the continuation of the project was at risk. Andy, who served as project lead, noted that, ‘the financial customer wished to cancel the course indefinitely until after the COVID-19 pandemic and their financial restrictions were resolved’. He noted that the university and project team sought to continue the project because ‘significant resources had been used to develop the prototype course … and concerns of losing the highly skilled, and in demand, instructor pool’. Initially, Andy wanted to facilitate the project remotely so that all instructors and students would be virtual; however, the client desired a hybrid model. Although the project team facilitated this shift, the transition was not easy, as Chris noted, because ‘the course content had already been developed’ with a face-to-face learning environment in mind. As project director and facilitator of the in-person learning experience, Andy was nervous about having to run the in-person section as it was the first time he had returned to the workplace since the pandemic began.

The pandemic forced changes to the course besides virtual instruction. For face-to-face instructors and students, protocols were implemented that limited the capacity to 10 for a conference room designed for 50 individuals, masks were required, a six-foot distance had to be maintained, and the project could not provide food or drinks for instructors and students. The 10-person limit forced most people online, and instructors found it difficult to provide directed feedback to students when they could not work with them side-by-side. In addition to adapting to pandemic protocols for face-to-face participants, there were challenges identified with the hybrid learning environment. In one meeting, Andy argued that concurrent teaching, the simultaneous teaching of in-person and online students, was ‘the worst way to teaching anything. You have to accommodate both teaching styles (in-person and online) but do not get to use the best parts of either’. Jim responded, ‘I wouldn’t undervalue the benefit of visual cues for immediate feedback’ available from in-person students, but not online students. Adding to the discussion, Chris replied, ‘in the hybrid setting, I feel that the in-person participants are likely to drive the discussion and the pacing due to the visual cues that can be directly assessed if the instructor is also in the room’. This issue was compounded by the fact that all three instructors preferred the relational elements normally found in in-person classes.

Having already taught the previous courses in the project face-to-face, the team had to anticipate issues that might emerge in the transition online for the course. For instance, Jim noted that course material had to be ‘polished’ by ‘more tightly coupl[ing]’ the presentations with the technology and coding experiences. He believed that teaching online would ‘reduce the ability to adapt on the fly’. As such, there was concern that students would struggle with retention due to issues of pacing, engagement, or support. However, there were certain benefits to shifting online. Because travel was not allowed due to the pandemic, in-person instruction took place at the university. Shifts in budget costs, such as no longer having to pay for instructor travel costs, allowed the project team to hire additional instructors who could provide additional support for participating students. But, the move to online instruction presented certain complications, like one Andy noted: ‘the ability to handle technical problems is decreased so if technical problems are faced then I see some real issues occurring’. He identified the additional instructors hired as one way they overcame this problem.

Perhaps the most significant adaptions in the course were pacing, and the amount of content taught. Andy stated that ‘we tried to stick to what we originally had planned’, however, the project team understood that online learning moves at a different pace than in-person learning. For instance, Andy noted, ‘the key lesson for me was that we will need to be able and willing to “water down” the course material … The focus has to be on clarity not quantity’. Additionally, instructors often had a more difficult time controlling online student learning experiences. Students might shift tasks or foci in ways not observed during in-person coursework. Such experiences had to be considered by the team during course design. Jim stated, ‘Structuring the content and code so that if a student needs to step away, they can at least quickly run the code to catch up to the current position makes it so they do not get lost easily’.

One facet that carried over from the original course design was the daily schedule. Andy noted the benefits of their existing approach,

All courses were built with a very clear daily schedule which included lots of breaks. Though the customer pushed back on this, it turned out to be a great asset in the hybrid classes as it allowed us to ‘touch base’ more often with the online students and deal with technical issues without students losing class time.

He also highlighted that the team acknowledged the potential for technical issues, therefore, they had ‘previously built lots of guides to get the software working. These guides were invaluable when dealing with software problems online as we could not take over control of [student] computers to fix the problems’. Chris believed they achieved success in the online transition due to the ‘large amount of effort that content developers put into every module to allow their content to be teachable by any of the team members’. Having taught together for the previous years of the project, Andy, Jim and Chris believed they had a good feel for each other’s teaching style. However, engaging in self-study allowed the group to openly express their instructional strengths and preferences that had not previously occurred, something we highlight in the next finding.

Although the team highlighted several successes in adapting the course online in a relatively quick manner, they also noted areas for improvement. In our data, we regularly discussed finding a balance between teaching new content and reinforcing prior learning through discussions and experiential exercises. At one point, Jim said, ‘We need a better way to solicit questions during lectures’. In response, Andy argued, ‘If we always allowed a question session after each break, and did not rush past it, this would work’. Chris added, ‘I also believe that to achieve this we need, and should, reduce content in many modules to accommodate this’. Adding time to ensure student understanding or address technical issues reduced the amount of content that could be covered, but the decision to provide these necessary supports limited the impact of larger issues that could negatively impact the overall learning experience. Andy, as project director and someone with an overarching view of the course, believed that it made the course more enjoyable for instructors and students.

Shifting the Course Online: Acknowledging the Personal Dynamic

In this theme, we explore the transition, to an online format, of the course considering the COVID-19 pandemic and its impact on students, instructors, and client. We begin with the shift from the client perspective, highlighting the team’s ability to balance the client needs, COVID protocols, and instructors’ knowledge of best practices, and the need to adapt the course to an online environment. As we previously identified, the customer initially wanted a fully face-to-face program. Andy noted, ‘There was a conflict between the technical customer and the ODU team due to the technical customer wanting to run the course at full capacity’. Pandemic-related policies delayed the start of the course, and the team was eventually able to convince the customer to use a hybrid approach to teaching and learning.

There was a question as to whether the customer would accept the new hybrid format and deem it acceptable for the future. For project leaders, experience with online instruction helped make a case for changing the mode of instruction rather than delaying the course altogether. Although the two sides came to an agreement regarding the mode of delivery, there were some disagreements regarding the content delivered. Andy noted that, ‘The technical customer wanted the course to contain as much advanced data modeling material as possible, at the cost of overview material and basics; this was driven by a need to show value for money’. Alternatively, ‘the instructors, based on the experience with the student body, wished to include overview and review of basic material out of a concern that some students would feel “lost” otherwise’. Andy, because of his participation in the self-study, came to accept that he was overcautious due to his fear of leaving any student behind.

For the team, there was a shared belief that experience mattered, but not all experiences were equal. Andy noted that one instructor, who was not a part of this study, struggled with the conversion to online instruction. Jim highlighted his approach to teaching, based on the Socratic method, was contrary to the highly structured learning environment of the online course. As a result, Jim found the ‘the development of the material, not the presentation of the material’, more interesting. He added, ‘The presentation – I think it’s boring, the development of the material is the interesting part and that’s why I’m still participating … ’ Andy was surprised by this admission from Jim as he previously thought Jim enjoyed teaching the project material. Jim’s expression of his preferences and uncertainty about his place in the project resulted in his roles and responsibilities changing in following iterations to better align with his interests.

Andy had extensive online teaching experience he could draw on in the design of the project, but as project lead, he had few opportunities to teach in the course, which he found disappointing. He was also concerned about the new online learning environment negatively impacting the course, noting, ‘I’m worried … that the students aren’t “getting it”, they’re just going to switch off and go do something else, while they’re just playing us in the background’. This uncertainty created hesitations for Andy, ‘I’ve been teaching 17 years but in this project, I feel the need to push myself back to being a novice in terms of my understanding and what I’m going to do in this new environment’.

Chris’s key role in the project was to provide technical support for students, which reflected his expertise. He noted that prior teaching experience was not ‘the only criteria necessary for a successful [online] conversion; knowledge and experience in interactive, online teaching is also needed to help prevent technical issues from detracting from the learning experience’. Chris believed that if enacted effectively, course instructors could use the online learning environment and pedagogical approaches to ‘direct participants’ focus on the primary takeaways’ of the course.

The pandemic presented several personal challenges for instructors and students. A primary consideration was the need to find work-life balance. Many instructors and students were now working remotely from home; some had family members and small children. Andy highlighted one advantage of the online shift for instructors, noting that,

During the normal prototype the six instructors are present. This is a huge burden for six busy academics. However, the customer would notice if an instructor was not there. The online format allowed the instructors to come and go more easily so they could deal with their other projects and demands.

The project team also accommodated student needs. Jim stated that the ‘assumption was that students were 100% vested in the course like when in-person, but having to deal with kids, etc. as distractions and trying to schedule to give them proper time’. Chris noted, ‘We received a request at the start of the course from students to modify the break schedule in order for them to better navigate their at-home responsibilities because their children had also transitioned to at-home status’. Although students benefited from these breaks, there was a feeling breaks were a hardship on the instructors because they had to create artificial stopping points throughout their teaching. This problem led to a discussion that presented several solutions:

Jim: … You need to adapt quickly and find convenient break points on the fly. The ‘rigid’ structure we started with might have made this look more choppy than normal, but the team handled it well …

Chris: Knowing this now, we can incorporate this into our future schedules and have our breakpoints situated in advance. Additionally, we can add in specific break slides within the presentations that are natural points for polling the audiences to get a feel for understanding and progression …

Andy: I think an interaction with students is good. One thing that I do in my lectures is ‘no questions in a break, but after every break [there is] time for questions’.

The three instructors perceived a sense of apprehension on the part of students at the beginning of the course as well as in the instructors. We attributed this to the ongoing pandemic and concerns related to family and health safety, and a need to find work-life balance while engaged in a strenuous professional learning course. However, Jim noted that, ‘By the end of the week, students seemed very comfortable, exploring the capabilities of the [learning] environment on their own, sometimes to the detriment of spending time on the material’; this sense was shared by the instructors.

There also existed challenges in building relationships with both in-person and online students. Andy found himself having to be the pandemic ‘rule enforcer’ as in-person students became increasingly blasé about social distancing and mask-wearing as the week proceeded, this included some of the instructors present, which Andy found disappointing. These same requirements, as Andy mentioned, also made it ‘difficult to socialize with the students at lunchtime, even those that were in the class, [and] this made the course, quite honestly, less fun’. Working with online students, the combination of pandemic policies and online instruction meant that ‘relationships could not be built with students online over a week and, as such, it was difficult to understand their situation’.

Adapting to the Technological Demands of an Online Course

In this final theme, we focus on the necessity of shifting the course online and the implications for the instructors as they adapted technology use and implementation. When the pandemic started, the group found it easy shifting to remote work as each had access to appropriate technologies. Jim noted it ‘was easier to teach at home from a technology standpoint than to teach at [the university]’ due to his technology setup. Chris echoed this sentiment, saying, ‘I wasn’t impacted too heavily. I was already set up to be able to work remotely effectively. I already had two computers that were specifically set up’. For the team, access was not the concern, it was the use of unfamiliar digital tools. The university had begun moving from WebEx to Zoom when the pandemic struck, and as such, the team had experience with Zoom. However, it was decided to use Microsoft Teams for the project, Jim noted, as the customer was ‘pretty adamant about Teams, because that was the only environment that they had approval on’. For the group, the move to Teams was more about customer comfortability as instructors had limited exposure to the software; though it turned out that the students were comfortable with Microsoft Teams and many were more experienced than the instructors.

With the project moving online, Andy – as project lead – was most concerned about student technology access beyond the Learning Management System (LMS). His concerns were ‘the inability for students to connect to the course due to hardware issues, and the inability for students to use the technology at home’. This was a particular concern for online students as some were using employer-provided computers with outdated hardware and software. Jim echoed this concern, but from a different perspective, as the prototype course worked effectively when students had access to multiple monitors. Initially, the thought was that two monitors would be sufficient, but students ‘pointed out that it would not solve the problem as they really needed three monitors, one for [presentation] slides, one for instructor code (RStudio), and one for their own code (RStudio)’. Many online students only had access to one monitor, which impacted their ability to keep pace with instructors and practice their work. Chris identified a clear difference in terms of technology challenges for in-person and online students:

I felt that the students participating from the on-site location were able to follow along about as well as in the normal course setup, but with increased difficulties along the communications routes for the online instructors and students. For the students participating online, (1) assistance time for code and software setup were increased versus the normal course layout, (2) the ability to follow along while using a single monitor was hindered, and (3) for pacing purposes, our ability to gauge where the students’ progress was during the modules was almost non-existent.

Chris added, ‘I felt the delivery of the course met the minimum requirements of the stakeholders, but with room for improvements in most areas’.

Jim argued, ‘With proper technology, I am a big proponent of the hybrid mode [of instruction]. It provides more immediate feedback to the instructor in terms of visual cues’. However, the reality was far different in the course offered. Students’ access to microphones was an issue, and ‘none of the software required was installed on the provided computers’. Jim noted that if an online instructional environment was to be used in the future, there was a need for ‘testing the environment to make sure it works as expected before the instructor walks in’. In addition to technology access, internet connectivity presented a problem for students. Some experienced bandwidth issues, which resulted in a struggle to engage with instructors and fellow students, leading to what Jim identified as ‘an unprofessional experience’.

It is worth evaluating some of the benefits and challenges in identifying and navigating an effective LMS. As we noted, the project team was asked by the customer to use Microsoft Teams instead of other LMSs and video conferencing software. Chris noted that,

Microsoft Teams was selected as it provided the necessary audial, visual, and textual chat capabilities for instructors and students, it provided the creation of channels for aid in organizing course materials and practicums, and it allowed both ODU and DOD emails to access the same team spaces.

Jim argued, ‘I am not sure that Teams would be my choice based on your points over say Zoom except for the last point’. He continued, ‘I thought the integration of the communication environment with the organization/storage of files was the biggest benefit’. Although Microsoft Teams provided certain benefits not identified in other LMSs and video conferencing software, there were issues with navigating Teams. As the technical support for the course, Chris found Teams ‘more tedious to navigate for group activities than the in-person learning environment. However, I was pleasantly surprised that it was much less difficult to navigate, facilitate communication, and greatly improved file sharing options than expected’. Jim pointed out that the system was not adaptative once the course was enacted because to reorganize content and learning experiences would require ‘a major effort’. As such, the use of Teams required the team to be certain about course design and enactment prior to start.

During the course, the team identified ways to use digital tools effectively in the online environment. For instance, Chris found that file sharing was ‘smoother than anticipated’. He noted that some students effectively used channels to ‘directly post their feedback for each day’s content’. Although not consistently used during the course, Chris added that the use of polls appeared advantageous ‘for facilitating responses in future iterations of the course’. A clearer design and use of the Teams space was required to maximize efficiency and the student learning experience, but the opportunity to revisit the course through self-study provided the opportunity to identify several solutions for using technology effectively in the course. There is some level of irony with the instructors’ reticence with Microsoft Teams at the outset, as they grew so familiar with the LMS that they preferred its use for data collection for this study.

Discussion

Online professional learning offers many advantages, from reducing time spent on travel to the convenience of students being able to work from a familiar environment. However, there are also disadvantages, including reduced instructor-team communication, workspace distractions, and difficulties overcoming technical problems. As such, many changes to course development and deployment were needed to adapt an in-person course into an online course.

At submission, 13 online week-long analytics short courses have been completed since the completion of the prototype course discussed in this article. Our self-study of the prototype course provided a starting point for the improvement of these online courses. The self-study provided the breathing space the instructors needed to think and reflect, which, they believe, would not have happened otherwise. Implications from our study are discussed below, following the three themes covered in the findings section. The discussion includes how the lessons have been applied to subsequent course offerings as well as to the development of the next course in the sequence. In addition, we consider the implications of our work for engineering educators, including implications relating to the conduct of self-study itself.

Implications for Transitioning Courses Online

There are five main implications from this study related to transitioning short courses online especially those that are technology related, which we discuss in turn. These can be briefly summarized as: adaptability, pedagogical action, personal conflicts, technology access, and an effective LMS. Following the format in the findings section, we align these implications with the three themes.

Adapting to Online Teaching

The group has years of experience teaching hybrid courses in a classical semester environment, but techniques for successful student engagement in that environment (e.g., flipped classrooms, discussion groups) are difficult in the compressed online short course. This was not an issue when teaching courses in-person because it was easier to engage students as they were more willing to ask questions. Instructors have experimented with techniques to create an effective Question & Answer environment. One example was Andy’s approach to resume after breaks with a Q&A, keeping the breaks as ‘breaks’, but giving and encouraging interaction time. In another example, Jim built in more exercises where students are broken into groups to work together in virtual breakout rooms. Instructors floated between rooms answering questions, which was an attempt to replicate laboratory experience in the online environment.

The lesson here is that there are many adaptations that need to happen when converting from a face-to-face (F2F) to an online environment (Cutri et al., 2020, Wang et al., 2021). Such adaptations are necessary and influence every aspect of the pedagogical enterprise, including the need for reduced pacing, accommodations that slow the learning experience, and technology changes that impact student engagement (Barrett, 2010). The point here is that instructors cannot simply recreate F2F teaching and learning in an online environment. There is a tendency for educators to prefer past teaching practices and to resist new technology requirements and best practices when transitioning online (Tondeur et al., 2012), in contrast to determining how technology can best be used for the learning experience. As we move away from the pandemic, these ‘old’ teaching practices have started to creep back into instruction. To counter this, Andy now engages in a reflective session in all his courses to better understand the technical programs that students are currently using.

Personal Dynamics

As part of the online adaption, we anticipated a need to reduce course pacing. This increased the time allocated for conducting exercises but reduced the amount of material taught. However, during the prototype, it became obvious that the course schedule would need to be adjusted to accommodate student learning needs. This relates to a common theme in the literature on adult learners’ accessibility to learning opportunities (Burton et al., 2011). An online environment still presents challenges in how to create appropriate learning opportunities as opposed to an in-person classroom. Consideration of the personal needs of instructors and students was present throughout our study, and reflects current understandings of the necessity of personalizing the online teaching and learning experience (e.g., Wozniak, 2020)).

For example, we adjusted the rigid daily, four 1.5-hour sessions with half-hour breaks in the morning and afternoon and a lunch break to more fluid morning and afternoon sessions with shorter, more frequent breaks. This helped instructors maintain ‘flow’ in their instructional efforts and set stopping points based on their professional teaching experience. Course facilitators now track the time and recommend breaks if the instructor loses track of time. An attempt is also made to ensure a single instructor covers a morning or afternoon session to ease the transition between modules because a break may not fall at that time. From a program management standpoint, Andy believes that the break schedule helped with handling the fatigue induced from being in class for a full eight-hours a day.

The need for reduced pacing and break flexibility did result in conflict with the customer, who wanted to ensure that the maximum amount of educational material was covered. However, due to the instructional team’s extensive teaching experience, we successfully argued for pedagogical actions that positively supported student learning. This implication is unique to professional-level courses funded through contracts, and future research is needed that explores the tensions and relational dynamics that exist among the sometimes-competing demands of customer interests, student needs, and best teaching practices. Instructors believe using reflective practice with the customer might help in reaching a common understanding.

From a relational dynamic, we introduced an administrative team member to act as a facilitator for each course. The duties of the administrative support staff are to facilitate the non-pedagogical activities; this included managing breaks, dealing with minor technological issues students might have each day, and ensuring the course remained on schedule. Another key duty of the facilitator is to help encourage students to see instructors as a friendly and approachable group of individuals. The intention behind this activity was to ‘release the energy’ of the students (Knowles et al., 2014), especially in encouraging student participation. Instructors felt this made the course more engaging for all involved. Additionally, the online format allowed instructors to come and go from the course; however, it was critical we maintained a solid schedule to ensure that extra technical support was available if required. This schedule provided an anchor for the instructors and students to work around.

There is a need to acknowledge that students and instructors exist as individuals with their own conflicts and challenges separate from the course. It is imperative that online courses are designed to support them effectively. As Crompton and Sykora (2021) stated, technology change should empower students’ learning, which, by implication, means it should not hinder learning. Andy now incorporates these personal concerns more readily in short course development, and in his undergraduate and graduate instruction at the university.

Technology Demands

A key demand when dealing with technology in the classroom is the need to provide technical assistance. In our study, we anticipated that long breaks between modules would provide time for technical assistance, but the break structure was modified to accommodate students’ preferences. Several approaches have been implemented to better support the students’ need for technical assistant since the course, used in this self-study, was conducted. First, a dedicated instructor is available to meet students in breakout rooms for emergency technical support. Second, previously developed pre-course instructions for software installation have been revised and strongly encouraged to be completed prior to the course. Third, problematic software has been removed from the learning experience. Concepts pertaining to the removed portions are still introduced, but the results of the exercises are now shown, not actively executed during lecture, to reduce the cognitive burden and lengthy run times.

We found that any technology used must be accessible by students and instructors for an effective learning environment to be created. This accessibility might change over time, for better or worse, as technology changes. This need to be dynamic in accessing the technology usage is an argument against using the already mentioned standards, which could become formulaic in their application (Serdyukov, 2020). Obviously, accessibility means that resources are available to acquire the necessary technology for students and instructors (Jovanovic et al., 2022); however, the COVID-19 pandemic might be the catalyst needed to get those resources (Campos-Remon et al., 2021).

In a project with multiple instructors, communication is an important factor in achieving student success. Prior to COVID-19, instructors had regular in-person communication, with the capacity for conversations to take place ‘in the moment’ during instruction. With the shift online, we needed to identify new ways to remain connected. For example, instructors were encouraged to use a dedicated private social media channel (through Microsoft Teams). Also, pre-meetings between instructors become more important due to the lack of capacity to adapt of online courses. Communication with students also became more problematic and, as such, we highly recommend using a facilitator or ‘master of ceremonies’ for courses to set a jovial and friendly tone at the start of each day. We have found this approach highly effective in avoiding the wall of silence that can occur in the online format.

Thus, MS Teams was an effective LMS for our purposes, mainly because the students were experienced with it. We realize that we were fortunate in this experience compared to others (Webb et al., 2021). Though many resources were made readily available to institutions, instructors, and students during the COVID-19 pandemic, that does not mean they will continue to be available (Alturki & Aldraiweesh, 2021). Whatever the LMS used, an important lesson from our work is ensuring effective LMS knowledge and use, whether that is a mandated one from a university or one agreed upon with the customer.

Ultimately, success with the LMS environment was, most likely, due to the students’ familiarity with it, even if it caused hardship for instructors. Andy has taken this point forward in his future teaching practice and now believes that ‘meeting the students where they are’ includes technology not just knowledge level. For example, Andy uses Discord, an instant messaging platform for gamers, for his mentorship activities even though other senior facility members do not see it as an acceptable technology. The instructor team believes that an LMS can be identified as effective not by any praise given by students, but the lack of issues experienced with the technology throughout the course. That is, a technology should be ‘frictionless’ (Leachman & Scheibenreif, 2023).

Self-Study and Disciplines Beyond Teacher Education

Collaborating through self-study provided Andy, Jim, and Chris an opportunity to reflect on the prototype course and communicate with one another outside program planning and enactment phases. This communication went beyond learning about each other’s concerns regarding the conversion to an online environment. It also included opportunities for the instructors to better know each other. As we previously noted, the project lead, Andy, originally assumed that Jim preferred teaching over course development when the opposite was true. This led to changes in the project structure allowing Jim to lead the development of a short course (Data Management), completed in late 2022. The self-study highlighted communication concerns such as this, which was compounded by the loss of the informal conversations that would have occured between instructors, especially when the course was taught off-site. Initially, this loss of communication was compensated for using an instructors-only chat feature in MS Teams and by conducting a review of activities (known as ‘hot washes’) at the end of each instruction day.

Andy, Jim, and Chris were encouraged by their ability to assess the course using self-study. COVID-19 did not provide the time or opportunity to perform a classical educational study. However, self-study provided the opportunity to reflect on the course in a structured manner, to extract appropriate takeaways, and to share those results with the engineering education community. Self-study provided an approach for generating transparent results that can be shared with the community. There are several topics that seem particularly well-suited for self-study within engineering; such as improving the teaching of qualitative methods to quantitative-focused engineering graduates in systems engineering (Blockley & Godfrey, 2000).

Although Nilsson (2010, 2013) shared insights from a self-study she conducted with engineering educators, her studies lacked the engineering educators as co-researchers. Their voices and learning experiences were absent from those studies. As such, we believe this study constitutes the first self-study authored by engineers, which leads us to consider an important question. Pithouse-Morgan and Samaras (2015) asked, ‘What can other fields learn from a methodology that has been largely used by teachers, for teachers?’ (p. 5). Calls have been made for a self-study of professional practice, rather than self-study of teacher education practices, in hopes that self-study will be more accepted by those in disciplines outside teacher education. And there have been some entry points for self-study into what Polkinghorne (1988) calls ‘practices of care’, particularly within nursing (e.g., Wood, 2021) and counseling (e.g., Ieva et al., 2021, Williams et al., 2021). However, self-study research is still largely limited to teacher education – not in disciplines like applied sciences.

Yet there is a growing research base that ‘privilege self in the research design’ (Hamilton et al., 2008, p. 17). These methods include some already used in engineering, like autoethnography (e.g., Secules et al., 2021, Sochacka et al., 2016). Such research has provided evidence of the impact collaborative inquiry into one’s practice and identity can produce in engineering. We argue that self-study of professional practice is another path available to engineers, and others outside teacher education, to improve their teaching and understandings of self.

However, self-study is perhaps more complex than other ‘narrative-I’ methods (Hamilton et al., 2008). While autoethnography and narrative inquiry document one’s (or a collaborative’s) personal and/or professional stories, these interrogations often look to the past to tell a story of what was. Alternatively, the purpose of self-study research is to simultaneously investigate past and present experiences so that future practices and conceptions of self can be improved. This requires researchers to open their selves and practices to critique from their peers and the scholarly community, which can be an unsettling experience for new self-study researchers (Gregory et al., 2017). For novice self-study researchers, there is a benefit in collaborating with experienced self-study researchers who can guide the novice through the self-study learning process (Diacopoulos et al., 2022, Ritter et al., 2018). In our study, Brandon served that role as he identified pertinent self-study literature for the group to read, answered questions related to self-study research, and led the development of a proper methodological approach to conducting self-study. But his role extended beyond the methodological – as engineers, Andy, Jim and Chris have been trained to think quantitatively and dispassionately, and expectations for scholarship in their disciplines devalue reflective scholarship. As a result, it became Brandon’s responsibility to not only provide methodological guidance and ask difficult questions related to course content and teaching experiences as their critical friend, but to also challenge them to become vulnerable with one another, expressing their uncertainties and challenges related to their identities and practices as engineers in ways they had not previously accomplished.

Limitations

There are several limitations to this study. First, our self-study was conducted from the instructor’s viewpoint. This only gave us one view of the learning experience. Perhaps future self-studies of online course transitions will encompass additional voices, such as those of customers and students, to gain deeper insights into the complicated work of teaching and learning in online courses. Because our self-study used only dialogue and written prose to collect data used in our findings and the study took place after the prototype course had been taught, it is feasible that some problems that needed to be addressed were missed; that is, the ‘unknown unknowns’. Additionally, other information solicitation techniques, like problem structuring methods (PSM), which are commonly used in systems engineering (Hester & Adams, 2017), could have been employed to reveal new insights into the problem domain. We also believe that rich pictures, a part of Soft Systems Methodology (Mingers & Taylor, 1992), might have been useful for this exercise. However, because this was the first time the course instructors (Andy, Jim, Chris) conducted self-study, we felt that using a ‘traditional’ approach to self-study research was appropriate rather than complicating data collection using multiple methodological approaches. Future research could be conducted that combines PSM and self-study methods.

The instructors clearly saw the benefit of self-study from this exercise. What they found most useful was that self-study enabled them to move beyond a single problematic student or event and focus on the whole course experience. However, Andy remains concerned about the acceptance of self-study within the engineering community due to their focus on metric-driven teaching portfolios and quantitative research methods. He was also concerned about the shift in writing style, having written extensively for an engineering audience, whose professional writing style requires a rigid formal, and most importantly, impersonal approach. This article represents the group’s (Andy, Jim, and Chris) first attempt at breaking this writing mold to write from a personal, reflective approach.

Conclusions

The self-study highlighted several challenges that required addressing following the completion of the prototype online short-course. Before the prototype course, many of the instructors were not convinced that a technical short-course originally developed for in-person learning and reliant on learning a programming language could be successfully completed in an online environment while maintaining the teaching pace, covering the breadth of the material outlined in the original syllabus, and achieving the desired learning objectives. These concerns were based on the standard practice of being able to ‘look over a student’s shoulder’ during the exercise sessions, which instructors had found was an effective way to observe and support the students in the prior in-person teaching environment. However, self-study produced a discussion of potential solutions that have resulted in modification of course material to change the pace, create more structured Q&A sessions, and create more laboratory time where students can work on short assignments. The prototype and follow-on courses are testimony to this possibility.

Our study produced five implications for others considering transitioning highly technical and/or face-to-face short courses into an online format. The first implication is the acceptance that F2F courses cannot simply be recreated in an online environment, and adaptations are necessary. Second, this adaption might require developers to demonstrate their pedagogical understanding to ensure that the course customers understand the need for adaptation. Third, any adaptation cannot be purely technical, and there is a need to consider the personal needs of instructors and students in the online environment. The final two implications are there is a need to consider the technology access and understanding of the LMS used. Our hope is that readers see connections between the experiences we shared in this article and their own as higher education and professional learning increasingly shift online. And, we hope this study serves as an exemplar for educators, involved in teaching technology-based subject areas, who are interested in using self-study research methods to improve their practice.

Acknowledgments

The course and project described in this paper were supported by the United States of America’s Naval Sea System Command [grant number N00024-18-F-B057].

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Naval Sea Systems Command [N00024-18-F-B057].