Design principles for learning analytics information systems in higher education

ABSTRACT

This paper reports a design science research (DSR) study that develops, demonstrates and evaluates a set of design principles for information systems (IS) that utilise learning analytics to support learning and teaching in higher education. The initial set of design principles is created from theory-inspired conceptualisation based on the literature, and they are evaluated and revised through a DSR process of demonstration and evaluation. We evaluated the developed artefact in four courses with a total enrolment of 1,173 students. The developed design principles for learning analytics information systems (LAIS) to establish a foundation for further development and implementation of learning analytics to support learning and teaching in higher education.

ACCEPTING EDITOR:

• Pär Ågerfalk

ASSOCIATE EDITOR:

• Jonny Holmstrom

KEYWORDS:

• Design Science Research
• learning Analytics
• information Systems
• higher Education

Disclosure statement

No potential conflict of interest was reported by the authors.

Step 1: Familiarisation with the research data

The initial step of the analysis involved repeatedly reading of the interview transcripts in an active manner to become immersed in and intimately familiar with the data. While reading and re-reading the manuscripts, we actively looked for potential key patterns and meanings in the interviews.

Step 2: Coding

After becoming familiar with each interview, the transcripts were coded line‐by‐line for specific themes. In accordance with the objective of the interview, we used a deductive approach to develop the coding and themes, and this process was initially directed by existing notions about the two initial themes at the highest level:

(1) The perceived usability and usefulness; and

(2) Difficulties and concerns related to the use of LAIS.

Step 3: Searching for themes

We inspected the codes and collated data to check for patterns of variability and consistency across all transcripts. Furthermore, significant broad patterns of meaning were used to identify any additional potential themes. We also identified more specific subthemes for each candidate theme.

Step 4: Reviewing themes

We examined and refined the candidate themes against the dataset to determine whether they present underlying meanings of the data and address the objective of the interviews. We also reviewed the themes to ensure that the coded extracts of participants’ accounts formed a coherent pattern.

Step 5: Defining and naming themes

We analysed the revised themes in detail and determined the scope and focus of each theme to explore the story of each one. In this step, we developed an informative name for each theme:

• Perceived usability and usefulness and

• Subjective interpretation of reported information.

Step 6: Writing up

We created the analytic narrative and themes and then contextualised the analysis in relation to relevant literature. The results were written and reported in this research paper.