Curriculum nativeness – measures and impacts on the performance of engineering students

ABSTRACT

In times of rapid transformation of society in general and domains of technology in particular, questions are raised on how to effectively organise higher engineering education. As a response, this study examines the curriculum composition of eleven engineering programs to investigate curriculum nativeness, a novel approach for assessing curriculum characteristics. In addition to forming the construct nativeness, this study establishes a way to measure curriculum nativeness by determining the number of credits originating from what is characterised as native courses. Native refers to the way curriculum content reflects the main subject classification, connecting the content of the profession with the content of the program curriculum. This measure is used in correlation analyses and other dependency studies to assess performance of the students during their first year, including total grade point and attrition. The measure of curriculum nativeness is also used to compare programs. The results indicate that the level of native content in a curriculum influences student performance comparable to that of other learning types that are known to promote student achievement. In addition, this study indicates that native content credits are more frequently earned than non-native credits.

KEYWORDS:

• Curriculum design
• engineering education
• multidisciplinary education
• course construction
• engineering identity

1. Introduction

This study seeks to explore and identify parameters that characterise the composition and execution of engineering curricula in relation to the technical domains that engineering curricula addresses. One of these parameters is the ‘nativeness’ of the course content relative to a degree label, indicating whatever profile, specialty, or field of engineering the curriculum as a whole is intended for. At the university that is subject for this study, over the last decades there has been a steady increase in the types of professional engineering degree programs offered, at both bachelor and master level corresponding to Level 6 and Level 7 of the European Qualifications Framework (EQF 2020). The number of offered professional degrees has increased from five during the early 1970s to an average of over 50 during the 2010s. This growth in offered degrees seems to mirror the increase in complexity in society in general, particularly within existing and emerging domains of technology. It appears that the institution, over the years, has been trying to satisfy the needs of the industry by offering more specialised and discipline specific curricula. However, one could imagine an opposite approach, with more generalised and interdisciplinary curricula of the kind often referred to as an integrated engineering curriculum (see e.g. Froyd and Ohland 2005). Such curricula are not necessary pointing toward any specific professional degree or domain of technology, allowing students to independently form their own conception of their choice of studies. This line of reasoning motivated the study of curriculum elements that add to the specialisation and ultimately the label of degree programs. That is, this study investigates whether student performance is affected by how much the curriculum content clearly connects to the degree title. Hence, a (for this study) central definition of such curriculum content is: content that is associated to the program degree label and the engineering domain the program degree is designated for. The question then arises on how to measure this in practice? For this study, we utilise the course classification system at the studied university that indicates what domains of engineering the course addresses. By doing so we are able to quantify the extent to which the course content relates to the degree label, which in turn allows for the establishment of a measure for the nativeness of the curriculum.

Exploration of the term native content calls for a holistic approach, consequently this paper also elaborates on what we call curriculum nativeness, proposedly defined as the ability of a curriculum to provide contextual engineering identity. Moreover, since both the chosen means of operationalisation of curriculum nativeness as well as the construct itself is judged as novel, the study also searches for an item that could be used for comparison purposes that would support the assessment of the operationalisation of the nativeness measure, but also the conclusions from the statistical analysis. Consequently, by using a similar operationalisation of measures, this study also examines the composition of different curricula regarding characteristics that are known to promote student achievement. More specifically, the way assessment methods are classified and labelled at the studied university allows for identifying course content that are likely to be more oriented towards student interactions rather than traditional lecturing. Within the engineering research community, the usage of modules where students are actively or experientially involved in the learning process rather than passively learning (Bonwell and Eison 1991), is a well-established and recognised method to increase student performance. For example, Freeman et al. (2014) concluded that student performance increased by just under half the standard deviation with such type of learning compared with lecturing, and that students in traditional lecture courses are 1.5 times more likely to fail than students in courses promoting interactive engagement. Bonwell, Freeman and others explicitly refers to the term active learning when discussing learning styles based on student interaction, collaboration, cooperation and problem-based learning. From the perspectives of the present study, aspects of active learning are further elaborated in the theoretical framework. However, from hereon and for readability purposes, the term ‘active learning’ is used for referring to the specific categorisation of course learning styles, which in this study is based on the assessment method of the studied curricula.

The data for this study was collected from the first year of the studied curricula, which is not without relevance in terms of motivation for conducting the study. Within the fields of engineering education and curriculum design, the first year is often referred to as the most critical and consequently many well-cited studies have focused on the freshman year (Daempfle 2003; Hutchison et al. 2006; Jones et al. 2013). The young freshman student is not only at a turning point in life, but is also concerned with managing higher education studies in general and the field of their choice in particular: Did I really pick the right path? The faculty, on the other hand, are usually concerned about drop-out rates: Now that we have attracted these students, how can we ensure that they stay and manage their courses? Another relevant factor is the number of students that engage in higher engineering studies. With Sweden as an example, the ratio between the number of high school graduates (with higher education admittance possibility) and the number of students admitted to any three or five-year professional engineering program was on average 26.7% between 2014 and 2018. That is, about 13,000 students are admitted every year. During the same period, the number of students that annually earned a degree from these types of programs was on average 6468, indicating that roughly only half of the students graduate (Statistics Sweden 2019; Swedish National Agency for Education 2019). Furthermore, the number of high school graduates who engage in higher studies within two years fluctuates drastically, reflecting the fluctuations in society such as changes in economic conditions. Again, with Sweden as an example, over the last 20 years, the percentage of first year students being admitted within a year from high school graduation varies between 12.8% and 18.6% (Statistics Sweden 2019). A higher number of students entering higher education directly from high school means that students generally have less work experience, so they have fewer references supporting the conception of their choice of studies along with their anticipated career path. From this viewpoint, it makes sense to search for characteristics of the curriculum that might improve this situation for both students and faculty.

1.1. Research aims

This study introduces the term curriculum nativeness and demonstrates a way to measure the characteristics of a curriculum in terms of nativeness, i.e. the amount of native content. Based on empirical findings, this study discusses the effects of native content on the curricula. Does native content, as defined and measured in this study, influence student performance in engineering degree programs? Does native content foster persistence? In a wider perspective and given the means of measurements presented in this paper, this study also discusses the usefulness of measuring native curriculum content and other quantifiable properties of the engineering curriculum. The questions raised above are addressed using statistical analysis of curriculum composition and student performance data.

1.2. Paper outline

The structure of this paper is as follows. Section 2 provides a theoretical framework aimed at supporting a construct of curriculum nativeness. The section elaborates on integrative learning as well as active learning, which constitute not only a central part of the subsequent discussion about engineering identity and curriculum nativeness but also the validity of proposed measures. Section 3 presents a way to measure curriculum nativeness along with a corresponding measure of course modules that promotes student interaction, i.e. active learning, later used for comparison purposes. In addition to providing an overview of methods, Section 3 defines the independent and dependent variables that are analysed in Section 4. Section 4 also presents the main results that are discussed in Section 5, and Section 6 presents the overall conclusions of the study.

2. Theoretical framework

This section provides a theoretical background and introduces the proposed concepts necessary for evaluating and discussing the case study. This section also discusses engineering identity, nativeness, and other curriculum characteristics and how these concepts relate to the empirical observations at the studied university.

2.1. Curriculum nativeness as a construct

Further on, this paper investigates the relation between the performance of engineering students and the composition of the curricula in terms of nativeness of the content. Specifically, nativeness of the content is considered in light of the degree label of the program: e.g. M.Sc. in Mechanical Engineering or M.Sc. in Industrial Management. Consequently, there is a need to discuss and define a construct for the term curriculum nativeness.

2.1.1. Conceptual definition

Native is defined as an ‘ability or characteristic that a person or thing has naturally and is part of their basic character’ (Cambridge English Dictionary 2019). However, this study uses native in a more strict sense: native content refers to curriculum content that is associated to the program degree label and the engineering domain the program degree is designated for. Nonetheless, for something to be native, there has to exist a notion of identity for the parties involved such as professional identity and engineering identity, concepts often used in the literature. Meyers et al. states that ‘engineering identity is believed to relate to educational and professional persistence. In particular, a student’s sense of belonging to the engineering community is critical to that path’ (2012). Several studies also focus on the students’ perceptions of engineering to develop a way to measure engineering identity. For example, Godwin collects measurements that ‘can be used to understand how students see or do not see themselves as the type of people that can do engineering’ (2016). Other researchers refer to pure social identity theory. Pierrakos et al. identified four common themes that provide insights on the development of engineering identity: ‘(1) knowledge of profession and exposure to engineering, (2) interest in engineering and influences to enter major, (3) sense of preparedness and (4) sense of belonging, fit, and commitment’ (2009). Pierrakos et al. conclude that ‘[engineering] persisters are on the path to develop an identity as an engineer through participation in a community of practice (engineering community) and that their initial foray is supported by their prior knowledge about engineering and their initiatives once they are on campus’ (2009). These perspectives highlight the role of curriculum content as a provider of the ‘knowledge of engineering’. This proposition has at least one corollary question: To what extent is a curriculum capable of providing such knowledge?

Engineering curricula could directly affect two of Pierrakos et al.’s themes: knowledge of profession and exposure to engineering and sense of belonging, fit, and commitment. In addition, the curricula could provide students with a sense of preparedness. The common denominator of these themes is essential for the objectives of this paper; that is, different curriculum content has different capabilities with respect to providing knowledge of engineering through the development of engineering identity. For example, Dehing et al., in an exploratory study from a teaching staff perspective, found that curriculum design influences how students understand engineering identity:

[T]eachers working in the discipline-oriented curriculum report limited growth [in developing engineering identity] during the first two years, when education is focused on acquiring the knowledge base. Teachers working in a professional-oriented curriculum, for example, in civil engineering, see that identity development starts from the very beginning, when students have to perform professional roles in projects. (2013)

Dehing et al.’s make a clear distinction between a discipline-oriented curriculum and professional-oriented curriculum, where the former spend the first two years to ‘build up the disciplinary scientific body of knowledge’ and the latter ‘combines the acquisition of scientific knowledge with its application in realistic engineering tasks from the very beginning of the curriculum’ (2013). The study of Dehings is conducted in a Dutch higher engineering education setting, where the above described distinctions refers to two different types of tertiary institutions, one being the ‘Universities of applied sciences’ (Hogeschool in Dutch) and the other traditional universities, where the former are more focused on profession rather than scientific research. Nevertheless, Dehing et al.’s professional-oriented curriculum could be interpreted as more native, given the observation of stronger engineering identity build-up among the students. Therefore, it is reasonable to conclude that the ability of a curriculum to provide engineering identity, in general or relative a specific domain of technology, is part of understanding the concept of curriculum nativeness.

A logical next step is to turn to the literature to search for curriculum concepts that promote identity and nativeness in the context of engineering curriculum design. The capstone course, commonly given as a final-year project-based course, links academic and professional experiences (Jones et al. 2013), consequently providing a meaningful context where the student can process previously achieved knowledge in a native context. This context is native because the project assignment mirrors the reality the students will face as professional engineers, further developing their identity as professional engineers. However, capstone courses typically occur at the very end of a degree program as capstone courses are designed to tie together knowledge and experiences from previous course work. Presumably, before the capstone courses, students are unable to connect the title of their program to their professional identity. Therefore, the main drawback with capstone courses is their position in the curriculum, at the very end. A way to tackle this aspect of capstone courses is to simply place them at the very beginning of the curriculum. Such courses are usually referred to as Freshman Introductory Courses, First-year Classes, or Large-enrolment Courses (e.g. Ambrose and Amon 1997; Deslauriers, Schelew, and Wieman 2011; Vallim, Farines, and Cury 2006). These courses, like capstone courses, are intended to help students understand what is meant by professional identity. The often-utilized project assignments points towards something that has not yet been experienced (i.e. specialised courses/subjects later in the program), at the same time trying to establish a conception of the chosen profession degree the program is aiming for.

The freshman introductory course and the capstone course respectively represents specific type of courses, with specific positions in the curriculum. But what other curricula elements, or characteristics, could have the ability to provide nativeness as defined in this paper? Few would argue against engineering in the twenty-first century being an interdisciplinary profession. Aspects of interdisciplinarity are, on curricula level, often represented by means of integrative measures, i.e simultaneously providing curriculum content from different areas of engineering. In the literature, integrative learning is spoken of as a key concept for mirroring the real-life situation of the practicing engineer. Huber and Hutchings (2004) speaks of varieties of integrative learning as ‘connecting skills and knowledge from multiple sources and experiences; applying theory to practice in various settings; utilizing diverse and even contradictory points of view; and, understanding issues and positions contextually’, and continues to state that ‘integrative experiences often occur as learners address real-world problems, unscripted and sufficiently broad to require multiple areas of knowledge and multiple modes of inquiry, offering multiple solutions and benefiting from multiple perspectives.’ Considering reasonings such as this it is close at hand to state that integrative characteristics of a curriculum also are related to the ability of the curriculum to provide nativeness, relative to the degree label of the program. Furthermore, context ought to be central here as any contextual considerations from either the student or the tutor could be said to relate to the nativeness of a curriculum element or activity.

If searching the literature further in order to establish a relation between integrative learning and curriculum nativeness, a number of relevant contributions emerges. Notable are cases where whole curricula are subject of integrative measures. One such example is the reformations of undergraduate engineering programs at UCL (Mitchell et al. 2019; Tilley and Mitchell 2015) based on the premise that ‘the modern engineer should understand the context of the problems they address, appreciating the ethical, societal and financial connotations of their design decisions’ (Bains et al. 2015). Moreover, since the founding in 1997 the Olin Collage of Engineering has gained much attention for its organisation and curriculums development processes. Classes at Olin emphasise context, interdisciplinarity and emotional engagement. The curriculum is characterised by integrated course blocks that allows for exploration of relationships between main subject engineering calculus and physics (Goldberg and Somerville 2014) and ‘throughout the curriculum, students stay engaged by working on projects connected to real-world challenges. In their first year, students also begin to explore the arts, humanities and social sciences as well as entrepreneurship’ (Olin College of Engineering 2020). Another similar example where integration is taken even further is the so-called New Model in Technology & Engineering (NMiTE), a recently established purpose-built teaching-only engineering and technology institution in Hereford, UK. The NMiTE initiative emphasises ‘multidisciplinary teamwork, entrepreneurship, liberal arts and leadership, as well as science and math’ (Grose 2017). From the discussions regarding the NMiTE initiative and its strong orientation towards the role of engineers and engineering in relation to the surrounding society (Kozinski et al. 2017), one can observe that both context and identity are crucial to these types of educational settings.

By examining the outspoken justifications of such endeavours, one finds clear connections to the notion of curriculum nativeness that this paper seeks to introduce. Bains et al. states that ‘the modern engineer should understand the context of the problems they address, appreciating the ethical, societal and financial connotations of their design decisions.’

Yet another apparently related reasoning is found in the work of Jamison, Kolmos, and Holgaard (2014) when referring to the concept of hybrid learning. Following an exploration of the contractiveness between market-driven and academic strategies of response to the challenges facing the engineering industrial and academic community, Jamison argues for an integrative approach to engineering education that ‘combines a scientific-technical problem-solving competence with an understanding of the problems that need to be solved.’ Jamison elaborates further that such an integrative approach would involve ‘a mixing of scientific education and practical training in technical skills with a cultivation of a broader cultural understanding of the implications of science and technology in general and reflections on one’s own role as an engineer in particular.’ Given the purposes of this contribution the reasoning of Jamison could be argued for as related to the concept of nativeness. In particular where Jamison argues for the necessity of reflecting on roles of engineering, which automatically would impose assessment of learning situation regarding nativeness.

Another way of ensuring students are exposed to nativeness is to expose them to higher degrees of nativeness throughout their studies (at least during one semester) by implementing profiling tracks that remind students of the context. For example, Berglund and Heintz (2014) conducted a study in a similar learning environment as the one that is subject for the present study, where they investigated a course that stretches over the first three years with students from all three years taking the course together. The course was developed with representatives from industry with the overt purpose to address ‘Professionalism for Engineers’ with a strong reference to engineering identity. The course content covers knowledge areas that are identified as important for both the study period as well as for the students’ professional careers after graduation. Similarly, Hallberg argues for a platform-based framework that ‘allows for realistic training of several engineering disciplines concurrently throughout the curriculum’ (2018). Similar initiatives can be found elsewhere.

However, the above examples of how to encourage professional identity should be seen as selective measures. A complementary view is to study the composition of the curriculum as a whole rather than as only a single course: What curriculum content clearly connects with the title of the program and does it make a difference compared to other programs with a different set of native content? These questions can be answered by identifying and comparing what courses are systematically classified by subject or field. Because such classification systems are in place at many universities (including our case study), it is possible to compare different curricula based on the same classification system. As will be demonstrated, the proposed concept of curriculum nativeness relies strongly on this approach.

2.2. Other curriculum characteristics

For comparison purposes regarding the forthcoming operationalisation of the nativeness measure, other characteristics of the studied curricula are taken into account. Motivated by the significant influence of active learning – e.g. student interaction, collaboration, cooperation, problem-based learning etc. – on student performance (see e.g. Freeman et al. 2014) and the above reasoning regarding native content of the curricula, the study explores the relationship between active learning and the native content of the curricula. Active learning is fundamentally learning by doing. It is reasonable to assume that what teachers choose as the content of an active learning module is influenced not only by their audience’s needs (i.e. students’ needs) but also by the profession the curriculum represents, and consequently affecting the engineering identity formation process. There are many examples of studies that relate active learning and engineering identity. For example, Du concludes that

an engineering university is a place where students not only learn technical knowledge and engineering skills, but also develop a sense of belonging to the engineering profession in order to prepare themselves for the future workplace […] this active learning process [PBL], in which students learn to manage different resources to cultivate a community of practice, is also a process of developing a professional engineering identity. (2006)

From a pedagogical view, the concept of active learning within engineering education has been a relevant curriculum ingredient since the early 1980s (Bonwell and Eison 1991). However, active learning lacks an established and unified definition. Nonetheless, while partly referencing Bonwell, Freeman et al. settle on the following definition:

Active learning engages students in the process of learning through activities and/or discussion in class, as opposed to passively listening to an expert. It emphasizes higher-order thinking and often involves group work. (2014)

Others, such as Prince (2004), provide more thorough definitions that discuss additional subsets of active learning. These subsets include Collaborative learning (students work together in small groups toward a common goal), Cooperative learning (groupwork where students pursue common goals while being assessed individually), and Problem-based learning (an instructional method where relevant problems are introduced at the beginning of the instruction cycle and are used to provide the context and motivation for the learning that follows).

Moreover, within the field of engineering education, some research deals with active learning as a catalyst for student performance and therefore tries to measure its effectiveness. Prince (2004) finds support for all studied forms of active learning, with examples such as enhanced remembrance, enhanced academic achievement, etc. Freeman et al.’s (2014) well-cited meta-study concludes that students in traditional lecture courses are 1.5 times more likely to fail than students in courses that use active learning as the pedagogical modality.

In addition, initiatives such as the Conceive Design Implement Operate Initiative (CDIO) deal with engineering education frameworks that promote the use of active learning modules throughout the curriculum. The CDIO consists of a set of standards (or guidelines) for developing a curriculum (Bennedsen, Georgsson, and Kontio 2016; Crawley et al. 2011). The CDIO community advocates active leaning as a crucial component of any engineering curriculum (Malmqvist et al. 2006). Our case study university is a member of the CDIO Initiative, a circumstance that is not without relevance for the present study since the teaching environment has a long tradition of emphasising and implementing active learning modules throughout the curricula, according to the CDIO framework. Thus, as there is a well-established classification system in place that identifies active learning modules in the curricula, it becomes natural to utilise this system for comparison of the impact of native content.

3. Research design

This study was conducted at a large Swedish university offering a wide range of professional degree programs within the humanities, medicine, and engineering, mainly corresponding to Level 6 (Bachelor level) and Level 7 (Master level) of the European Qualifications Framework (EQF 2020). Figure 1 provides a schematic overview of the study, describing data sources and dependent and independent information generating events that preceded the study, as well as the tools used for the data processing and analysis. The study included first-year students on 11 different engineering degree programs (n = 790, 211 females and 579 males). Upon enrolment, background data is automatically registered in the admission & student registry, including age, gender and high school rank. Before starting their studies, the students perform a diagnostic math test. The test is performed by everyone granted admission to the institute of technology at the studied university, indicating the mathematical readiness of the student. The result from this test, which is used as a variable in the present study, was manually gathered and prepared for analysis Furthermore, the study investigates the curricula composition of the first year of the eleven engineering programs (see Table 1) with respect to native content and learning styles (based on assessment methods) as well as the performance of the students in these programs in terms of credits, grades, and attrition. The data source for this information was gathered from the result database, and the curricula and course database at the university.

Figure 1. Schematic overview of the study describing data sources, dependent (red) and independent (green) information generating events (chronologically from left to right), and tools used for data processing and analysis.

Table 1. Composition of studied programs during the first two semesters based on main subject occurrences for courses in the program. For each main subject, the number of occurrences of each program is specified. Grey cells indicate subjects that were judged to be native for the program, given its degree title. The total number of courses differ from the sum of all main subjects because a course might be classified with up to three main subjects. Rightmost column provides program reference numbers to be used throughout the paper.

	Main subjects
Degree title	Mathematics & Applied Mathematics	Computer Engineering	Computer Science	Product Development	Mechanical Engineering	Energy and Environmental Engineering	Industrial Engineering and Management	Physics	Applied Physics	Electrical Engineering	Chemistry	Chemical Engineering	Chemical Biology	Biology	Biotechnology	Engineering Biology	Engineering	Other subjects	Total number of courses	No. of courses classified with one or more native main subjects	No. of credits with native classification	NC (%)	Program reference no.
M.Sc. Computer Science and Engineering	4	1	6							2							1	2	12	6	34	57	P01
M.Sc. Design and Product Development	4		1	3	1														9	3	24	40	P02
M.Sc. Mechanical Engineering	4		1	2	4	1	1												10	4	24	40	P03
M.Sc. Applied Physics and Electrical Engineering	5	1	1					2	3	2									10	4	18	30	P04
M.Sc. Energy – Environment – Management	4		1	1	3	4	1				1								10	5	30	50	P05
M.Sc. Industrial Engineering and Management	6		1				2												9	2	16	27	P06
M.Sc. Engineering Biology	3										4	4	1	2	1	1			10	7	42	70	P07
B.Sc. Computer Engineering	2		4							3								1	9	6	40	67	P08
B.Sc. Engineering Electronics	2		3							4								1	9	6	40	67	P09
B.Sc. Chemical Analysis Engineering	2										6	7	1		1	1			9	7	42	70	P10
B.Sc. Mechanical Engineering	3			1	4					2									9	4	26	43	P11

Furthermore, apart from operationalising a construct of curriculum nativeness, the aim of this study is also to measure the effect of different levels of native content of the curriculum. In an effort of doing so, the following sections describes the establishment of different independent variables that will be used for statistical analysis. The analysis will then serve as a foundation for a discussion regarding the research aims as stated in 1.1, i.e. ‘Does native content, as defined and measured in this study, influence student performance in engineering degree programs? Does native content foster persistence?’ The studied variables are divided into three categories of measures: the amount of native content (i.e. measuring curriculum nativeness); the amount of course modules indicating active learning (measured for comparison purposes); and a description of the students.

3.1. Curriculum nativeness – an operational definition

As curriculum nativeness is the main construct explored in this study, the first step is to define how to express curriculum nativeness in terms of one or more measurable variables suitable for a statistical analysis. The measure Native Credit percentage (NC) is proposed as the sum of all courses classified within a main subject that clearly connects with the profession degree divided by the total number of credits (European Credit Transfer System, ETCS) for the studied period or program: (1) $\text{NC}={\displaystyle \frac{\sum \left(\text{ECTS}\text{classified}\text{as}\text{native}\right)}{\text{Total}\text{ECTS}}}$(1) To establish such a variable, curricula data provided by the studied university are used. The university classifies courses by a set of predetermined main subjects. The classification system has several purposes. In addition to internal control and distribution of funding between departments, it serves as a vital tool for program directors responsible for the content and composition of curricula. There are also legislated degree requirements connected to the classification of courses; for example, to obtain a Master of Science degree in Mechanical Engineering, a student needs to earn a predetermined number of credits classified within the main subject Mechanical Engineering.

Nineteen different main subjects are used to classify all the courses offered by the eleven programs studied. A course classification can consist of a combination of up to three different main subjects. See Table 1 for the number of main field occurrences for each studied program. The greyed cells indicate which main subjects are native for each studied program.

Based on the above described classification of courses, the Native Credit percentage can be established for each program. This is done by pointing out courses classified with one or more main subjects that clearly connect with the profession degree of the program, adding together their credits and dividing by the total number of credits for the first two semesters that are part of the subject for this study (i.e. 60 ECTS). For example, a ‘Bachelor of Science in Chemical Analysis Engineering’ makes up its NC percentage with the number of credits from courses classified with the main subjects such as Chemistry, Chemical Engineering, Biotechnology, etc. Likewise, a ‘Bachelor of Science in Mechanical Engineering’ accumulates its native credits from courses classified with the main subjects Product Development, Mechanical Engineering, Physics, etc. In this particular study, the authors unanimously and consensually determined which main subjects should be considered as native for each program. This was deemed methodology sufficient given the background of the authors as members on the board of studies responsible for four of the eleven studied programs, and in close collaboration with representatives of the other seven. Altogether, the judgement on nativeness was based on thorough insights into the structure, functioning and regulations regarding engineering curricula, both at this university as well on the national level.

Using this calculation, we conclude that the native content of the first year of the programs studied varies between 27% and 70% (i.e. more than a factor of two) (Table 1). The main subjects of mathematics and applied mathematics are not considered native for any of the studied programs, although the number of math courses taken by the students differ between the programs. In addition, neither of the courses in any of the programs were classified with a combination of a native main subject and one or two of the math subjects (Mathematics and Applied Mathematics), nor were the two math subjects labelled separately in any course (hence the merged column in Table 1). Evidently, the curriculum characteristics of the programs showed a clear distinction between pure math courses and other disciplines related to courses with a classification eligible for being native or non-native.

3.2. Additional operational definitions

As previously stated, the present study searched for comparison items that would support the assessment of the NC measure. Following a review of the existing categorisation of course modules, other curriculum characteristics known for influencing student achievement were identified, namely those who indicates some sort of student interaction and engagement in the learning process. As active learning (e.g. student interaction, collaboration, cooperation, problem-based learning etc.) can increase student performance in science, engineering, and mathematics (Freeman et al. 2014), this study compares the impact of such a measure on student performance, with the impact of the measure native content established in 3.1. As with NC, an Active Learning Credit percentage (ALC) can be calculated using the percentage of ECTS classified as active learning in relation to the total ECTS: (2) $\text{ACL}={\displaystyle \frac{\sum \left(\text{ECTS}\text{classified}\text{as}\text{active }\text{learning}\right)}{\text{Total}\text{amount}\text{of}\text{ECTS}}}$(2) Consequently, a measure has to be established that indicates the amount of active learning taking place in the studied programs. The measure has to be robust and equal regardless of the course or curricula being studied. However, a hands-on investigation of every single course module would be impractical as there are 149 different course modules involved in this particular study. Furthermore, classifying a module as active learning risks investigator bias. Therefore, as with the approach for the NC measure (also a faculty-specific classification system), we used examination codes extracted from the individual course plans to determine the classification of active learning modules. The codes correspond to the assessment methods used for a particular course module, but the codes also indicate the pedagogy style used – active or passive.

Four methods were used to assess knowledge achieved: written exams, hand-in assignments, laboratory exercises, and project assignments. The individual course plans make it clear that written exams and hand-in assignments typically constitute assessment of learning modules that is in line with traditional teaching, such as teacher-centered classroom lecturing. On the other hand, laboratory exercises and projects assignments typically assess knowledge achieved from modules with a higher degree of active learning. These module classifications were also determined using the studied course plans. Such modules typically include laboratory sessions where the students interact with different set-ups or project assignments. These assignments, often resulting in a product produced by a group of students, are expected to produce unique outcomes. However, a written exam does not mean that a lecturer does not use some sort of active learning during the classes preceding an exam although this approach is less likely compared with other forms of explicit active learning assessment methods, such as laboratory exercises. Moreover, given the large number of courses involved in this study (69 in total), the effects of mis-leading examination codes were deemed marginal.

Table 2 describes the composition of the first two semesters for each studied program/curriculum with respect to occurrences of different assessment methods. By adding the number of credits originating from assessment laboratory exercises and project assignment modules and dividing this sum by the total number of credits for the first two semesters (i.e. 60 ECTS), an Active Learning Credit percentage (ALC) can be established for each program/curriculum.

Table 2. The types of assessment methods for each program, the distribution between the different methods, and the resulting Active Learning Credit ratio (ALC).

				Counts					Credits (ETCS)					Ratios (%)
		No. of courses	No. of courses with grades	Written exam	Hand-in assignment	Laboratory exersise	Project assignment	∑	Written exam	Hand-in assignment	Laboratory exersise	Project assignment	∑	Written exam	Hand-in assignment	Laboratory exersise	Project assignment	ALR (%)
Program	P01	12	9	6	7	5	2	20	28	9.5	16	7	60	46	16	27	12	38
	P02	9	7	5	9	1	4	19	29	15	2	15	60	48	24	3	25	28
	P03	10	7	7	5	9	3	24	35	7.5	12	6	60	58	13	20	10	30
	P04	10	8	8	4	4	1	17	39	11	7.5	3	60	65	18	13	5	18
	P05	10	10	10	4	5	1	20	39	8	11	2	60	65	13	18	3	22
	P06	9	9	9	2	3	1	15	45	4.5	9	2	60	74	8	15	3	18
	P07	10	9	9	3	5	1	18	44	4.5	7.5	4.5	60	73	8	13	8	20
	P08	9	6	7	6	5	2	20	26	13	11	11	60	43	21	18	18	36
	P09	9	6	7	5	5	1	18	32	11	11	6	60	53	18	18	10	28
	P10	9	8	9	1	6	1	17	43	1.5	11	4.5	60	72	3	18	8	26
	P11	9	7	8	5	4	2	19	37	8	11	4	60	62	13	18	7	25

Table 2 reveals that the curricula differ rather much in composition and that the ALC measure can vary by a factor of two. Notably, two programs (P02 and P05) have almost the same number of assessment modules, although P05 has double the number of written exams. Furthermore, the ALC ratio can differ regardless of the number of written exams. This is the case with P01 (ALC 38%) and P04 (ALC 18%).

3.3. Descriptive variables

This study is based on students enrolled in the eleven programs for the fall semester of 2017 (N = 790; 211 females and 579 males). Descriptive variables for the cohorts are age (Enrolment Age), final high school grade (High School Rank), and the performance on an initial test for mathematical readiness (Math Test).

The age of each student is used as a variable for the time between high school graduation and program enrolment (High School/Enr. Delay). Assuming that most students start elementary school at the same age and pass directly through the school system, the age at time of enrolment can be used as a measure of the time between high school graduation and enrolment. The literature remains ambiguous weather age influences student performance in higher education. Some finds that it does matters (e.g. Sear 1983; Woodley 1984) while others remain inconclusive (e.g. Richardson et al and Richardson and Woodley 2003). The decision to include the High School/Enr. Delay variable is primarily based on experience at the studied institution, where data of student performance indicates that (higher) age has a negative influence on the performance of the students.

The final high school grades were used to estimate the students’ relative standing compared to their peers upon enrolment. This measure was not necessarily the one that granted admission to the university. Admission might have been granted due to student’s Scholastic Aptitude Test (SAT) scores as admission offices place a higher value on the SAT scores than on high school grades. Typically, about 30% of the available admission spots are reserved for applicants with sufficiently high SAT scores, but unlike the high school grade, far from all admitted students have taken a SAT test and had the result registered, which make SAT scores unsuitable for measurement of relative standing from a statistical perspective. In addition, grade data for some individuals were missing due to non-translatable grades of students with non-Swedish high school diploma or outdated grades of older students.

Another measure of relative standing was also used: the individual grade on a diagnostic math test performed on the first day of enrolment. This test is performed by everyone granted admission to the Faculty of Technology at the studied institution to determine the students’ mathematical readiness. There are three strong reasons for including the test in this study. First, the high school rank grade reflects the capability of the students based on a wide set of knowledge domains. Some of these can be seen as less relevant for a career path within the domains of engineering, such as religion or physical education (which are both mandatory subjects in upper secondary school). The math test, on the other hand, clearly corresponds to the domains of engineering that the students are about to study. Second, the registered high school grade could in some cases be less relevant due to a delay in enrolment in higher studies as preparation studies might have occurred and employment might have had a strengthening effect on motivation. Third, the student’s high school grade might not have been used as the reason for acceptance into the program. For example, in some cases, there is the possibility that access has been granted as the result of the student’s SAT or American College Testing (ACT) scores, which are used in the United States and elsewhere. See Table 3 for studied student population descriptions. The literature also provide support for similar (positioning) tests for predicting student performance, where in particular previously achieved math skills from secondary school strongly relates to first-year achievments (see e.g. Pinxten et al. 2019; Vanderoost et al. 2014). Moreover, an observation that is subject for discussion later in this paper is that the High School Rank and results on the Math Test are strongly correlated. Point plotting High School Rank means by Math Test results yields a linear fit of R²= 0.95 (see Appendix H), basically meaning that in ninety-five percent of the cases the score on the Math Test corresponds to the student’s High School Rank.

Table 3. Studied population descriptives.

		Gender			Enrolment Age		High School Rank (0–22.5)			Math Test (0–26)
		F	M	Total	Mean	SD	Mean	SD	Missing,	Mean	SD	Missing,
		N%	N%	Count, N					%			%
Program	P01	9.6	90.4	94	20.1	1.7	19.2	1.7	11.7	16	5	7.4
	P02	48.3	51.7	60	20.8	1.5	19.3	1.6	11.7	13	5	55.0
	P03	13.4	86.6	119	20.5	1.3	18.9	1.5	9.2	14	5	7.6
	P04	16.9	83.1	89	20.3	1.9	19.7	1.8	13.5	17	5	3.4
	P05	61.5	38.5	65	21.1	2.4	19.4	1.9	15.4	14	5	7.7
	P06	34.0	66.0	162	20.5	1.4	20.4	1.4	17.3	16	5	4.9
	P07	60.0	40.0	35	20.9	3.1	19.0	1.9	11.4	11	5	5.7
	P08	3.2	96.8	63	21.5	3.0	16.3	1.4	11.1	10	4	7.9
	P09	5.3	94.7	19	22.8	3.3	15.4	2.5	5.3	8	5	5.3
	P10	63.0	37.0	27	23.8	5.2	16.7	2.3	25.9	-	-	100.0
	P11	10.5	89.5	57	21.5	2.3	16.9	2.0	10.5	10	5	5.3
	Total	26.7	73.3	790	20.8	2.3	18.9	2.2	13.2	14	5	13.0

3.4. Dependent variables

With the aim of exploring the concept of curriculum nativeness using the NC measure operationalised and described in 3.1, there is a need to establish dependent variables to be used for a statistical analysis. Table 4 summarises eight variables and the values for these variables for each of the studied program:

• The ECTS credits earned during the two first semesters, gathered from the result database, is used as a generic measure of student performance. In this particular study, the variable accounts for credits earned from exams taken between September 2017 and August 2018, including re-exams.

• Cases where the maximum of 60 ECTS credits is reached renders an issue since the ECTS variable does not provide grades for individual courses. Consequently, the total achieved Grade Point (GP) is calculated as the sum of the product between grade and credit for each assessment module. Grades are given as 3, 4, or 5 for graded courses, with 5 representing the highest grade. For modules where only a fail or pass grade is given, the pass grade corresponds to the grade 5 when calculating the GP. The reason for this is that the number of Pass/Fail only modules differ slightly between the studied programs. Thus, if the Pass grade would correspond to 3 in the model, it would result in different maximum credits for each program, which in turn would make the comparison between groups of students more difficult. This calculation yielded a measure where GP can equal a maximum of 300 for each student.

• The drop-out rates registered after the first (fall) semester (1st sem. drop-out rate) and after the second (spring) semester (2^nd sem. drop-out rate) yielded a total drop-out rate (Total drop-out rate), which is used as a measure of student attrition.

• A Native Credits Achievement percentage variable (NCA) is calculated. The measure, individual for each student, is the calculated ratio between ECTS credits from passed course modules classified as native for each program/curriculum and the sum of all ECTS credits classified as native (for the program the student is enrolled in). Note that Table 4 declares for a mean of individual NCA for each program: (3) $\text{NCA}={\displaystyle \frac{\sum \left(\text{Passed}\text{ECTS}\text{classified}\text{as}\text{native}\right)}{\text{Total}\text{ECTS}\text{classified}\text{as}\text{native}}}$(3)

• A corresponding measure, non-NCA, is established by measuring the ratio of passed course modules classified as non-native for each program/curriculum. Note that Table 4 declares for a mean of individual non-NCA for each program: (4) $\text{nonNCA}={\displaystyle \frac{\sum \left(\text{Passed}\text{ECTS}\text{classified}\text{as}\text{nonnative}\right)}{\text{Total}\text{ECTS}\text{classified}\text{as}\text{nonnative}}}$(4)

• Finally, the NCA measures were used to calculate performance ratios between modules with native content and non-native content – i.e. the NCA/non-NCA ratio. Note that this measure is a mean of individual NCA/non-NCA ratios for different programs.

Table 4. Student performance.

		ECTS		GP		1st sem. drop-out rate (%)	2nd sem. drop-out rate (%)	Total drop-out rate (%)	NCA (%)		non-NCA (%)		NCA/non-NCA Ratio
		Mean	SD	Mean	SD				Mean	SD	Mean	SD
Program	P01	46.6	12.2	193.2	48.8	9.8	2.1	11.7	88.2	17.7	64.0	27.3	1.7
	P02	47.1	13.5	207.9	57.0	10.5	5.0	15.0	89.5	20.6	66.8	28.5	1.8
	P03	49.6	12.6	218.9	54.6	5.3	6.8	11.7	90.1	20.0	73.5	24.6	1.3
	P04	43.9	16.9	184.3	70.2	15.6	8.6	22.9	86.1	22.2	59.9	33.1	2.3
	P05	49.7	12.9	214.2	55.4	10.5	5.0	15.0	92.7	19.8	67.1	28.9	1.6
	P06	48.5	15.9	207.7	70.0	5.3	6.8	11.7	92.6	22.7	70.2	31.2	1.9
	P07	45.2	16.0	182.7	66.9	15.6	8.6	22.9	86.3	27.5	41.0	35.2	4.0
	P08	44.1	18.0	196.0	81.7	8.6	7.9	15.9	76.1	30.8	66.2	37.9	1.7
	P09	43.6	15.0	191.6	70.2	11.1	5.3	15.8	77.2	25.6	62.1	36.8	1.8
	P10	38.8	16.8	156.1	67.7	4.0	7.4	11.1	69.6	29.0	46.2	42.2	1.0
	P11	41.2	18.9	173.0	80.5	13.5	8.8	21.1	65.6	31.5	69.6	33.4	1.2
	Total	46.4	15.4	198.9	66.8	9.2	6.5	15.1	86.0	24.6	65.6	31.9	1.8

3.5. Data sources and tools

The three main data sources are the university result registry, the admission database, and the study guide, which includes curricula and course plans. All collected and used data for this study falls under national legislation of public access to official records. Necessary statistical analysis of gathered data – i.e. correlation analysis, regression models, goodness-of-fit tests and graph studies – was performed using MS Excel and SPSS™.

4. Statistical analysis and results

To determine how NC variables relate to other independent variables, we describe the relevant statistical analysis performed within the scope of the study. Appendix A summarises the independent and dependent variables defined and used for this study. An initial bivariate correlation analysis of the variables revealed some notable and significant associations (Appendix B). As expected, High School Rank correlates with both ECTS and GP performance albeit this correlation is moderate (ρ_HSrank,ECTS= 0.261, ρ_HSrank,GP= 0.277). Furthermore, there is a negative correlation between High School Rank and both ALC (ρ = −0.307) and NC (ρ = −0.446), essentially meaning that programs that provides less NC and ALC tends to be populated by students with a generally higher relative standing in terms of High School Rank and vice versa. This can also be observed by comparing corresponding columns in Table 1 (for program NC percentage), Table 2 (for program ALC percentage) and Table 3 (for program mean High School Rank). The phenomenon is also illustrated in Appendix I by plotting program NC and program ALC against program mean High School Rank respectively, thus using the same data found in Tables 1–3.

Gender is moderately associated with High School Rank (women: ρ = 0.242). In addition, programs with less ALC occurrence seem to attract slightly more women (ρ = −0.223). The correlation between enrolment delay and NC (ρ = 0.189) suggests that programs with a higher level of native content attract more students who do not enrol directly after high school.

Dropping out or change of status seems primarily associated with low performance as indicated by the significant negative correlation with earned ECTS credits (ρ_Status,ECTS= −0.464) and GP (ρ_Drop-out,GP= −0.342). Furthermore, a stepwise multiple linear regression analysis of each of the independent variables yields four significant predictors, potentially influencing the student performance in terms of GP. In Table 5 and Appendix C, all variables are re-scaled to the range between 0 and 1: score on the diagnostic Math Test performed upon enrolment (β = 0.308); relative standing upon enrolment measured by High School Rank (β = 0.575); number of active learning assessment modules (β = 0.236); and time between high school graduation and program enrolment (β = 0.354). The coefficient of multiple determination for this model is R²= 0.187.

Table 5. Resulting model for predicting GP after stepwise multiple linear regression analysis.

	Unstandardized Coefficients		Standardized Coefficients	t	Sig.
	B	Std. Error	Beta
(Constant)	−0.195	0.110		−1.779	0.076
High School/Enr. Delay	0.354	0.093	0.159	3.796	0.000
High School Rank	0.575	0.114	0.245	5.024	0.000
Math Test	0.308	0.043	0.295	7.106	0.000
ALC (%)	0.236	0.045	0.208	5.216	0.000

4.1. Alternative models

Although the NC variable is excluded in the stepwise regression analysis, it approaches significance. If forced together with the initial four significant variables, a new model is established with R² reaching 0.192 (Table 6).

Table 6. Alternative model for predicting GP including both NC and ALC.

	Unstandardized Coefficients		Standardized Coefficients	t	Sig.
	B	Std. Error	Beta
(Constant)	−0.264	0.115		−2.288	0.022
High School/Enr. Delay	0.354	0.093	0.159	3.805	0.000
High School Rank	0.621	0.117	0.265	5.321	0.000
Math Test	0.321	0.044	0.307	7.325	0.000
ALC (%)	0.171	0.057	0.151	3.013	0.003
NC (%)	0.106	0.056	0.099	1.893	0.059

Furthermore, a model where ALC is simply replaced by NC as the sole predictor (besides the stronger High School Rank, Math Test, and High School/Enr. Delay) shows that NC seemingly has a similar influence on student performance as in the case with the initial model in Table 5. This model can be seen in Table 7 and Appendix F, where β_NC = 0.209 (compared with β_ALC = 0.236 in the initial model) indicates that NC might have a comparable influence on student performance as in the case of ALC. This model yields an R² of 0.180.

Table 7. Alternative model for predicting GP including NC instead of ALC.

	Unstandardized Coefficients		Standardized Coefficients	t	Sig.
	B	Std. Error	Beta
(Constant)	−0.172	0.112		−1.535	0.125
High School/Enr. Delay	0.310	0.093	0.139	3.348	0.001
High School Rank	0.567	0.116	0.242	4.883	0.000
Math Test	0.339	0.044	0.325	7.766	0.000
NC (%)	0.209	0.045	0.195	4.641	0.000

Investigating combinations of predictors for other dependent variables yields no significant models apart from Native Credits achievement with predictors including Hight School Rank (β = 0.748), enrolment delay (β = 0.343), and Math Test (β = 0.153) (R²= 0.12).

As one of the research aims of this study is to explore the impact of native content of the curriculum on student performance, it would be meaningful to analyse the ‘fulfillment of native content course modules’ in relation to non-native content. The dependent NCA, non-NCA, and NCA/non-NCA Ratio variables were established for this purpose. When comparing the performance of the students in relation to the nativeness of the courses (i.e. the NCA/non-NCA ratio data in Table 4), it is clear that the students earned more ECTS from native courses. On average, the students manage to earn double the amount of credits from native course modules compared to non-native course modules, and for some programs the relation is as high as a factor of four (P07), a difference which is also verified with significance (see Appendix G).

To elaborate further, the means of the NCA/non-NCA ratios are plotted against results on the Math Test, from 1 to 26 (Figure 2). A linear fit line yields an R² of 0.593. Having shown that the Math Test is strongly associated with the more general capability measure of High School Rank (see Appendix H), this observation consequently suggests that students with less relative capability upon enrolment are more likely to pass courses classified as native than courses classified as non-native. However, it is also reasonable that students who score low on the Math Test achieve a higher NCA/non-NCA ratio while typically failing to pass the large math courses, which are all classified as non-native. Students who do well on the Math Test typically pass the math courses and consequently achieve a lower NCA/non-NCA ratio.

Figure 2. Summary point plot of means of NCA/non-NCA ratios by math test results. Possible results on the math test are discrete values between 0 and 26.

Furthermore, a comparison of the differences between NCA and non-NCA (Appendix B) reveals that both variables are significantly associated with change of status (Status) and the Drop-out variable. The correlation is negative in both cases, but notably NCA shows a stronger association than non-NCA. The observation could be explained by the fact that these students were more dedicated regarding their choice of studies and tend be more persistent.

4.2. Analysis of program curricula

From the data collected in this study, other observations can be made that do not focus on student performance, but on comparisons between programs. Table 8 shows two color-scaled comparison matrices sorted by descending GP (left) and descending NC (right) along with group level means of the studied variables. Thus, the same data can be viewed in Tables 1–4. The bivariate correlation analysis of all studied variables found in Appendix B reveals that a high occurrence of ALC also is associated with high level of NC (ρ = 0.623). This means that programs with a high degree of native content also tend to have more active learning modules in the curriculum. This relation is also visualised in Table 8 in the right matrix.

Table 8. Grey-scaled comparison matrices sorted by descending GP (left) and descending NC (right), including group level means of studied variables.

		GP	NC	ALC	High School Rank	Math Test	Total Drop-out rate	NCA/non-NCA Ratio	Age			GP	NC	ALC	High School Rank	Math Test	Total Drop-out rate	NCA/non-NCA Ratio	Age
Program	P01	193	57	38	19.2	15.9	11.7	1.7	20.1	Program	P07	183	70	20	19.0	10.7	22.9	4.0	20.9
	P02	208	40	28	19.3	13.3	15.0	1.8	20.8		P10	156	70	26	16.7	–	11.1	1.0	23.8
	P03	219	40	30	18.9	13.8	11.7	1.3	20.5		P08	196	67	36	16.3	10.3	15.9	1.7	21.5
	P04	184	30	18	19.7	16.5	22.9	2.3	20.3		P09	192	67	28	15.4	8.3	15.8	1.8	22.8
	P05	214	50	22	19.4	14.2	15.0	1.6	21.1		P01	193	57	38	19.2	15.9	11.7	1.7	20.1
	P06	208	27	18	20.4	15.8	11.7	1.9	20.5		P05	214	50	22	19.4	14.2	15.0	1.6	21.1
	P07	183	70	20	19.0	10.7	22.9	4.0	20.9		P11	173	43	25	16.9	10.1	21.1	1.2	21.5
	P08	196	67	36	16.3	10.3	15.9	1.7	21.5		P03	219	40	30	18.9	13.8	11.7	1.3	20.5
	P09	192	67	28	15.4	8.3	15.8	1.8	22.8		P02	208	40	28	19.3	13.3	15.0	1.8	20.8
	P10	156	70	26	16.7	–	11.1	1.0	23.8		P04	184	30	18	19.7	16.5	22.9	2.3	20.3
	P11	173	43	25	16.9	10.1	21.1	1.2	21.5		P06	208	27	18	20.4	15.8	11.7	1.9	20.5

The comparison matrices in Table 8 also reveal the relation between High School Rank, the Math Test, and the resulting GP. As expected, high performing students, in terms of High School Rank and result on the Math Test, have higher GP. Moreover, drop-out rates show an inverse relation to the Math Test in particular. Another seemingly inverse relationship is the one between NC and High School Rank and the Math Test (i.e. programs offering less native content have more students with stronger relative standing at enrolment). Age, however, is related to a higher level of NC, which also is indicated by the correlation matrix found in Appendix B (ρ = 0.189). Notably, as pictured in Table 8, an association between NC and the NCA/non-NCA ratio is not obvious, which also is confirmed by the bivariate correlation analysis (Appendix B) (ρ = 0.023). Hence, there is no indication that students are more likely to pass native courses in programs where there is more native content in the curriculum.

5. Discussion

This study introduces and explores the concept of native curriculum content and provides an approach that quantitatively measures levels of curriculum nativeness. This section elaborates on the overarching research aims declared in 1.1, namely does native content, as defined and measured in this study, influence student performance in engineering degree programs? Could it be measured and used to compare and develop engineering curricula?

5.1. Effects of curriculum nativeness

The number of credits earned from courses classified as native for the student’s chosen career path are shown to have a significant relevance on student performance. Does this mean that native content is a driver of student performance in engineering profession degree programs? The prediction capability of the established models is low and cannot predict individual performance of students based on the identified statistical relationships. Nevertheless, although native content and course content characterised by active learning are shown to have less yet significant impacts on student performance compared to already known factors (e.g. high school grades, age, etc.), both factors seem to be associated with better student performance. Furthermore, considering the model in Table 6, the results seem to indicate that the presence of native content produces impacts comparable to that of presence of active learning modules. This finding alone calls for further studies that explore curricula from a native content perspective.

The ratio between earned credits from native courses and credits from non-native courses implies that students more frequently pass courses with a higher amount of native content. Ten out of the eleven studied groups of students showed a positive ratio, up to a factor of four. This finding constitutes a valuable insight for curriculum developers who seek to improve student retention and suggests that programs should adapt non-native courses to be more native. Connected to this finding, the study also indicates that students with fewer relative capabilities at enrolment (measured using the Math Test) (Figure 2) are more successful when a native content approach is applied. One obvious explanation for this, although not explicitly backed up by data of this study, would be that students find large first-year math courses (e.g. calculus and algebra curses) the most difficult and these courses are classified as non-native for all programs included in this study (Table 1). Nevertheless, given the strong relationship between high school rank and math proficiency (see Appendix H), further discussions are needed about how and when math is taught as well as about the amount of math necessary. At the studied university, these courses are often given cross curricula, usually engaging a very large number of students, and consequently the course plans of these courses have to ‘fit everyone’ resulting in very little, if any, native content in the eyes of the individual course participant. One plausible conclusion could be that students taking these math courses would benefit from a more native approach to content, perhaps through integration with other native courses. In addition, such an approach might improve student retention and performance.

The statistical analysis also confirms a number of previously known factors that have a significant impact on the performance of engineering students, such as high school grade, mathematical readiness and age (French, Immekus, and Oakes 2005), as well as the number of credits originating from assessment modules that indicates achievement from active learning. However, if further elaborated, the study of levels of active learning – as operationalised and measured in this paper – serves as an important complement to one of the main objectives of this study: assessing the usefulness and legitimacy of native content as a driver of student performance. Elements that promotes activation of students during higher engineering education have previously been shown to positively impact performance, which also is confirmed in this study. Hence, the finding that such learning styles is positively associated with level of native content speaks in favour of native content being a driver of student performance, which is further strengthened by the fact that for most studied programs students tend to perform better in the native compared to non-native courses.

5.2. Aspects of validity

As presented in this paper, measuring curriculum nativeness using the distribution of main subjects calls for questions regarding face validity. Are we really measuring what we think we are? To comment on this, we need to consider the way the NC measure is operationalised (i.e. no individuals could produce biased answers, since the classification already was in place at the time of the study). Thus, this study forms a new purpose for the classification system, which is in need of further evaluation. Nevertheless, one can argue for a kind of concurrent validity of the demonstrated operationalisation of curriculum nativeness in the sense that the ALC measure not only shows a strong correlation to the measure of native content, but also originates from the same source – i.e. the legal documentation of the curriculum and the syllabi they consist of.

Regarding the validity of the NC measurement itself, one has to consider the context where the underlying data of the measurement are being used (i.e. the main subjects and their use for classifying courses). At the studied university, each curriculum is developed and maintained by a committee that consists of teachers, researchers, and industry representatives. The committee has the ultimate responsibility for ensuring that the curriculum meets the expectations of the students already in the program, prospective students, and the future employers. It is from this viewpoint we should assess not only the validity of the NC measurement but also the validity of curriculum nativeness as a construct. Given how the studied faculty organises their curriculum development process, and if one had to point out a single guarantee formally responsible for identity aspects of a certain program, it would be the board of studies including the above mention committees. The board possesses the complete overview of the curriculum and makes use of the same classification system that the NC measurement is based on in order to control the composition of the curriculum.

Section 3.2 describes the operationalisation of a measure representing the amount of course content where the source to student achievement is likely to be based on learning through e.g. interactions, collaboration, cooperation etc. – a type of learning that we for this study has labelled active learning. The ALC variable is established as the reference for comparison of quantitative findings, and, as previously stated, the presence of native content in the curriculum has shown to impact student performance comparable to that of active learning. However, one issue when assessing active learning is how it should objectively be measured. For this study, it was decided that out of the four assessment methods – written exams, hand-in assignments, laboratory exercises, and project assignment – course modules examined by laboratory exercises and project assignments should be classified as active learning modules. It could be questioned if the amount of active learning could reliably be measured by studying the assessment methods alone. Obviously, this is a simplification, however it was judged as the most reliable and practical approach as existing non-biased data could easily be extracted from the course database at the studied institution. The alternative of gathering quantitative data from hundreds of courses, with as many different teachers, using qualitative methods would take a tremendous effort and introduce a larger bias towards the ones executing the study. The proposed measure is not dependent on the persons conducting the study and relies only on data already available at most higher education institutions, which in turn fosters reproducibility of the study.

Furthermore, it could be questioned whether hand-in assignments should be considered as non-active learning. In cases where such assessment modules consist of home assignments (e.g. essays to be submitted within a time limit), it is clear that the distinction from written exams is insufficient to alter the classification. However, some studied curricula include learning activities preceding the hand-in assignments that are characterised by many active learning attributes such as modules dealing with computer-aided engineering, where the students work with tutorials that are designed for and even based on interaction with the software (Hallberg and Olvander 2012). Such modules are often assessed and labelled as hand-in assignments although they apparently constitute an active learning experience for the student. However, the statistical analysis also includes variables for individual ratios of the four assessment methods, the variable for written exams is excluded in the stepwise regression model, but the ALC variable is not. This indicates that the ratio of both written exams and hand-in assignments are a better estimation of non-active learning content than the ratio of written exams alone.

Furthermore, the study reveals a clear and significant correlation between ALC and NC. That is, native content is more often represented in curricula that to a higher degree use active learning assessment modules. This seems reasonable because native content is implemented by examining and demonstrating phenomena and procedures of the profession that the curriculum is intended for. Exemplifying as oppose to theorising is a strong driver for situated and transactional learning styles, where the teaching is based on collaboration, self-directed problem-solving, and engagement in authentic learning environments (Kolb and Kolb 2005; Lave and Wenger 1991). In turn, such learning environments tend to use active learning as the assessment method (e.g. laboratory exercises). Not only does this speak in favour of validity for the NC measurement but also emphasises the relationship between active learning and curriculum nativeness.

The statistical analysis and the examination of curricula only covers the first two semesters. However, this choice of time frame has several motives that speak in favour of validity. The vast body of research on student attrition focuses on persistence beyond the first year of higher education, but studying the first year can help identify factors with significant influence. Studies such as Stewart, Lim, and Kim (2015) and Hutchison et al. (2006) tend to focus on pre-conditions and performance the first year after enrolment, examining factors such as family background, pre-collage performance, and attitudes. Furthermore, attrition beyond the first year is in itself problematic when studying student performance. Naturally, a cohort is most consistent upon enrolment, but as time passes more factors have to be considered, such as program switchers or the fact that programs often offer multiple tracks later in the program (typically year two or three), so there are different sets of courses for students in the same program.

5.3. Usefulness

Occasionally, program developers and other people responsible for curricula lack the complete picture of what their program looks like, especially in relation to other programs. Although this lack of perspective is an issue at large institutions such as the one studied here that provide multiple engineering profession degree programs that involve many program developers that need to communicate, plan, and share resources, smaller institutions can suffer from the same lack of perspective. The same applies for situations where competing programs of different universities are subject to comparison. The proposed NC and ALC measures presented in this study together with the formal way they are operationalised could be useful tools for all the comparisons outlined above, as well as for the development of individual curricula.

Students are usually interested in any quantitative measure that helps them make the right choices in higher education, but such measures ought to be of interest to program developers who want to attract and keep students. Not covered in this study, and thus yet to be investigated, is the response from the industry on the concept of native content; after all, they are the final customer in the education of engineers. It is likely that the industry would also find the concept of curriculum nativeness useful when assessing prospective employees who have recently earned their professional credentials.

Furthermore, a higher percentage of NC potentially enables teachers to more often connect teaching activities to the student’s future profession, consequently enabling for the students to reflect on the content being taught from a professional standpoint, which in turn adds to the profiling/encouraging effect. In addition, the more native content early on during a multiple year program, the higher the chance that students settle on ‘the right choice of a career’ sooner rather than later, avoiding wasting time and resources on their way to a career.

5.4. Outlook and critical reflections

From a statistical methodological perspective, one could debate the appropriateness of including the NC variable in the regression analysis presented in section 4, even though is does not reach significance in the stepwise generated models. Although widely used in the literature for presenting and interpreting results from statistical data analysis (see e.g. Carpenter et al. 2007; Smith et al. 1999), the usage of terms such as ‘approaching significance’ has also received justified criticism (see e.g. Schafer 1993). Nevertheless, we judge this observation worth reporting of given the circumstances. Moreover, the literature sometimes suggests using effect size (i.e. Cohens t-test) as a complementary measure in cases like this (e.g. Daniel 1998). However, given the design of this study (i.e. the absence of before/after data) such measures cannot be deployed. Moreover, the usage of the score on the diagnostic Math Test as an independent variable constitutes a methodological limitation with respect to reproducibility of this work, as such tests are absent at many institutions.

As this study focuses on the first year of the programs, lower percentage of NC does not necessarily mean that the program as a whole involves fewer profiling elements, but could imply that the curriculum developers have chosen to introduce such content later in the program, a possibility that deserves further investigation. However, it is plausible that there are fundamental differences between types of degrees regarding, for example, prerequisite courses, where profiling content must be preceded by non-profiling content. Other programs might offer specific and more profiling tracks as early as the third semester, resulting in less native content during the first two semesters. However, by studying the NC measure, these differences are brought up for discussion, which is certainly valuable for program developers.

Furthermore, related to the processes of knowledge acquisition, there is reason to problematise the reference to engineering identity as the founding construct for curriculum nativeness. For example, while discussing the development of disciplinary knowledge (as opposed to domain or discipline-general knowledge), Stevens et al. states that ‘distinctly different forms of knowledge are counted as disciplinary knowledge in different situations, at different times, and by different people […] the assumption of a stable body of knowledge that is progressively acquired by members of a discipline is challenged’ (2008). From the perspective of this study, this view highlights the challenges associated with developing a quantifiable measure for curriculum nativeness. Moreover, the method of characterising the native subjects for each of the programs (described in Section 3.1) could be further developed by an expansion involving multiple stakeholders (e.g. the students themselves) and their perceptions of nativeness of curriculum elements. Such measures would also add to the generalizability and repeatability of the concept of curriculum nativeness.

Finally, regarding future work there are multiple take-aways from this contribution; first and foremost, the operationalisation of the NC measure needs to be verified at multiple institutions. Also, given the way math is taught at the studied institution and the impact this have on the statistical analyses, influence of native content needs to be studied on programs where math is being taught in a more integrated manner.

6. Conclusion

This study introduces the novel term curriculum nativeness with reference to characteristics of engineering curricula and engineering profession degrees. The study also introduces a quantitative way to measure the amount of native content of engineering curricula by using the course classification system at the studied institution. Assessment of validity and usefulness were based on a statistical analysis of the performance of nearly 800 students enrolled in eleven engineering profession degree programs. Statistical models were established to assess the correlation between the performance of the students, relative high school grades, mathematical readiness, native content in the curricula, as well as the amount of course content with assessment methods indicating learning styles that promotes student interaction, i.e. different type of active learning. Results indicates that the level of nativeness in a curriculum impacts student performance comparable to that of such learning styles. In addition, the study indicates that credits are more frequently earned from native courses compared to credits from non-native courses. Furthermore, programs with a high degree of native content also seems to have a higher degree of active learning modules. Finally, curriculum nativeness and levels of native content are perceived as a useful measure for comparing engineering curricula. However, this study also calls for further assessment of the proposed concepts and measures.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Peter Hallberg

Peter Hallberg is a junior lecture and former director of studies at the division of Machine Design, Department and Management and Engineering, Linköping University, Sweden. He is mainly active on the Mechanical engineering bachelor and master programs, within the fields of computer aided engineering and product development. He also chairs the committee responsible for the curriculum design and development of the Mechanical engineering bachelor program at Linköping University.

Johan Ölvander

Johan Ölvander is a professor in Engineering Design at the Department of Management and Engineering at Linköping university, Sweden. Dr. Ölvander’s research interest revolves around product development and how it could be enhanced by incorporating computational tools. He has authored more than 130 papers in international journals and conferences. Johan Ölvander is also chairman for the education board of Mechanical Engineering and Design at Linköping University, including the overall responsibility for curriculum development for five master programs and three bachelor programs within the broad area of mechanical engineering.