Metacognitively ALERT in science: literature synthesis of a hierarchical framework for metacognition and preliminary evidence of its viability

ABSTRACT

The development of student metacognition has the potential to provide some of the greatest learning gains in science education, even outstripping the contribution of general intelligence. While models for metacognition are in broad agreement about their nature, they vary widely in essential elements and the relationships between them, especially between metacognitive knowledge and metacognitive skills. Recent systematic literature reviews have not untangled the concept of metacognition as they are not suited to crafting a synthesised conceptualisation for a controversial topic. This article, then, presents an integrative literature review of metacognition studies that draws together metacognitive knowledge and metacognitive skills into a hierarchical framework. The framework comprises, from the foundational level, metacognitive knowledge, called self-Aware of cognition, then various metacognitive skills; self-Monitor cognition, self-Evaluate cognition, self-Regulate cognition and self-Transfer cognition (AMERT). As a preliminary test of its viability, the AMERT framework is used to analyse interview data in which there was evidence of rich metacognitive thinking by students in the fourth, research-focused, year of a science degree. Rich epitomising statements were found in interviews for each level of the AMERT hierarchy, providing tentative evidence of its viability for understanding metacognitive processes when students learn in science.

KEYWORDS:

• Metacognition
• cognitive labels
• self-awareness
• self-monitoring
• self-regulation
• transfer of cognition

Introduction

A recent review of studies on approaches to learning in science education found that the most used theoretical frameworks pertained to metacognition (Google et al., 2023). Metacognition is portrayed in the literature as qualitatively similar across schooling and universities (Perry et al., 2019) and beneficial for learning across these sectors (Howe et al., 2019; Perry et al., 2019). Research has determined a number of educationally important characteristics of metacognition that make its investigation a priority in science teaching and learning (Adler et al., 2016; Zepeda et al., 2019). There have even been claims that metacognition promises a more substantial contribution to learning than general intelligence (Veenman et al., 2004).

Benefits of metacognition

Many studies have shown the potential benefits of well-facilitated metacognition for learning and problem solving in science (Adler et al., 2016; Schraw et al., 2006; Taasoobshirazi & Farley, 2013; Thomas, 2013; Zepeda et al., 2019), a result mirrored in studies on Artificial Intelligence (Cox et al., 2022). Recent studies on students have shown substantial gains in learning due to metacognition across primary, secondary and tertiary years, including primary school Mathematics classrooms where the metacognitive talk was stronger versus those where it was weaker (J. M. Smith & Mancy, 2018); secondary school Physics, where student metacognition measures correlated with higher levels of performance (González et al., 2017); and Chemistry classes for those in initial teacher education (Adadan, 2020). For example, after inquiry-based instruction, participants with high-level metacognitive knowledge, when compared to participants with low-level metacognitive knowledge, were more likely to change their non-scientific conceptions of lunar phases to science-oriented ones (González et al., 2017). Additionally, the participants with high-level metacognitive knowledge developed a more coherent and consistent understanding of gas behaviour, and retained their scientific understanding months after instruction (González et al., 2017).

The importance and benefits of metacognition are mirrored in recent studies that are broader than Science Education, but relevant to it, where student metacognition is as follows: vital for design thinking, including experimental design (Kavousi et al., 2019); pertinent to integrative learning (Youngerman et al., 2021); crucial for problem-solving, involving an ‘extensive entanglement between metacognition and manipulation in working memory’ (Shea & Frith, 2019, p. 568). A meta-analysis of metacognition studies asserted that some facilitated metacognitive strategies substantially increased the expected amount of student learning when compared to standard instruction (Hattie & Zierer, 2017; Hattie, 2008).

Definitions and characteristics of metacognition

There has been broad agreement on what comprises the core components of metacognition, including across school and university studies (Lai, 2011). In general terms, metacognition is the form of cognition whose subject is cognition (Acar, 2019) or, colloquially, ‘thinking about thinking’ (Gough, 1991). Ongoing agreement concurs with Flavell’s (1976) early characterisation that metacognition consists of metacognitive knowledge and metacognitive experiences and that metacognition ‘regulates any aspect of any cognitive endeavour’ (p. 906). A recent study set up in physics similarly operationalised metacognition as comprising two categories: First, an awareness of certain skills or strategies and resources to perform certain tasks effectively; second, the ability to use self-regulatory mechanisms to ensure the successful completion of that given task’ (Wade‐jaimes et al., 2018, p. 715). Self-awareness and self-regulation of cognition are terms repeatedly used in the literature on metacognition (Schraw et al., 2006). While cognitive strategies are used for undeniably complex activities, such as problem solving or planning tasks, these activities may or may not involve metacognition (Panahandeh & Asl, 2014). Metacognitive strategies are employed when students self-monitor, self-evaluate, control and/or understand the cognition used during those complex activities (Panahandeh & Asl, 2014). Here, self-monitoring cognition means that students are actively checking their thinking, using their mind’s eye to look at their learning processes. Self-evaluation of cognition occurs when students weigh-up their thinking and make judgements about the appropriateness and effectiveness of that thinking and its fitness for present purposes. Control, or more commonly, self-regulation, means to actively and intentionally change one’s thinking processes.

Recent exploration showed the primacy of the affective domain to enable higher levels of metacognition, such as self-regulation (J. Willison et al., 2020). There is a strong and integral affective element to cognition and metacognition, however affect is outside of the scope of this current article. Research has also explored group metacognition (J. M. Smith & Mancy, 2018) but this too sits outside of this current article’s focus on individual metacognition.

Challenges of metacognition

While the literature shows the potential of student metacognition for learning in science, there are numerous challenges that are seemingly endemic to the concept. Despite this potential, the advantages of metacognition may be overstated due to problems associated with measurement (Akturk & Sahin, 2011), and these are, in large part, due to the lack of agreement on specific aspects of metacognitive models (Azevedo, 2020). Almost 10 years ago a literature review on metacognition in this journal concluded that ‘bridging the gaps in research regarding pre- and in-service teachers’ knowledge in the area of teaching metacognition in science education remains a critical challenge’ (Zohar & Barzilai, 2013, p. 154) and one of the great importance due to the potential contribution of metacognition to student learning in science. Zohar and Barzilai noted that ‘the field still awaits future theoretical work that would provide a thorough theoretical analysis and integration of the extensive literature in this area to attain a unified definition of this construct’ which is ‘inconsistent and lacks coherence. Numerous researchers define metacognition and its components differently’ (2013, p122). This inconsistency and lack of coherence are endemic and current (Azevedo, 2020) and without consistency, any systematic literature review of metacognition is subject to the same problems of definition, which ripple onto construct validity and raise questions about measurement and external reliability. Some blame for enduring inconsistency of terminology has fallen on Flavell’s (1976) original definition of metacognition for addressing ‘both knowledge about cognition and regulation of cognition’ (Georghiades, 2004, p. 372), aspects that should be kept separate, from that perspective.

The lack of clarity about metacognition is amplified by the many different terms that are used to indicate it, including metacognitive awareness, metacognitive knowledge, higher-order skills, thinking strategies, learning strategies, self-control, self-regulatory strategies, and monitoring of comprehension (Veenman et al., 2004). The different terminology and measures add confusion to our understanding of the nature of metacognition. These differences can lead to arguable results that contradict between or even within studies, such as one study’s ‘… unexpected finding … that the more naïve belief in certain knowledge may begin to have a more positive effect on use of metacognitive strategies among students … ’’ (Paulsen & Feldman, 2005, p. 386). The quote suggests that increasing naïvete causes metacognitive strategies to improve. While such a counter-intuitive finding may be justifiable, it is more likely to be a result of measurement problems due to sub-optimum constructs for metacognition.

Challenges of measurement

Without a consistent and applicable framework for metacognition, teaching and learning effectiveness is limited, and research on metacognitive processes stymied (Barzilai & Chinn, 2018). Possibly, as a function of this lack of consistency, studies of tools to enhance student thinking have focussed primarily on the cognition involved, even though it is metacognition that promotes and regulates ‘thought processes concerning the source, nature, and justification of knowledge’ (Tang, 2020, p. 474). Some studies have recognised the limitations in terms of measuring interventions in metacognition. For example, a study set in physics classes that were using ‘metacognitive tools’ returned ambiguous results in short to medium timeframes, finding that ‘students most likely failed to either deeply process the information provided during metacognitive training and were either unable or unwilling to apply this knowledge to their learning supported by the simulation’ (Moser et al., 2017, p. 959). The study concluded that ‘Further, training over an extended period of time might have been beneficial’ (Moser et al., 2017, p. 959). The study’s negative finding of the short-term use of metacognitive tools in science education may provide a cue about time for development of metacognitive skills and/or may suggest that the training, measures or constructs the research was based on are faulty.

Resolving the relationship between metacognitive elements

To deal effectively with the dilemmas and problems above, theoretical progress in the field of metacognition has been called for before further data is generated (Sobocinski et al., 2020). Without a sound theoretical model, measures of and outcomes attributed to, metacognition may be overestimated due to the reliance on quantitative data generated with instruments that have debatable construct validity. In the rush to quantify and generate generalisable results and guidelines, there may have been and continue to be measures of metacognition that are not valid or reliable.

The current divergent perspectives on metacognition may be resolved if the relationship between metacognitive elements was clear. Such a clarity of fundamental metacognitive elements and the relationship between them would enable a consistent and coherent metacognitive framework. Without clarity of the construct of metacognition, systematic literature review methodology is not suited to resolving the problems of definition. The variable definitions, terminology, constructs and measures make systematic reviews of the literature on metacognition premature. The prematurity is because the synthesis of results about metacognition that is defined in divergent ways is not meaningful, and so recent systematic reviews of metacognition and related concepts have not clarified the relationship between elements (Craig et al., 2020; Perry et al., 2019). This is because divergent constructs yield different, and even discrepant, measures and data and cannot be usefully combined. This paper uses an integrative literature review methodology to resolve terminology differences and clarify the relationship between metacognitive elements. A distinction between systematic and integrative literature reviews is made in the methodology.

The primary aim of this paper, then, is to present a hypothesis for the relationship between metacognitive elements in the literature in order to clarify the concept of metacognition. The hypothesis is the basis for a hierarchical metacognitive framework, presented below, that provides utility for researchers to test these relationships. The secondary aim of this paper is to conduct a preliminary test of the viability of the metacognitive framework for articulating the nature of metacognition that is evident in rich qualitative data. If the framework subsequently proves to be viable in other studies that are set up in diverse contexts and inform reliable measures of metacognition, it may then provide a functional understanding of metacognition for empirically sound research. With enhanced construct validity for informing primary research, the framework could ultimately inform effective and insightful literature reviews of metacognition.

From this paper’s perspective, many articles on metacognition do not deal with metacognition exclusively but mix cognitive and metacognitive elements. Because there has been confusion about metacognition at many levels, this paper is written for a readership that is new to the concept or seeking a clarification of it, including both researchers and teachers. A clearer conceptualisation of metacognition may help bridge the gap between metacognitive theory and science education practice.

Two-part structure of this paper

This paper is presented in two parts that reflect the two aims. Part A addresses the primary aim of the paper by providing an integrative literature review methodology and then outlining the emerging hypothesis and resulting metacognitive framework. Part A answers the question ‘what is the nature of core elements of metacognition and their inter-relationships in the literature?’ The resulting framework comprises the following metacognitive elements: self-Aware of cognition, self-Monitor cognition, self-Evaluate cognition, self-Regulate cognition and self-Transfer cognition (AMERT).

Part B addresses the secondary aim of the paper, presenting the methodology for the generation of rich, qualitative data in interviews with science students and the analysis of the data using the metacognitive AMERT framework from Part A, followed by the results and discussion. Part B addresses the question ‘does the AMERT framework provide a viable understanding of student metacognition evidenced in interviews with students?’

Part A: an integrative literature review towards a framework for metacognition

Methodology: integrative review of literature

This paper adopted an integrative review of literature, which ‘… reviews, critiques, and synthesises representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated … to resolve inconsistencies in the literature’ (Torraco, 2016, pp. 404–405). An integrative review methodology was chosen to overcome the problems associated with systematic literature reviews of metacognition when the conceptualisations and measures of cognition show marked variation. An integrative review of the literature is a qualitative research design, where the attempt is not for representative, statistics-based or generalisable results, but for a deeper understanding of phenomena, in this case of metacognition (Toronto, 2020, p. 2). Integrative reviews use searches from the literature and ‘employ narrative and/or thematic analysis with descriptive and qualitative synthesis’ (Toronto, 2020, p. 3). The integrative review approach ‘supports a wide range of inquiry, such as defining concepts, reviewing theories, or analysing methodological issues’ (Toronto, 2020, p. 2), making it ideal for developing a deep understanding of metacognition. In contrast, a systematic review of qualitative studies would determine major themes, combine results and represent these in quantified terms (Mills et al., 2005, p. 1081). This integrative approach looked ‘more broadly at a phenomenon of interest than a systematic review and allows for diverse research, which may contain theoretical and methodological literature to address the aim of the review (Toronto, 2020, p. 2). Where systematic reviews have a result-orientation (e.g., Craig et al., 2020; Perry et al., 2019) integrative reviews are orientated towards definition or construct clarification. The aim was not to conduct an exhaustive search, such as in a systematic literature review; this would yield too many papers to integrate and so be counterproductive. The aim of this integrative review was to capture a broad range of ideas about metacognition and to determine a coherent synthesis of the literature that could provide a hypothesis about the nature of metacognition.

The literature search was limited to journals that pertained to education and psychology and to the title of articles to ensure only those articles whose focus was on the major aspects of metacognition mentioned above were included. The key search terms were metacogniti*, self-aware* and self-regulat×. The databases searched were ERIC, Education Research Complete and A⁺ Education and, additionally, Google Scholar search engine was used.

From the preliminary search, journal names, article titles and abstracts were scanned to determine if articles concerned metacognition in education. If so, articles were read to determine metacognitive elements represented in each article and any relationships stipulated between elements. Literature that provided relationship cues for elements of metacognition was especially valuable because these explicitly addressed the primary research aims and questions.

The process used to integrate the literature into a framework follows:

• Identified elements of metacognition that were prevalent in the literature.

• Excluded those elements that are not metacognitive by default.

• Identified parallel terms and adopted one, frequently used, term. For example, ‘control’ or ‘self-control’ is used in some literature, but ‘self-regulation’ is used more commonly.

• Identified where pre-conditional elements are necessary for an element of metacognition to be realised.

• Synthesised a framework using the key metacognitive elements and the relationship between them.

These analyses of the literature led to the development of a hypothesis for the relationships between different elements of metacognition and provided a basis for a conceptual framework for metacognition.

Results: integration of literature

The problems of developing a consistent and coherent understanding of metacognition from the literature were identified as:

- conflation of the roles and processes of metacognition

- conflating non-metacognitive elements with metacognitive ones.

- definition of terms: same terms, different meaning; different terms, same meanings.

- unclear relationships articulated between metacognitive elements.

While the first three problems are endemic and current, views of the relationship between metacognitive elements, have seen a partial, albeit unclear, relationship emerge among the metacognitive skills in recent literature.

Metacognitive processes only

Metacognition is frequently conflated with other forms of complex cognition (Tang, 2020). A recent commentary on the ongoing discussion in the field has been the complex interaction between cognition and metacognition that continues to challenge researchers … This represents a dilemma of having a higher-order agent overlooking and governing the cognitive system while also simultaneously being part of it (Azevedo, 2020, p. 92). One controversial aspect of this dilemma concerns the status of metacognition as an order of thinking that is higher than other forms of cognition. When metacognition is activated, its role is to understand, diagnose and govern other forms of thinking, like a cognitive boss. However, there are times when learners are not and need not be, metacognitive for the management of cognitive processes. At times, other forms of cognition have a priority over metacognition, for example, due to the limited capacity of working memory and the high load placed on it by metacognition (Martinez, 2006; Shea & Frith, 2019).

In terms of governing thought, metacognition may be called higher order, but in terms of processes, it is not higher than other forms of cognition. For example, analytical processes are used during cognition and metacognition: for cognition the analysis is of external phenomenon, whereas for metacognition it is of the internal world of one’s own thinking and sentiments. Metacognition seen as higher-order-thinking emphasises the role of metacognition to govern cognition, like a ‘top layer’ of cognition, an idea evident in a hierarchical model for cognition (Kayashima et al., 2011). A higher-order perspective is useful to understand what metacognition does, but not what it is, its processes or how it may be developed or taught. Disambiguation of roles and processes helps clarify terminology that is appropriate for elements of metacognition. The elements presented below focus on the cognitive processes that comprise metacognition, rather than its role.

Metacognitive elements only

Any framework for, or measure of, metacognitive processes must consider aspects that are genuinely metacognitive. The inclusion of non-metacognitive elements in metacognitive models compromises construct validity and therefore affects measurements in questionnaire design and observational studies that are based on these models. A commonly used, efficient method of evaluating metacognitive skills is self-report questionnaire, however there ‘is a wide range of questionnaires that measure a variety of components of metacognition’ (Craig et al., 2020, p. 156), and the variation in components flags uncertainty about the construct validity of such surveys. As an example of including non-metacognitive elements, one study’s Likert scale items that were used to measure metacognitive knowledge included ‘I really pay attention to important information’ (Acar, 2019, p. 658). Paying attention to important information may reflect metacognitive knowledge but could just as easily be interpreted as learning for the test and lacking in metacognitive characteristics.

Observation-based research too includes non-metacognitive elements. In a well-cited study employing observation-based research that used quantitative measures for metacognition, the ‘frequency of scrolling back to earlier experiments’ was used as ‘a positive indicator of metacognitive skilfulness’ (Veenman et al., 2004, p. 98). Therefore, a student who did not scroll back to previous experimental data was measured on this item as having lower metacognition than a student who did. However, some students who did not scroll back may not need to because they deeply understood the experimental design or numerous other possibilities, which may not relate at all to metacognition. The same study (Veenman et al., 2004) characterised hypothesising and drawing conclusions as metacognition. However, these sophisticated forms of thinking are forms of cognition, not metacognition, for student awareness or control of thinking processes do not, by default, occur when hypothesising or drawing conclusions. In another study, Wade‐jaimes et al. (2018, p. 718) states that the teacher ‘would ask metacognitive questions that require the student to make predictions about a novel circuit configuration, such as “What will happen in the circuit when you add/remove this component?”’ Predicting the results of removing a component is a focus on the phenomena, not on students’ cognition and therefore does not necessarily prompt student metacognition. The elements listed below are metacognitive and other forms of cognition are not considered.

Synonyms for metacognitive elements

Synonymous terms used in the literature create redundancies that lead to ambiguity. Metacognitive knowledge and self-awareness of cognition are synonymous in the literature. ‘Self-aware’ is appropriate for a framework to reveal metacognitive processes rather than the noun of ‘knowledge’. Self-regulation and planning are used synonymously in the literature. ‘Planning’ is a common term in the literature; however, it is typically associated with a ‘phase’ concept, as a first step of metacognition in a sequence (e.g. Young & Worrell, 2018). The idea of students first planning what to think before they engage in an activity is really a focus on the role of metacognition to govern thought. When metacognitive planning does happen, it needs to involve the will and decision to change or consolidate current cognition, and this is the realm of the term ‘self-Regulation’. Planning is a form of self-regulation of cognition, and so having both terms in a framework creates confusion. ‘Self-regulation’ is an almost ubiquitously used term for defining metacognition, so ‘Planning’ should be excluded from a metacognitive framework except to unpack what self-regulation of cognition involves.

Transfer, too, is used synonymously with self-regulation but also has ambiguous aspects. Transfer may refer to the use in a different context of content knowledge, such as knowledge of calculus developed in mathematics and used in experimental physics. In a metacognitive framework, however, the transfer must only be related to the transfer of cognition. ‘Self-Transfer of cognition’ is used in this article as a clear metacognitive term that ensures metacognition is only addressed.

Self-Transfer of cognition is an intentional act of applying cognition learned in one context to another context, for example intentionally using thinking developed when analysing physics equations to analysing equations in mathematics. Self-Transfer of cognition is a form of self-regulation that wilfully transports the application of cognition to a context in which it was not developed. If self-Regulation takes place, then self-planning is a natural component, as shown below. However, during self-Regulation, Transfer of cognition frequently does not happen because it is a complex and hard-to-achieve form of self-Regulation, leading to common frustration over student inability to transfer (Jackson et al., 2019). Therefore, self-Transfer of cognition needs to be articulated explicitly in a metacognitive framework, albeit in a way that recognises it as a form of self-Regulation.

Relationship between metacognitive elements that demonstrates process

A common feature in the literature is the unclear relationship between metacognitive elements. Indeed, ‘… the majority of researchers separate metacognitive knowledge from metacognitive skills’ (Perry et al., 2019, p. 485) and the separation does not indicate or allow for relationships between them. For example, Tarricone (2011) provided ‘the most comprehensive conceptual framework of metacognition … [which] clearly distinguishes between metacognitive knowledge and skills’ (Azevedo, 2020, p. 92), however, the article’s diagrams and tables do not portray the relationship between them. Because these representations do not demonstrate the connections that exist between metacognitive knowledge and skills, they leave a need for clarification about this relationship, a clarification that may provide insight into the development and understanding of metacognitive processes (Bannister-Tyrrell et al., 2014).

Another source of unclear relationships is the use of a ‘phases’ concept when representing metacognitive elements (e.g. Azevedo, 2005) because the implication of a linear sequence from one phase to the next is not true of metacognitive processes. The sense of sequentiality confounds our understanding of metacognitive processes, because metacognition is far more complex and iterative than a sequence of steps or phases, whether represented linearly or cyclically. As noted above, the sequentiality suggested in planning cognition may be true for the role of metacognition but not for metacognitive processes.

Metacognitive terms that fit the criteria

The following terms for elements of metacognition fit the above criteria and were typical in the literature. Each term used for a metacognitive element reflects a student’s active thinking about their inner cognitive world, the processes of metacognition, so verbs are prioritised over nouns in the framework. Overall, the following terms for metacognitive processes were chosen from the literature to prioritise consistency and coherence by removing redundancies, ambiguities and non-process words, and to maximise a sense of relationship between the terms.

Self-Aware of Cognition. Self-Awareness of cognition involves a learner’s realisation of the processes and skills involved in cognition (Efklides, 2008). As noted above, Self-Aware of cognition is a synonym for ‘metacognitive knowledge’, but the former term is prioritised here for consistency in using verbs and choosing terms that together emphasise the relationships between metacognitive skills. As students become aware of their cognitive processes, this metacognitive knowledge may be insightful, empowering and enabling, or trivial, minimal, incorrect or incoherent, especially if it is not introduced and supported expertly. Self-Awareness implies articulation to one’s self of the internal realms of thinking, and such awareness benefits from appropriate and communicative labels for different cognitive processes (Barzilai & Chinn, 2018; Cellar & Barrett, 1987).

Cognitive Labels. Cognitive labels name cognitive processes or skills and so enable awareness of one’s own thinking, the essential element of metacognitive knowledge. Cognitive labels may be formed by the learner himself or herself, as his or her pre-verbal, ‘core’ metacognitive knowledge emerges and becomes explicit (Goupil & Kouider, 2019) or may involve labels imparted by siblings, parents or teachers. Teaching metacognitive self-Awareness can involve making explicit such cognitive labels, which become a ‘metalanguage’ that facilitates student self-awareness (Cellar & Barrett, 1987). Cognitive labels may be framed by frameworks for cognition, such as the taxa of Bloom’s (1956) taxonomy, including understand, apply, analyse, synthesise and evaluate.

Cognitive labels must be internalised to effectively become self-Awareness of cognition, i.e. there must be a development from labels given, say by a teacher, to a deeper understanding by the students (Cellar & Barrett, 1987). Teacher impartation of cognitive labels may be a good beginning, but it may be a long way from student self-Awareness, especially if there is dissonance between the students’ existing knowledge of their cognition and the teacher’s labels. Moreover, what a teacher means by a label may be very different to the understanding of that label that each student constructs.

Self-Awareness of cognition as a Co-Condition for Other Forms of Metacognition. When considering the relationship between metacognitive knowledge and skills, there is an unambiguous dependency of metacognitive skills on metacognitive self-Awareness. Without metacognitive self-Awareness, cognitive skills necessarily function without the ‘meta’ component (Weil et al., 2013). The brain unconsciously and subconsciously controls numerous cognitive processes and the neo-cortex is the executive region of the brain that dictates much thinking when a non-reflex response is required (Stiles, 2008). However, Self-Awareness of cognition is a conscious act of knowing what one is thinking and therefore must remain at the level of consciousness in order for a student to operate in metacognitive mode, and so is a precursor for other forms of metacognition. To be metacognitive, metacogntive skills must operate with an Awareness of cognition as a pre-condition (Weil et al., 2013). This dependency of other metacognitive skills on self-Awareness of cognition has the same sense as Maslow’s hierarchy of needs (Maslow & Lewis, 1987) where the lowest level of ‘physiological needs’ is not only a pre-condition for all higher levels to happen effectively but is also a co-condition that must remain in place.

While metacognitive knowledge and skills are at times represented as a dichotomy, unless students can articulate their own metacognition to themselves and know what it is they monitor they cannot self-monitor cognition, let alone engage in any other metacognitive skills. The term self-Aware of cognition rather than metacognitive knowledge enables this foundational aspect of metacognition to be placed on a skills continuum, rather than being disconnected from it or vague in relationship to it.

Other metacognitive skills

Other core metacognitive skills are indicated in numerous similar-but-different lists. For example, one list includes that pupils ‘monitor, plan, evaluate and regulate … as well as strategies that consciously help pupils solve novel problems (Perry et al., 2019, p. 486). Another list of skills, based on self-awareness, is students ‘monitor, evaluate, control’ (Panahandeh & Asl, 2014, pp. 1409–10) cognition. The skills in each list are in an intentional order and together suggest a core list of metacognitive skills based on self-Awareness of cognition:

• Monitor

• Evaluate

• Regulate

• Transfer

As argued above, to plan cognition’ is a form of self-Regulation, and control is a synonym for regulation and so not stated in this top-level list of metacognitive skills. The transfer in the first list is phrased as ‘solve novel problems’, with the emphasis on ‘novel’ being contexts new to the learner. However, Transfer is by far the most common term used for this concept in the metacognition literature (Ford et al., 1998; Heggen, 2008; K. Smith et al., 2007).

Taken together, this provides metacognitive processes as comprising self-Awareness, self-Monitoring, self-Evaluation, self-Regulation and self-Transfer of cognition, AMERT for short. AMERT is represented in Table 1, below. We argue that the order is important, not in a phase sense, but a ‘co-condition’ sense. For example, ‘self-Evaluate’ is after ‘self-Monitor’ because evaluation requires ‘data’ to evaluate, data that is harvested in the self-Monitoring of cognition. Therefore, self-Monitoring cognition is a pre-condition for self-Evaluating cognition.

Table 1. The hierarchical AMERT framework for metacognition, showing five levels of metacognition, description and key verbs for each level.

Lower levels are pre-conditions and co-conditions for higher levels		Levels ofMetacognition	Description	Key verbs
		Self-Transfer cognition	Intentionally uses and adapts cognitive processes to suit contexts new to one’s self. A form of self-Regulation.	Adapt, apply
		Self-Regulate cognition	Intentionally changes or consolidates thinking processes, based on self-Evaluating	Plan, change, consolidate
		Self-Evaluate cognition	Compares and contrasts current thinking with desirable thinking, based on self-Monitoring.	Compare, contrast, judge
		Self-Monitor cognition	Actively gauges thinking processes, based on self-Awareness.	Detect, gauge, check.
		Self-Aware of cognition	Conscious of and understands own thinking processes, uses internalised cognitive labels.	Metacognitive Knowing & understanding

Rhodes notes this pre-conditional sense, evident in the relationship between elements, observing that ‘monitoring of cognition plays a causal role in self-regulation of cognitive processes’ (Rhodes, 2019, p. 168). However, it is more accurate to say ‘co-conditional processes’ rather than ‘causal role’ because causal suggest that, if self-Monitoring occurs, then self-Regulation will occur. Co-conditional means that one element has to be happened for another element to be enacted but does not imply that it will happen. We then say that self-monitoring is co-conditional for self-Evaluating to happen. These skills are co-conditions of other skills entailing a hierarchical relationship in the framework introduced next. The elements higher-up in the hierarchy depend on the elements below not only happening but continuing to occur.

For self-Monitoring of cognition to enable self-Regulation of cognition such monitoring must be appropriate and fit for purpose (Mueller et al., 2016) and so self-Evaluation is the essential bridge between them. While Rhodes jumps from self-monitoring to self-regulation and skips self- evaluation, authors typically see self-evaluation as vital and based on self-monitoring, e.g. Alghamdi (2021).

Self-Monitor Cognition. Self-Monitoring explicitly gauges what thinking is currently taking place, based on student self-Awareness of cognitive activities. Self-Monitoring will be off-track and impractical if an individual’s self-Awareness is based on their own cognitive labels that are not functional or a shallow understanding of appropriate labels. Self-monitoring activates thought processes that encourage critical thinking in the student (Facione & Facione, 1996), and without self-Monitoring, there are no internal data for effective self-evaluation of cognition. Therefore, self-Monitoring is foundational to, and precedes or occurs simultaneously with self-Evaluating in AMERT.

Assertion 1:

Self-Awareness of cognition is a pre- and co-condition for self-Monitoring cognition

Self-Evaluate Cognition. Self-Evaluation of cognition involves decisions about whether current cognition is good enough to get the job done based on each student’s thinking patterns (Elder & Paul, 2004). The information on which to make such judgements is provided by self-Monitoring of cognition and provides the data for self-evaluative contrast and comparison. Evaluative thinking is not dependent on metacognition; however, self-Evaluation of cognition is a metacognitive skill because it is focused on cognition. Self-Evaluation of cognition, based on self-monitoring, compares current cognition and perceived needed cognition and so provides a way to ‘identify whether epistemic processes are indeed resulting in good epistemic outcomes’ (Barzilai & Chinn, 2018, p. 369). The contrast between where one is cognitively and where one needs to be provided the internal, self-Evaluative impetus for a change in cognition that involves self-Regulation.

Assertion 2:

Self-Monitoring of cognition is a pre- and co-condition for Self-Evaluating cognition

Self-Regulate Cognition. Self-Regulation of cognition is typically characterised as a change that works to improve or optimise student learning processes (McInerney et al., 1997). However, self-regulation involves not only intentional change but also intentional maintenance and consolidation of currently effective cognitive strategies, especially when managing one’s behaviour and undertaking tasks to achieve an intended goal, process, desire (Efklides, 2008; Pintrich, 1999; Samsonovich et al., 2008) or reference point (B. J. Zimmerman, 1995). Self-regulation involves ideation of, or planning, a desired cognitive place to be (Efklides et al., 2001) and volition to go or stay there.

Because self-Regulation operates on the basis of knowing what cognition is currently operating, effective self-Regulation is contingent on ‘self-Evaluating’ such as ‘comparing and contrasting’, which are vital to see the similarities and differences between where one is cognitively and where one wants to be. Regulation of cognition that is beyond conscious control happens continuously (Stiles, 2008), but intentional self-regulation of cognition needs an impulse and the impulse is provided by self-Evaluation. It is self-Evaluation that sees the contrast with ideated cognition and so provides the impetus to self-Regulate cognition in that new direction. When self-Regulation makes an intentional cognitive shift, self-Monitoring and self-Evaluation must continue to function, otherwise it is not possible to know where one has ended up cognitively and if further movement or rather maintenance is required.

The consolidation aspect of self-regulation is as crucial as the change aspect and involves a process whereby self-evaluation suggests that present cognition is ideal or fit-for-purpose, resulting in an active decision to uphold cognition, to stay at present, optimum levels (B. Zimmerman & Schunk, 2011). Such an intentional stabilisation of cognition, for example maintaining cognitive focus (Efklides, 2008; Samsonovich et al., 2008), may be a factor in maintaining immersion in learning, or ‘flow’, by the learner’s intentional removal or reduction in factors that could otherwise distract from the present fluid cognition (Landhäußer & Keller, 2012). Cognitive flow is by nature primarily unaware of all, but its focus subject, not typically metacognitive; however, maintaining flow may sometimes require subtle self-monitoring of shifts away from the flow as well as self-regulation back into the flow.

Self-Monitoring and self-Evaluating cognition are precursors to, and co-conditions of, self-Regulating because intentional changes in thinking processes cannot be made effectively without knowing what is already happening cognitively. Self-regulation without self-evaluation during a cognitive task is like running in the dark, unaware and potentially leading to a fall.

As noted earlier, planning of cognition is not a separate component in the AMERT framework but is rather subsumed in self-Regulation of cognition (Omarchevska et al., 2021). Planning cognition is complex, requiring co-occurring self-Monitoring and self-Evaluating processes (O’Leary & Sloutsky, 2019) and is necessary for cognitive Transfer to occur.

Metacognition cannot be treated as a panacea for learning. For example, self-Regulation may lead to cognitive overload, impairing students in the task at hand, in part because they are allocating finite cognitive resources to think about their thinking (Sweller, 2011). Self-regulation requires self-evaluation to determine when to discontinue metacognition and focus only on the task at hand rather than one’s own thinking about it.

Assertion 3:

Self-Evaluating cognition is a pre- and co-condition for self-Regulating cognition

Self-Transfer cognition. Self-Transfer of cognition, the fifth level of AMERT, is the use of cognitive strategies that are developed in one context and employed intentionally as strategies in a different context (Greene et al., 2021; Heggen, 2008; K. Smith et al., 2007; Tuomi-Gröhn & Engeström, 2003). Self-Transfer of cognitive skills to new contexts is not easy, because knowledge is acquired or developed in a language-and-culture-rich context, as are the metalanguages, or labels, used in metacognition (Cellar & Barrett, 1987). As cognitive skills like ‘analysis’ shift in meaning and nuance from context to context, the shift makes self-Transfer of cognition complex and, for many students, it requires facilitation by others.

The intentional Transfer of cognitive skills from one context to a less familiar context is a specific form of self-Regulation of cognition. For self-Transfer of cognition to occur, an individual must recognise prior cognitive knowledge and skills and how these need to be reconstructed or re-contextualised (Garraway et al., 2011) to meet the needs of the new context, by identifying what cognition is required and reflecting on any similarities to past experiences (Garraway et al., 2011). The Self-Transfer of cognition in this current paper is fully intentional and is an indicator of high-level metacognition in learning environments and situations that are different from where the cognitive skills were learned. The AMERT hierarchy suggests that if intentional Transfer of cognition is happening effectively, self-Awareness, self-Monitoring (Perkins & Salomon, 1992), Self-Evaluating and self-Regulating (Barzilai & Chinn, 2018; McKeachie, 1987) are taking place simultaneously as pre-conditions and co-conditions.

As noted, the concept of planning cognition is also treated in AMERT as a form of self-Regulation of cognition (Barzilai & Chinn, 2018), in keeping with a literature review on metacognitive thinking (Lai, 2011). Planning cognition for tasks that are relatively close to other experiences is called near Transfer of cognition, and planning for unfamiliar tasks is far Transfer of cognition (McKeachie, 1987). AMERT’s focus on process represents planning as part of the self-Regulating process which relies on the lower levels of the framework to be effectively activated.

Assertion 4:

Self-Transferring cognition is a specific form of self-Regulating cognition

The hierarchy of the AMERT framework

In the AMERT hierarchy, higher levels occur on the basis of lower levels and depend on how effectively these lower metacognitive processes are functioning and remain functioning. The AMERT hierarchy of processes means that, for example, self-Regulating cognition is ineffective without self-Evaluating and self-Monitoring cognition, which in turn are ineffective without learners being self-Aware of cognition. This hierarchy does not mean a sequence of steps up, such as first being self-Aware, then self-Monitoring and then self-Evaluating. Rather, for a higher level of AMERT to operate effectively, it must do so on the basis that the lower levels are also operating simultaneously.

Table 1 depicts the hierarchical nature of the AMERT model in which the lower levels, starting from self-Awareness of cognition, are pre-conditions for those above all the way to self-Regulation and its highest form, self-Transfer of cognition. Weak self-Awareness of cognition then, such as that due to the use of poor cognitive labels, drastically reduces the effectiveness of higher levels of metacognition. Moreover, students may intentionally change their cognition without self-Monitoring and self-Evaluating it, but that regulation would be uninformed, random, lack a feedback loop, and lack discernment into the current cognitive state and the effects of any changes. Change always shifts, whereas self-Regulation of cognition may involve maintenance of a steady state, enabled by ongoing self-Evaluation and the lower levels of AMERT.

The AMERT levels, exampled through the cognition associated with the analysis, are as follows: Self-Awareness is knowing different forms of cognition, e.g. ‘I know and understand the thinking involved in analysis’. Self-Monitoring is the comprehension of one’s current cognitive status: e.g. ‘I am analysing’. Self-Evaluation is to see if cognition is sufficient or should be different e.g. ‘I need to be more critical in my analysis’. Self-Regulation is to intentionally cognitively shift or to actively consolidate the type, quality or intensity of cognition, e.g. ‘I am going to critique more from a perspective that challenges the ideas in my current analysis’. Self-Transfer is to intentionally move cognition to a new context or adapt cognition to fit a new context, e.g. ‘I will adapt and apply to my mathematics investigation the analytical thinking that I used in my science investigation’.

To use an analogy with heart rate to explain the AMERT hierarchy, self-Awareness is like distinguishing the sound or feeling of one’s own heart, understanding its significance and labelling it a ‘heartbeat’; self-Monitoring is actively taking one’s pulse; self-Evaluating is like comparing or contrasting the monitored rate to what it should be, depending on rest, exercise or stress levels; self-Regulating of cognition is like actively taking steps to increase, reduce or maintain heart-rate; self-Transfer is like applying this regulating capacity to other contexts, such as a personal trainer might with a client. An ineffective label that would lead to poor self-Awareness, in this analogy, is exampled by the term ‘liverbeat’, which would render all the higher levels misleading.

Assertion 5:

There is a hierarchical relationship between metacognitive elements, from the foundational self-Aware of cognition through to self-Monitoring of cognition, self-Evaluation of cognition, self-Regulation of cognition and self-Transfer of cognition.

An example applied to AMERT

Two quotes from a study that found evidence of student metacognition (J. Willison & Buisman-Pijlman, 2016) but did not have available an appropriate metacognitive framework to probe this in detail, provide an example of the use of AMERT to unpack metacognitive statements:

… you can see all the levels; you can see where you are. You compare yourself to the data. It takes a skill to be honest to yourself; that’s the first skill. When you look at the different levels, you can see where you are fitting and then you look at the levels ahead, at what are your areas of improvement so you can improve yourself (J. Willison & Buisman-Pijlman, 2016, p. 71).

• Self-Aware: … you can see all the levels

• Self-Monitor: … you can see where you are

• Self-Evaluate: You compare yourself to the data. It takes a skill to be honest to yourself. … then you look at the levels ahead, at what are your areas of improvement.

• Self-Regulate: … so you can improve yourself

So you can improve yourself has a strong planning sense and the potential for transfer to new contexts.

Self-Transfer is explicitly noted by another student in the same study: ‘… because they have been consistently applying this structure to all of our assignments, we have come to think that way for science’ (J. Willison & Buisman-Pijlman, 2016, p. 75). This is a transfer from a range of specific assessment tasks to a scientific way of thinking more globally.

In summary of this section, there are good grounds in the literature for the synthesis of a hierarchical framework for metacognition, with self-Awareness of cognition being foundational to the metacognitive skills of self-Monitoring, self-Evaluating, self-Regulating and self-Transferring cognition. Part B of this paper outlines the context and methodology of the empirical study designed to test the viability of the hierarchical AMERT framework, and then presents the results, discussions and conclusions.

Part B: preliminary test of AMERT’s viability

AMERT is an interpretation and synthesis of the findings on metacognition over the last four decades; however, how effectively does this hierarchical framework represent metacognition? Using López-Campos et al. (2008) framing, three questions need to be answered before the AMERT framework should be used broadly:

1. Does the AMERT framework provide a viable interpretation of accounts where metacognition is evident? (Framework is functional and provides insight.)

2. Does the AMERT framework provide a valid understanding of metacognition? (Framework has construct validity.)

3. Do AMERT-based instruments provide reliable scores of metacognition within and between studies? (Framework informs instruments that demonstrate internal and external reliability.)

In terms of viability, is the framework functional, easy to interpret and readily able to help researchers distinguish different levels? Testing of viability may be done with rich qualitative data, to see if the different levels correspond to actual experiences. Researchers from outside the team that formulated AMERT are also needed to test viability before testing of validity is warranted. Testing the construct validity requires using AMERT a priori as a framework to generate data. However, it would be premature to devise an a priori empirical quantitative tests given the calls in the literature that prioritize deep qualitative understanding and the need to devise better theorisations of metacognition, such as AMERT, before further data is generated, as mentioned earlier (Adadan, 2020; González et al., 2017). Therefore, the general question that addresses the secondary aim of this study concerns viability only:

Does the AMERT hierarchy provide a viable understanding of student metacognition that is evidenced in interviews?

Data comprising interviews with students who have rich learning experiences can provide some of that detail and nuance (Adadan, 2020).

Study context

The empirical data used for a preliminary test of AMERT’s viability comes from interviews with students in the fourth, optional and research-focused year of a Bachelor of Animal Science, called ‘Honours Year’ in Australia. The degree was primarily taught on the regional campus of the Australian Research University. At the time of the study, the programme used the Research Skill Development (RSD: J. Willison & O’Regan, 2007: see Appendix 1) framework to conceptually frame teaching, learning, and assessment in pertinent courses across the degree (J. W. Willison et al., 2014).

Well before the study began, one course in the first year began using the RSD for teaching and learning and to frame assessment tasks, followed subsequently by two second-year courses. In these courses, the RSD facets were used repeatedly to facilitate the development and assessment of the skills associated with research and so were used, in effect, like cognitive labels. The RSD articulates six facets of research thinking, which are represented in this paper as verb-pairs linked by an ampersand: embark & clarify, find & generate, evaluate & reflect, organise & manage, analyse & synthesise and communicate & apply (see J. Willison, 2018 for details). The use of the RSD to enable student research is in keeping with recent studies of inquiry skills, yet ‘… extends these by considering the metacognitive aspects … more fully and explicitly’ (Barzilai & Chinn, 2018, p. 354).

This repeated exposure of RSD use to frame assessments in different course contexts and different ways of being operationalised yielded an opportunity for facets to be internalised by students and become cognitive labels. Repeated exposure increases the chance that students would deeply understand them as cognitive labels and so would determine, heighten or clash with their existing metacognitive knowledge. Students were not exposed to RSD rubrics in the third or fourth years of their degree. The interviews, conducted in the fourth year, one-and-a-half years after the last exposure to the RSD, provide a long-term retrospective by the students and provide some sense of the sustained influence of cognitive labelling on metacognition. The published interview data used above as an example of AMERT analysis was from students who had a similar experience across programme use of the RSD facets in a medical science degree and yielded rich metacognitive elements (J. Willison & Buisman-Pijlman, 2016). Animal Science students were therefore more likely than a random cohort to have some sense of metacognition and provide evidence of it, and so the interview data were a useful data set to test the viability of the AMERT framework.

This data set was useful for testing AMERT because it provides a best-case scenario: selecting epitomising statements from students who chose to be interviewed, from a cohort who chose to do the fourth research-intensive year and from a programme that uses RSD facets as cognitive labels. If AMERT is unable to disentangle metacognitive elements from the rich data of Animal Science students, then its viability would be immediately questioned. However, if AMERT yields valuable insights into student metacognition evidenced, then this article provides the Science Education community with a tangible exemplar of analysis within the framework and suggests further testing of viability should be carried out.

A total of 41 students were invited through a third party by email to participate in the study, in keeping with the ethics-approved protocol, which required no direct initial contact. Nine students (22%) agreed to be interviewed. The inclusion criterion was students enrolled in their research-oriented fourth and final year of the Bachelor of Animal Science. Seven participants were enrolled in a full-time honours degree, one part-time, while a full-time zookeeper and one participant had just completed the requirements of the degree but were still enrolled. Seven participants were female and two were male.

Research question

Set in the research context, the general question about AMERT’s viability above was adapted to formulate the research question for Part B:

Does the AMERT hierarchy provide a viable understanding of student metacognition for each facet of the RSD as evidenced in interviews with students enrolled in the 4^th year of a Bachelor of Animal Science?

Methodology

Data generation

The current study uses a semi-structured interview strategy (Wengraf, 2001; Whiting, 2008) to generate rich and descriptive data with thick descriptions and that are particularly suited to provide a deeper understanding of metacognition (Adadan, 2020; González et al., 2017). The interviewer used the RSD facets as part of the interview prompts and showed students assessment rubrics based on these facets from the first and second years. (Hazel, 2011a, 2011b). The purpose of these prompts was to clarify and make concrete the questions being asked because these facets had not been revisited for the 1.5 years prior to the interviews.

In the semi-structured interview protocol (see Appendix 2), students were not asked directly to articulate their understanding of metacognition, but rather recount ‘particular happenings’ that reflect experiences of development and use of their research skills, where student ‘… choice of examples reveals essential features about their own ideas and experiences of learning’ (Soini, 2012, p. 847). The process of recounting specific situations provides a more authentic reporting mechanism than explicit self-commentary on the concept of metacognition, because participants in interviews are prone to provide what they perceive is what the interviewer wants to hear (ref)

The semi-structured interviews were conducted face to face over a 40–50-min period and were audio recorded on a digital device and sent to a third-party transcription service. Interviews were conducted by a research officer who was experienced in engaging students and eliciting deep understanding in interview situations. The 15 interview questions (Appendix 2) related to the development and use of research skills.

Data analysis

To address the research questions about viability, evidence at each level of the AMERT framework and relationships between them were sought in the interview data. The six cognitive labels of the RSD were used in the Animal Science degree and prompted in interviews, and so the RSD facets were used to organise the analysis of the transcripts. The analysis adapted the protocol developed by J. Willison et al. (2020), which was used to analyse interview data from the perspective of the affective domain and effective for dealing with rich and nuanced data.

• Research team members (Authors 2–7) were allocated one RSD facet each as a focus

• In order to provide time for deep and focused reading, one interview per week was coded by each research team member with reference to their allocated facet. Each week, an additional transcript was given to the team.

• The majority of statements for each facet were not metacognitive in nature, but were rather about what students did, and were excluded from categorisation.

• Statements perceived to have indicators of metacognition for each facet were coded by each team member with reference to the levels of AMERT.

• Together this process provided weekly two-tiered categorisation:

  • • Tier one was the text that pertained only to each researcher’s allocated RSD facet

  • • Tier two was the text from tier one categorised according to AMERT levels.

This sequence was chosen because, as noted earlier, each facet of the RSD stands as a form of cognition that students could be metacognitive about. It is a more transparent and reproducible process to first identify cognition in keeping with the facets and then indicators of metacognition, which may be co-located. The protocol was chosen to analyse the complex interview data because it was employed with an equivalent number of research analysts (J. Willison et al., 2020) in this current study, looking at one facet each and was determined to be a fitting way to analyse equivalent data.

• At weekly research team meetings led by the first author, each person presented their two tiers of coding for their facet, and disagreements around coding were negotiated until consensus was reached, as per Whiting (2008).

• The most indicative, or epitomising, interview excerpts were chosen to demonstrate the viability of the framework because they:

  • • were communicative for differentiating between the different levels of AMERT,

  • • pertained to several levels of AMERT, and this helped highlight the hierarchical relationships.

Following the above, the data from interviews are presented in a sequence of RSD facets with evidence of the AMERT levels within each facet. This provides a rich sense of contextualised metacognition that pertains to specific forms of cognition instead of generalised metacognition. Metacognition makes little sense without understanding the cognition that it is aware of, monitors, evaluates, regulates and transfers.

Statements that epitomised different levels of metacognition were sought from the interview data because these epitomising statements provided an opportunity for reducing ambiguity, increasing clarity and providing a sense of viability for AMERT. The data selected, evidencing metacognition at each level of AMERT across the nine transcripts, provide a sense of the ‘possibilities’ (Peräkylä, 1997; Talja, 1999) of metacognitive experience as framed by the hierarchical framework. Therefore, the following results elucidate AMERT but do not provide a sense of how effectively student metacognition was elicited in the programme. As stated above, the purpose of this analysis is to determine if AMERT is a viable framework for understanding metacognition.

Results

Statements were found from the interviews for each RSD facet at the self-Aware, self-Monitor, self-Evaluate and self-Regulate levels of metacognition. However, the statements that were at the level of self-Transfer were more generalised in nature, did not clearly fit with the RSD facets but did fit with simple demarcation in the literature as ‘near’ or ‘far’ transfer (McKeachie, 1987).

As indicated in the methodology, the analytical process first identified all student interview statements about cognition, in this case with reference to the RSD facets, before identifying which of those statements evidenced metacognition. Therefore, the resulting structure is organised facet by facet, with evidence of AMERT levels in each.

Embark & clarify

There are numerous ways to embark on sophisticated learning, but for the Animal Science degree it included formulating research questions, hypotheses, project aims or synonymous aspects that provide a sense of purpose or direction:

I think you’re aware of it, but as I say, I think in first year you don’t really understand what it means, like how to embark on inquiry and all of that sort of stuff; you’re not really sure where you’re going with it. But yes, it stayed the same. By third year, you know much more what you need to do. It becomes a lot clearer as you go on, yes. In first year you’re aware of it, but I just don’t think you understand it as well as you do by third year (Student 7).

Student 7 realised that, initially, she did not understand what was expected in assignments and that she did not have the full capacity to embark on inquiry or clarify the required direction. The statement implies that the cognitive label of embark & clarify was, for her, just that a label, which did not raise initially a foundational self-awareness of her cognition.

Self-Awareness of cognition is demonstrated in the statement ‘you know much more what you need to do … I just don’t think you understand it as well as you do by third year’. The growth from first year to the third year in metacognitive knowledge epitomises the shift from a teacher-given cognitive label to self-Awareness of skills of how she may embark & clarify.

Self-Monitoring of cognition is shown when the student thought that her cognitive capacity to embark stayed the same for some time, then as the student kept self-Monitoring this cognitive skill, by her third year she found that the process of embarking becomes a lot clearer as you go on. Self-Monitoring was the metacognitive process that recognised the shift over time but did not drive, direct, or guarantee it. To say it becomes a lot clearer, rather than say, I made sure that I clarified, provides a passive sense that this just happened to the student over time. In terms of hierarchy, when self-Awareness of cognition around embarking processes, such as posing hypotheses or framing questions, emerged over time, then self-monitored realisations, such as it stayed the same and a lot clearer, became possible. The quote implies a dynamic hierarchy, where there is a sense of a mutually enhancing cycle, which is not only based on self-Awareness as a pre-condition and co-condition but also reinforces the self-awareness of cognition.

In terms of cognitive self-Awareness and self-Monitoring being foundational to higher levels, one student stated:

[Without the RSD labels] … I’d say that maybe you wouldn’t really know where you’re meant to be headed towards in terms of development. You might end up just going around in circles and not pushing yourself further to go to a higher level. (Student 6).

Self-Evaluating cognition is shown when Student 6 contrasts going around in circles to a desirable cognitive higher level … where you’re meant to be. The student contrasts being conceptually stuck in a holding pattern with something higher, which implies the need to move to the next conceptual level, which may be more rigorous or richer experience, requiring broader or deeper knowledge. For Student 6, this self-Evaluation of cognition provides the impetus for change.

Self-Regulating the complexities of embark & clarify are demonstrated by the movement of intentionally pushing yourself to actually go to a higher level. While self-monitoring provides a sense of what is going on cognitively and self-evaluation provides a contrast with how things could be, self-regulation makes the shift happen. Self-regulation of cognition has a built-in planning aspect because it is always an intentional shift or an intentional consolidation. The student’s self-Regulation could not lift her higher without knowing just how high she was already, evidencing that planning cognition, as a form of self-Regulation of cognition, for Student 6 was dependent on self-Monitoring and self-Evaluating cognition.

Find & generate

This RSD facet involves generating one’s own data or, in this case, a process for finding others’ information:

I already had the knowledge of where to go and look for information and how to sort of write scientifically, to a degree, yes, whereas everyone else was coming from school and didn’t have that background. So I was ahead to start with, yes. (Student 5)

Self-Awareness of find & generate is shown by Student 5’s knowledge of information search skills.

Self-Monitoring of cognition is demonstrated by her statement that she had that knowledge.

Self-Evaluation of cognition by Student 5 is shown by her perceptions of its relative lack in other students being ahead to start with implies a comparative look, not of her own need to improve but perhaps the opposite, a kind of self-Evaluation which confirmed a sufficient capacity. This self-Evaluation could prompt underperformance if it was not well tuned to disciplinary standards.

Self-Regulating cognition is shown by Student 2, who said of data generation:

It’s basically making sure that the method, you’ve actually got to correct for finding out the data that you need to answer the hypotheses (Student 2)

Student 2 recognised that utilising the most appropriate method is imperative for generating data that tests a hypothesis. If self-Evaluation of cognition by making sure about the method, then shows a deficit in the methods, Student 2 said this drove a Self-regulation of cognition where the student could then take active control and intentionally correct the data generation method so as to obtain appropriate data that does address the hypothesis.

Evaluate & reflect

Like other RSD facets, evaluate & reflect are skills, which are cognitive in nature until the skills themselves become subject to inspection and so enter the metacognitive realm, as one student realised:

… not just this year but previous years, what they try and get you to do is to critically assess the different literature, not just kind of recite it but rather point out any potential arguments. I think where there are arguments in the knowledge, I think that’s applicable. (Student 6)

Self-Awareness of evaluate & reflect is demonstrated by the student unpacking the notion that you critically assess the different literature, not just kind of recite it.

Self-Monitoring is shown by Student 6’s reflection that assessing critically was in development not just this year but previous years.

Student 9 grappled with the status of data she generated on a topic that she had opinions on:

… one of my biggest things is I need to step back and not pick a side, and not be biased in it. Because of the topic that I chose, I already have preconceived ideas when I’m actually collecting data, it’s kind of changed my mind a bit, as well as it’s probably better to stay at the back a bit (Student 9).

Self-evaluating her own cognitive process of evaluation, is a wheels-within-wheels situation: The cognitive process is shown by the student evaluating her data collection; the metacognitive process is shown when she self-Evaluates the cognitive process as shown by the realisation that I already have preconceived ideas, and the alarm bells go off as biased.

Self-Regulating of cognition is triggered by the student’s self-evaluation, which realises the risk of bias and that this will lead to poor evaluation of the data. Her impulse then is to self-Regulate, to step back, which is an intentional regulatory shift to gaining a more distant, broader perspective and a more objective one. Student 9 articulates that self-Regulation to produce cognitive movement is needed when she is evaluating, because self-Evaluation of her current stance shows a position that is too close, with preconceived ideas and bias. Students can recognise the risk of being too close and choose not to be dominated by bias. She actively self-Regulates to reduce bias by conceptually moving further away.

Effective self-regulation also necessitates self-Evaluating the new cognition resulting from any cognitive movement to determine if the new cognition is appropriate and so maintains or potentially enacts further change to match the chosen position. Stay implies self-Monitoring and self-Evaluating to ensure ‘no movement’ once the student’s evaluation of the topic is self-Regulated to the correct, removed, location. Self-Regulation for Student 9 is not always about change and movement, then, but also may involve consolidation, requiring self-monitoring to ensure no movement from the perceived optimum and position at the back.

Organize & manage

Focusing on managing their research project, Student 5 stated:

… you still know how to look up what needs to be looked up, or fix a problem, or work through a problem with yourself, and seek help when you need it. (Student 5)

Self-Awareness of cognition is demonstrated by the knowledge of how to self-manage a variety of processes.

Self-Monitoring of cognition is shown by the ongoing sense that there are times when you know how, and times when you do not, and ongoing self-monitoring is the way to find out,

Self-Evaluation of cognition is shown by seeking help when you need it, which is very different from seeking help by default, or merely knowing how to seek help. This self-evaluation of cognition is sensitive to the need for change, appreciating the gap between where one is and where one needs to be.

Student 6 related the benefits of managing a research assignment by checking the assignment relative to the marking rubric provided:

I tend to sort of towards the end of an assignment go back to a rubric, something like this, and I read through them and as I’m reading them I think, yes, I’ve already done that, I’ve done that, I’ve done that … (Student 6)

Self-Regulation of management processes is primed by Student 6’s self-Evaluated need to check, which makes this student actively complete assignments by going back to a rubric to self-organise and check the assignment. Then, in a second iteration of self-Evaluation, the student determined if each facet is completed at a satisfactory level and found I think, yes. The intentional and student-instigated use of the rubric as an external aid for metacognition is an example of self-Evaluation and self-Regulation of metacognition that is built on self-Awareness fostered by the RSD cognitive labels. Rubric provision guarantees nothing, not even self-Awareness, but repeated use, as evident by Student 6 may lead to self-Awareness, self-Monitoring, self-Evaluating and self-Regulation of cognition.

Analyse & synthesise

Student 4 reflected on the need to analyse when writing:

… scientific writing was probably where I needed to improve the most… I’ve always got that in my head to go back and critically analyse, because it’s easy just to quote people and to find information without actually thinking about it. (Student 4)

Self-Awareness of cognition is shown by her constant awareness about the need to critically analyse which is always in my head. This front-of-mind awareness is possibly because she is aware that while scientific writing is multifaceted, a hallmark of the scientific writing process is the sophisticated analysis of data and synthesis of existing and emerging ideas.

Self-Monitoring of cognition is shown by the negative example, her realisation that, despite her own high level of self-Awareness, its easy to proceed without actually thinking about it and not to monitor these cognitive skills to critically analyse. When she does self-Monitor the current status of her analytical skills, it provides information for self-Evaluation.

Self-Evaluation of cognition by Student 4 contrasted where she was with where she needed to be in her scientific writing, to determine where she needed to improve the most, that is she would focus her cognitive resources on analytical processes.

Self-Regulation of cognition is impelled by the self-Evaluation of her greatest needs, leading to a focus of her limited cognitive resources on self-Regulating critical analysis in her writing to provide the required improvement. Her self-Regulation is a move from a more tokenistic approach to writing, where she would merely quote people and find information that provides some minimum appearance of success, to critical analysis in the justification for, and well-synthesised use of these sources in her scientific writing.

Student One reflected on self-Awareness of analytical thinking that was facilitated through an assignment, then provides an idea that is insightful for the role and processes of metacognition, when she notes how analytical thinking could operate below the level of consciousness:

… it did give me the analytical skills that I needed to do that assignment … though [now] I may not consciously think, okay, I need to find this knowledge, I need to read it, I need to analyse this knowledge, and find the gaps in the knowledge to develop an assignment … it will be something that I believe will be more unconscious (Student 1).

For Student 1, cognitive processes made explicit and clear could become so internalised that the self-Monitoring of them became, and intentionally will be, below the level of detection, implicit and unconscious. Here she alludes to cognition, such as critical analysis, rather than metacognition, such as self-monitoring her analytical skills, becoming the focus. In that non-metacognitive mode, she would be using her limited internal cognitive resources in the actual analytical writing process itself, not on how to self-Monitor, self-Evaluate or self-Regulate her analytical thinking. Student 1 still maintains her metacognitive knowledge, but her awareness is focused on writing processes. During that focus, however, her self-Awareness could rise from more unconscious to consciousness, and trigger the need to monitor where her cognition is at.

Student 1 may typically not consciously think about her cognition, but a variety of triggers may lead her to think more consciously about her thinking itself, such as the assignment that facilitated her awareness of analytical skills, or when there is a problem, cognitive dissonance or other prompts to be more conscious of her cognition.

Communicate & apply ethically

The ethical dimensions of communication hit home to one student:

… there’s definitely a lot of ethical and social issues, and definitely cultural, because everyone’s different on how they would perceive that. So you would have to be careful in how you would describe your opinion on whether to keep caged or not, because you could definitely offend a lot of people and make them think the wrong thing. (Student 3)

Self-Awareness of cognition is noted by Student 3 in the ethical and social issues connected to communication in which you would have to be careful in how you would describe your opinion.

Self-Monitoring of communicating, comprises not just to be careful about his opinion per se, but the process-oriented how you would describe your opinion. In order to take care not to offend, for example in word and picture choice or the structure of argument, this required a careful self-Monitoring of communication processes.

In terms of self-Evaluating and self-Regulating, Student 4 said:

I like to do better than previous assignments, so if I’d seen them as linked, I would have paid more attention to the feedback that I would have gotten in order to apply it to the next assignment to try and push myself up to the higher levels. (Student 4)

Self-evaluation of cognition is demonstrated when Student 4 contrasted her current performance with her ideal performance, saying I like to do better.

Self-Regulation of cognition is reflected in the interview when she stated that she did not realise there was a connection between the earlier assignments in terms of the six RSD facets. Had she realised, in hindsight, she anticipated that she would have used the feedback to self-regulate to higher levels. When applying to the next assignment, the student flags self-Regulation that applies to different but similar contexts, an example of self-Transfer of cognition to close contexts.

Self-Transfer of cognition

Evidence for the intentional Transfer of cognition, unlike self-Aware, self-Monitor, self-Evaluate and self-Regulate cognition, was rarely explicit for each RSD facet, but was more holistic in the interviews. Student transcripts evidenced near and far transfer (McKeachie, 1987) of cognitive skills. Near transfer here pertained to transfer from the first three years of their Animal Science degrees to the fourth, research-intensive ‘Honours’ programme, and far transfer involved transferring knowledge from a university setting to a ‘real world’ situation. In both instances, the individual adapts and adjusts their knowledge and skills to a task that differs from where the knowledge originally occurred (Ford et al., 1998) where nearness or farness is determined by the extent of the difference.

Enabling self-transfer of cognition from university study to different university study

Student 5 noted the transferability of skills used in research design:

… designing your own research project to some degree and following that through from start to finish would be really beneficial, because then people can realize, oh, it’s not just about working with mice in a lab, or working with cell cultures or that sort of thing (Student 5).

The statement shows a realisation that the design and implementation of an experiment with mice or cultures should not be the crux of the learning, but rather the cognitive processes used to manage a major project from beginning to end and then implementing these skills in other settings. Student 1 had a metacognitive realisation that such cognitive development was happening throughout the university programme:

It was everything all my assignments, all my feedback from my assignments and then the research methodology courses that we undertook, so all of my skills developed from my undergrad. (Student 1)

Student 1 was able to recognise the cognitive skills gained from every programme component. That student realised the metacognitive transfer enabled by the different components and understood that assignments are not just another assignment, but a metacognitively exciting opportunity of further developing and self-Regulating cognitive skills. When students see cognitive skill development everywhere, it may be because they are self-Aware of, understand and can use shared cognitive labels, self-Monitor and at times self-Evaluate and self-Regulate these thinking skills. The more contexts in which cognitive skill development occurs, the more blatantly a student may see it everywhere. The metacognition then would become self-perpetuating, no longer needing educators to point out rubrics or specifics of cognition. Self-regulation of cognition enacted across multiple-contexts makes learning how to self-Transfer cognition much more likely because students can ‘generalise their learning to the whole’ (Adcroft, 2011) and apply their cognition in new, less familiar contexts:

… we had a lot of research-based assignments where we had to go out and design it ourselves, get the results ourselves and then write it up, and that is what I found most beneficial. I could then apply that to stuff that I did in Animal Science degree … you had to go out and design the project, what you wanted to find out, how you’d go about finding that out. You’d go out every week and collect your results, and then you would sort of work them out yourself … (Student 5).

Student 5 realised skills developed were transferable within university study, leading to higher levels of learning autonomy in newer contexts, where the complexities of result analysis could be regulated by yourself.

Enabling self-transfer of cognition from university study to the world of work

Only one student explicitly mentioned that the learning at university could be applied in the workforce once they graduate.

The horse assignment was good because it was a really fixed scenario that we had to write about, how to try and correct it, so it was something that they were trying to get us to think more outside the university (Student 7).

Student 7 saw the benefit of utilising fixed scenarios within the university environment, learning and self-adjusting their cognition to situations that they would subsequently encounter in the workforce outside the university. The Bachelor of Animal Science degree included cognition relating to potential actions that should be taken when encountering unpredictable animals, owners, and their behaviours. This student, reflecting on a horse assignment showed an appreciation of the holistic and projected far Transfer of cognition.

In summary, the AMERT framework proved to be a useful lens to examine student metacognition evident in interview data. For each and every RSD facet, there was evidence of students being self-Aware of cognition, self-Monitoring cognition, self-Evaluating cognition and self-Regulating cognition. However, student self-Transfer cognition was rarer and was not evidenced facet by facet. There was, however, evidence of near and far Transfer of cognition. The hypothesised hierarchy of AMERT elements was not disconfirmed, however the hierarchy is implicit, rather than explicit, in the interview data.

Discussion

The analysis of the interviews with nine students provides a tentative sense of the viability of the AMERT hierarchy to provide insight into metacognition. The epitomising statements do not show how commonly students were thinking metacognitively, just that, across the nine interviews, there were rich instances of multi-faceted metacognition at each level of AMERT. These instances of metacognition do not generalise to each cohort, let alone to metacognition broadly conceived. Moreover, metacognition evidenced during the interview does not imply that students were metacognitive during their laboratory work or other research. However, the experiences of these nine students resonate with and bring to life the AMERT framework, providing preliminary evidence that the hierarchical framework is viable for providing understanding of, and organising data that is rich in, metacognitive statements.

Such evidence of viability is a small but promising step towards an empirically validated hierarchy of metacognition for teaching and research. If further evidence of viability emerges over subsequent qualitative studies in numerous contexts, AMERT could then be tested as a hypothesis for metacognition in quantitative studies, which consider its capacity to inform valid and reliable instruments.

The data in this study hints at a dynamic hierarchy, where repeated exposure to cognitive labels led to a developing self-Awareness of cognition and onto higher levels of AMERT. If each subsequent level reinforces the prior levels, the basis for metacognition becomes broader and more stable, perhaps further enabling higher levels. This could imply that once students have operated on a higher level of AMERT that the higher levels are more likely to emerge without needing overt facilitation. The quality of the labels and exemplars that teachers use for student metacognitive knowledge for self-Awareness may determine the extent of success in facilitating student self-Monitoring, self-Evaluating, self-Regulating and self-Transferring cognition to other contexts.

This study used the RSD facets as cognitive labels, and they warrant further research into their use and function for metacognition in Science Education. These facets, when used elsewhere, have shown potential to assist in cognitive self-Transfer (Ain et al., 2019), especially in transdisciplinary STEM (J. Willison, 2020), where otherwise the cognitive transfer process has proven to be awkward for students and teachers. This current study raises the tantalising possibility that if cognitive labels resonate with teachers and learners and are used on multiple occasions as thinking routines (Ritchhart & Perkins, 2008) for students, the resulting self-Awareness may actualise as self-Monitoring, self-Evaluation, self-Regulation and, ultimately, Self-Transfer of cognition. If that possibility were affirmed by multiple studies, teachers could focus on meaningful and repeated use of appropriate cognitive labels and allow higher levels of metacognition to spontaneously emerge. Such a process treats metacognition as enabled by normal human neuro-architecture and may provide a more realistic way to facilitate higher levels of student metacognition by regular classroom teachers than overt attempts to elicit higher levels of metacognition such as self-Monitoring or self-Regulation.

The AMERT hierarchy may also go some way to explain aspects that are still debated in the literature. As noted earlier, confusion about metacognition has been attributed to Flavel’s original characterisation of a single term for both knowledge about cognition and regulation of cognition (Georghiades, 2004). However, the confusion may not be in the term, but in the way the relationship between metacognitive knowledge and metacognitive skills such as self-Regulation has been articulated, or rather not articulated. As specified earlier, the separation of metacognitive knowledge and metacognitive skills in some models led to a lack of clarity about the relationship between them. If the AMERT hierarchy ultimately proves to be viable and can guide the production of instruments that are reliable, the clarity that the framework provides will help researchers and teachers to engage with the term metacognition in ways that improve students’ nascent metacognitive capacities.

Further research

Future research can test the hierarchy as a hypothesis, looking for evidence that higher levels of AMERT, such as self-regulation, are accompanied by evidence of levels that are lower in the hierarchy, such as self-Monitoring and self-Evaluating. Even though a majority of researchers now appreciate a ‘causal’ relationship between at least some metacognitive elements (Rhodes, 2019) not all recent research agrees (O’Leary & Sloutsky, 2019). An initial question, following on from this study, is:

• In subsequent research into metacognition, to what extent is there evidence of a hierarchical relationship between the levels of the AMERT framework?

That research should begin in a qualitative manner, including interview and observation studies, to determine if the hypothesis of the AMERT hierarchy is viable. If it turns out to be viable, then quantitative studies to test the hierarchy hypothesis would be warranted, asking:

• To what extent do instruments based on the AMERT construct return internally reliable data?

If the AMERT framework, or an improved version of it, were validated and began to return measures of reliability in instruments that were based on it, then it could be used to determine broad and far-reaching baseline and intervention questions, such as:

• To what level, frequency and extent are teachers, using a range of existing practices, developing student metacognition?

• What is the effect of a specific innovative intervention on student metacognition?

The affective aspects of AMERT were not considered in this paper, however motivation and dispositional aspects impact metacognitive strategy use and performance (Efklides, 2011).

• How do motivational and dispositional aspects of metacognition impinge on the AMERT framework?

Some of the clearest metacognitive statements at the level of self-Evaluation and self-Regulation used location-based metaphors for cognition, such as go round in circles and step back. It may be that explanations of higher levels of AMERT are difficult to conceptualise or explain literally. If that is the case, the use of literal survey questions will not effectively capture the nuances of metacognition, especially at higher levels. Moreover, metaphorical statements cannot be used effectively as survey items, because people’s interpretation of metaphors is wide and diverse (J. W. Willison & Taylor, 2006).

• What are the optimum strategies for measuring or determining each AMERT level?

Another line of inquiry concerns explicit instruction about AMERT, to determine to what extent this lifts student metacognitive capacity, if at all.

• To what extent do explicit strategies to facilitate AMERT levels enable metacognitive skill development?

The use of this paper’s analysis protocol by other research teams for primary or secondary analysis of rich interview data is one way to test the viability of AMERT efficiently. If viable, AMERT as a conceptual framework could inform systematic literature reviews of published studies that have represented rich qualitative data, such as direct quotes. Further research and development of the concept of metacognitive hierarchy may direct classroom practices and cognitive tools that help students transfer their cognitive skills to near and far locations.

Implications for instruction

If metacognition were expertly and effectively facilitated, student learning would be augmented by use of the associated knowledge and strategies. The problem is that such a facilitation requires a deep and fundamental understanding of metacognition, which may be lacking in teachers who focus on content knowledge and technical skills or ineffective with teachers who treat metacognition as domain-general and distinct from content. This study implicates cognitive labelling for self-Awareness as the basis of higher levels of the AMERT framework, however metacognitive labelling by teachers as a strategy to foster self-Awareness is relatively uncommon (Georghiades, 2004). Explicit cognitive labelling by teachers that ultimately enables metacognition at higher levels of AMERT may make a realistic (Barzilai & Chinn, 2018, p. 381) or even seismic, shift in learning. ‘Transfer’ is especially sought across the disciplines in Science, Technology, Engineering and Mathematics (STEM) education and other transdisciplinary work and studies, and guidance is needed for students to attain self-Transfer of cognition.

If there were advantages of providing the AMERT framework as an explicit metacognitive routine for students, the word ‘Monitor’ could be replaced by ‘Look’ so the acronym spells the more memorable ALERT:

• Aware of thinking: what thinking skills have you used previously?

• Look at your thinking: what thinking skills are you using right now?

• Evaluate your thinking: what thinking skills do you need to change or keep active right now?

• Regulate your thinking: how will you change (or maintain) your thinking skills?

• Transfer your thinking: how will you use in this current activity thinking skills that you learned in another activity?

Limitations and biases

This study has examined the viability of the AMERT hierarchy for metacognition with only a small sample of students from one cohort in one science degree. Confirmation bias may be present in the interview data where students possibly provided the interviewer with what they perceived the researcher wanted to hear, however this risk was to some extent mitigated by the interview's focus on cognition, not on metacognition. The analysis of data identified epitomising statements of students’ experience to provide evidence of each level of AMERT, but not what is typical. While there was evidence for the first four levels of AMERT for each facet, the evidence for a hierarchy between these levels was inferred, not direct. While disconfirming evidence is absolutely necessary, interviews that do not directly ask about metacognition have little capacity to provide such evidence. Only substantial further research on AMERT, first on its viability and, if warranted, then validity and finally on the reliability of measures from instruments that it frames, will be able to confirm or disconfirm this hierarchical framework by appropriately portraying metacognitive elements and their relationships.

Conclusion

If facilitated metacognition can contribute to student-learning gains to the extent claimed by the literature, then understanding metacognitive teaching and learning strategies should be one of the top-research priorities in science education. As metacognition can be a conceptual glue between diverse learning contexts for a student, metacognitive teaching and learning strategies should be particularly important in transdisciplinary STEM education. However, the value of research on metacognitive gains in the literature is equivocal due to concerns about construct validity and instrument reliability, where current disparate measures of metacognition reflect a diversity of metacognitive frameworks. The majority of frameworks for metacognition are non-hierarchical and some are overly complex with unhelpful overlap among metacognitive elements, inclusion of elements that are not metacognitive and unclear relationship between metacognitive knowledge and metacognitive skills. This paper presented a hierarchical, five-level metacognitive framework that integrates the literature from the 1970s to the present and comprises self-Aware of cognition, self-Monitor cognition, self-Evaluate cognition, self-Regulate cognition and self-Transfer cognition (AMERT). The use of AMERT to analyse data from the empirical study in this paper supports the viability of the framework in one context. Further qualitative studies will need to more fully test the viability of AMERT and, if the hierarchical framework is supported by diverse qualitative studies, quantitative studies are then warranted that determine the framework’s construct validity and the reliability of instruments developed from it. The development and testing of AMERT presented here is a step towards an empirically validated and practical metacognitive framework for researchers and ultimately may guide teachers to enable students to be metacognitively alert in science.

Acknowledgments

This work was supported by the Office for Learning and Teaching, the Australian Government, under Grant Number ID11_1984.

The research received ethics approval from the University of Adelaide Human Research Ethics Committee. H-0242006

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

We greatly appreciate the work and support of Dr Susan Hazel and a big thank you to the students who were interviewed for this project.

Appendix 1:

The Six facets of the Research Skill Development framework (Author)

Table

Embark & Clarify What is our purpose? Students respond to or initiate direction, clarify and consider ethical, cultural, social and team (ECST) issues.

Find & Generate What will we use? Students find information and generate data/ideas using appropriate methodology.

Evaluate & Reflect What will we trust? Students determine the credibility of sources, information, data and ideas, and make their own research processes visible.

Organise & Manage How do we arrange? Students organise information & data to reveal patterns/themes, managing teams and processes.

Analyse & Synthesise What does it mean? Students analyse information/data critically and synthesise new knowledge to produce coherent individual/team understandings.

Communicate & Apply ethically How do we relate? Students apply their understanding and discuss, listen, write, perform, respond to feedback and present processes, knowledge and implications of research, heeding ethical, cultural, social and team (ECST) issues and audience needs.

Appendix 2:

Interview Questions for Animal Science Honours students

1. What degree programme did you study? In what major?

2. What is your current study pattern and/or employment?

3. What are the strengths of your research skills?

Where/how were these developed?

Areas that you may still be developing?

• (4) How relevant are research skills to your current contexts?

• (5) In what ways are you using research skills in your employment/teaching/study?

• (6) In which course or year developed your research skills the most so far?

• (7) In Chicken & Egg e-Sim for Year 1 (Hazel, 2011b) and Companion Animal and Equine Studies for Year 2 (Hazel, 2011a) the lecturers made research skills explicit by showing these six facets of research skills (show list). How aware were you that they are designed to increase autonomy in assignments from the first year?

• (8) What would you say is the relevance of each of the following to you now in Honours and to any students who you teach/supervise/tutor? What do you think is the purpose of using this RSD in Honours?

• (9) Have you seen these six facets before (provide title of each in turn)? If yes, where, what differences has each made?

  1. Initiating/clarifying whether by posing questions/establishing problems or gaps/determining project goals

  2. Finding relevant information or generating appropriate data

  3. Evaluating information and data and reflecting on the process to generate

  4. Organising information/data and managing the process

  5. Analysing and synthesising

  6. Communicating and applying

• (10) What would have happened if you did not have structured guidance (or explicitly develop research skills) from the first year? What difference would have been made? (beneficial or hindering)

• (11) What would have happened if you did (explicitly develop research skills) have over consecutive years? What difference would have been made? (beneficial or hindering)

• (12) What is your current perspective of use of the RSD

For yourself

For undergraduate students generally

Across your degree programme?

• (13) Are there any other factors that we have not discussed that helped or hindered you to develop research skills?

• (14) Which assignments in your 3 years of university do you think helped you in developing your research skills the most?

• (15) Did the fenced dog park assignment help you to develop any research skills? If so, which ones?