Geography Curricula Objectives and Students’ Performance: Enhancing the Student’s Higher-Order Thinking Skills?

Abstract

This paper centers on to evaluate whether and to what extent the learning objectives of the geography curricula emphasize students’ higher-order thinking skills (HOTS), and whether students are capable of answering to HOTS-questions by using the Finnish upper secondary geography education as an example. The revised Bloom’s taxonomy was used as a framework for the content analysis. The findings show that geography has the potential to enhance students’ HOTS, but students experience difficulties when answering to HOTS-questions. The results could be used to evaluate the desired thinking skills and knowledge dimensions in geography education for to enhance students meaningful learning.

Keywords:

• Learning objective
• revised Bloom's taxonomy
• students' performance
• thinking skills
• geography curricula

Introduction

Higher-order thinking skills and geography education

The last twenty years have seen discussion of the weakened position of geography in education, reflected, for example, in falling student intakes, decreased credit hours, and geography being seen as an umbrella subject, or as an optional subject in the curriculum (see e.g. Bednarz, Heffron, and Solem 2014; Chang 2014; Lane and Bourke 2017; van der Schee, Notté, and Zwartjes 2010). At the same time, the news mainly concerns geographical themes like tourism, the threat of global pandemics, climate change, refugees and migration, the global economy, deforestation, forest fires, etc. However, these are not acknowledged as geographical phenomena. The media often portrays geography as a knowledge of topography and facts about the world’s places and regions (see e.g. Favier and van der Schee 2012, 666). However, rather than teaching simple facts about the world, according to Favier and van der Schee (2012, 666), geography should be seen “more like an activity that students can engage in”, and geography educators should assist students to learn, acquire, and use geographical knowledge, as well as to develop their geographical skills and attitudes to be able to do geography (Chang and Kidman 2019, 2; Favier and van der Schee 2012, 666; van der Schee, Notté, and Zwartjes 2010, 7).

Bednarz (2019, 521) proposes that geography has three “secret powers,” i.e. ways of thinking: spatial thinking; geographic thinking; and geospatial thinking. Additionally, it is suggested geographical knowledge has the potential to enhance people’s thinking and opinion-making skills, especially when combining concrete facts with abstract ideas and knowledge gained from geographical research (Béneker and Van Der Vaart 2020, 8). Moreover, geography educators should be able to communicate more effectively the thinking processes and core content of our discipline to the wider public (Bednarz 2019, 523). It is therefore interesting to examine the kind of thinking geography education can enhance.

In this paper, we have approached geographical thinking skills and knowledge dimensions through Bloom’s taxonomy, originally presented by Benjamin S. Bloom and revised by Anderson and Krathwohl in 2001 (Anderson et al. 2014). The revision includes six domains of cognitive processes divided into lower-order (LOTS) (remember, understand, apply) and higher-order thinking skills (HOTS) (analyze, evaluate, create), and four domains of knowledge: factual; conceptual; procedural; and metacognitive knowledge (Anderson et al. 2014; see also Virranmäki, Valta-Hulkkonen, and Pellikka 2020; and Tikkanen and Aksela 2012). It should be said that the division between LOTS and HOTS is contested: sometimes remembering is said to be the only lower-order thinking skill (see e.g. Anderson et al. 2014). However, the categories in the taxonomy are hierarchical, but they also overlap (Krathwohl 2002, 215), forming a continuum (Anderson et al. 2014).

It has been suggested that teaching and learning should focus on HOTS and metacognitive knowledge, because education can thus enhance meaningful learning (Airasian and Miranda 2002; Bijsterbosch, van der Schee, and Kuiper 2017; Krathwohl 2002). The research of Kumpas-Lenk, Eisenschmidt, and Veispak (2018) proposed that students were more motivated, engaged, and satisfied in their studies when learning outcomes were designed to demand higher-order thinking. However, the research of Stes et al. (2012) noted that designing the learning processes to target HOTS does not mean students will produce their answers at the same level, while Anderson et al. (2014, 21) state that “not all students learn the same things from the same instruction even when the intended objective is the same,” and eventually, not all learning outcomes can always be stated as objectives.

Concerning geography education, only a few studies have examined the revised Bloom’s taxonomy and objectives in curricula, or students’ ability to use HOTS in their answers. Research into the cognitive processes and knowledge dimensions of the geography assessment questions (see Virranmäki, Valta-Hulkkonen, and Pellikka 2020; Bijsterbosch, van der Schee, and Kuiper 2017; Wertheim and Edelson 2013) and geography textbook questions (see e.g. Jo and Bednarz 2009; Krause et al. 2017; Mishra 2015; Şanli 2019;Yang 2013, 2015) has concluded that LOTS are emphasized. Some researchers (see e.g. Collins 2018; De Miguel González and De Lázaro Torres 2020; Favier and van der Schee 2014; Liu et al. 2010) have noted that digital technologies, like digital representations such as digital maps and geographical information systems, may be suitable for enhancing students’ HOTS. However, it has been said there is insufficient empirical evidence to show that digital maps are better for improving students’ higher-order thinking (Collins 2018, 139).

We (Virranmäki, Valta-Hulkkonen, and Pellikka 2020) have earlier examined the geography test questions during the digitalization process of the Finnish matriculation examination between 2013 and 2019 and concluded geography test questions to require mainly understanding of conceptual and factual knowledge. Although the questions requiring analysis have been increased during the digitalization. Thus, the aim of this study is to evaluate whether and to what extent the learning objectives (LO) of geography curricula might emphasize students’ higher-order thinking skills, and whether students are capable of answering HOTS-type questions in both paper-based and digital tests. This information could be used to evaluate the desired thinking skills and knowledge dimensions in geography education.

Recent changes in Finnish upper secondary geography education

From the perspective of Finnish upper secondary geography education, there have been three major changes in recent years. These changes create the background for this research in which the main interest is to examine how the changes have influenced geography education. First, a drastic change occurred in 2014, when geography lost one of its compulsory courses in upper secondary education as a result of the Finnish government’s decision concerning the distribution of lesson hours between different subjects (Valtioneuvosto 2014). Second, the core curriculum for upper secondary education was revised in 2015. The former curriculum dated back to 2003 (see Finnish National Board of Education [FNBE] 2016, 2003). The result was that the content of the geography curriculum remained almost the same, but the order of the upper secondary school courses changed (see Appendix A). Curriculum reform, operated by the National Agency for Education, is usually carried out every tenth year. In this study, we focus on the 2003 and 2015 geography curricula. The curriculum has since been reformed in 2019, so students starting their studies in 2021 will be taught according to the new curriculum.

Third, the Matriculation Examination (M.E.) was nationally completely digitalized during 2016–2019 due to a decision of the Finnish government in 2011 (see more information in Virranmäki, Valta-Hulkkonen, and Pellikka 2020). This meant that, before the autumn 2016, all students took the tests in the paper-based form, and after the digitalization reform all students have taken the examination in digital form. The M.E. is the national large-scale (approximately 35,000 participants per year; held biannually (in the spring and autumn) simultaneously in all Finnish upper secondary schools) summative assessment of learning outcomes at the end of upper secondary general education. It aims to examine whether students have accomplished the skills and competences defined in the National Core Curriculum for General Upper Secondary Schools (FNBE 2003, 2016). The M.E. consists of at least four tests, of which only the mother tongue test is compulsory. Students can include the geography test in their M.E., but it is not mandatory. Geography, philosophy, and German (as foreign language) were the first subjects to be digitalized in the autumn of 2016; mathematics was the last in the spring of 2019. In addition to these major changes, the students’ acceptance for higher education in the spring of 2020 onwards is mainly based on their success in the M.E. instead of the current entrance exams. This will increase the importance of the tests in the M.E.

We have chosen to examine geography education in the Finnish Upper Secondary context, in which geography belongs to the natural sciences and is taught as a named subject with one compulsory and three optional courses for all students. We acknowledge that national educational aims and school systems vary widely (see e.g. Butt and Lambert 2014, 9), and in some parts of the world, for example in Sweden, the Netherlands, China, and parts of the United States and South America, geography is taught as part of social studies. However, geography’s status as a named subject in upper secondary education is quite common across the globe: for example, in Sweden, the Netherlands, China, parts of the United States, Argentina, Brazil, Chile, Guyana, Paraguay, and Uruguay (Bednarz, Heffron, and Solem 2014; Brooks, Quian, and Salinas-Silva 2018; Uhlenwinkel et al. 2017). Furthermore, researchers have noted that geography seems to have a general “body of knowledge” (Butt and Lambert 2014, 1) and a general understanding of geography education’s goals (Chang and Seow 2018, 32). Keeping these and the major changes that have occurred in the Finnish upper secondary education scene in mind, we suggest that Finland offers a good and interesting example of geography education.

The purpose of this study is to analyze the geography objectives of the Finnish National Core Curricula for General Upper Secondary Schools published in 2003 and 2015 in terms of the levels of cognitive and knowledge domains of revised Bloom’s taxonomy, and to examine students’ higher-order cognitive outcomes in geography tests in the paper-based (between the autumn of 2015 and spring of 2016) and digital (between the autumn of 2016 and spring of 2017) forms. Our research questions are: (1) To what extent – if at all – does the geography learning objectives of the Finnish upper secondary curricula (published in 2003 and 2015) reflect higher-order thinking skills and different geographical knowledge dimensions? (2) Are students capable of demonstrating their higher-order thinking skills when answering the Finnish M.E.’s geography test questions in the paper-based and digital forms?

Research design

Analysis through revised Bloom’s taxonomy

Krathwohl (2002, 212) describes the revised Bloom’s taxonomy as “a framework for classifying statements of what we expect or intend students to learn as a result of instruction.” We have applied the revised Bloom’s taxonomy in geography education and produced a framework (Tables 1 and 2) in which we represent the thinking skills and knowledge dimensions in the context of geography education by using the LOs of the Finnish geography curricula, and students’ performance when answering the HOTS-type questions in the paper-based and digital M.E. in geography as an example. The Tables present the type of skills that students are expected to show in their answers when demonstrating different cognitive processes and knowledge types. In this research, we use the term HOTS-type questions to refer to questions that require students to analyze, evaluate, or create conceptual or procedural knowledge. Here, metacognitive knowledge could have also been involved in HOTS-type questions. However, as our previous research (Virranmäki, Valta-Hulkkonen, and Pellikka 2020) concluded, there were no questions requiring metacognitive knowledge in the geography tests of the Finnish M.E. between 2013 and 2019. For this reason, we did not analyze students’ ability to use metacognitive knowledge in their answers.

Table 1. The cognitive process domains of the Taxonomy Table applied in the context of the objectives of geography and students’ answers to the geography tests (based on Anderson et al. 2014 and Virranmäki, Valta-Hulkkonen, and Pellikka 2020).

		The student is capable of showing that they…	Examples of the objectives of the 2003 and 2015 geography curricula. The student…
Lower-Order Thinking Skills LOTS	Remember	Recognize geographical symbols from the presented material	Knows different cultures (GE2, 2003)
	Retrieve relevant knowledge from long-term memory (recognizing, recalling)	Remember simple facts, and recall concepts and pictures from long-term memory	Knows which professions and work duties require geographical competence (GENERAL, 2015)
	Understand	Describe different geographical phenomena by listing and explaining concepts	Is able to use the basic concepts of human geography (GE2, 2003)
	Construct meaning from instructional messages, including oral, written, and graphic communication (interpreting, exemplifying, classifying, summarizing, inferring, comparing, explaining)	Give examples, compare and classify geographical concepts	Understands how and why natural landscapes change and knows how to interpret the structures, origins, and development of natural landscapes using images and maps (GE2, 2015)
		Infer and explain how geographical processes work from information presented
		Translate and summarize the information of a given representation into a different form
	Apply	Apply simple geographical models or theories to explain different phenomena	Is able to use geographical information system applications (GE4, 2015)
	Carry out or use a procedure in a given situation (executing, implementing)	Apply knowledge of geographical methods, e.g. calculate a surface area	Is able to use information and communication technology in acquiring, analyzing, and presenting data on global issues (GE1, 2015)
Higher-Order Thinking Skills HOTS	Analyze	Select relevant information from the presented material and organize it to form a coherent conclusion so that causalities between phenomena or concepts are visible	Is able to analyze the demographic development and settlement in different regions of the world, as well as the causes and consequences of urbanization (GE2, 2003)
	Break material into constituent parts and determine how parts relate to one another and to an over-all structure or purpose (differentiating, organizing, attributing)	Analyze the values and attitudes of the presented material	Is able to analyze positive development in different areas in the world and the factors affecting it (GE1, 2015)
	Evaluate	Draw conclusions and judgements from given phenomena, based on known criteria and standards by justifying views	Is able to compare and assess the susceptibility of different areas to risks globally and locally (GE3, 2003)
	Make judgments based on criteria and standards (checking, critiquing)	Be critical, i.e. critical thinking is visible	Is able to evaluate how regional characteristics of human activities are affected by the opportunities provided by natural resources and the environment (GE3, 2015)
	Create	Put elements together in a way that forms a coherent whole that is a new way to see phenomena and hypothesize how the phenomena are going to proceed – i.e. by answering a “what then?” -question	Is able to consider potential solutions for economic problems and social inequality (GENERAL, 2003)
	Put elements together to form a coherent or functional whole; reorganize elements into a new pattern or structure (generating, planning, producing)	Show creative and holistic thinking by reorganizing elements	Is able to pose geographical questions and use geomedia in solving geographical problems (GE4, 2015)

Table 2. The knowledge domains of the Taxonomy Table applied in the context of objectives of geography and students’ answers to the geography tests (based on Anderson et al. 2014; and Virranmäki, Valta-Hulkkonen, and Pellikka 2020).

	The student is capable of showing that they…	Examples of the objectives of the 2003 and 2015 geography curricula. The student…
A) Factual knowledge Aa) Knowledge of terminology Ab) Knowledge of specific details and elements	Demonstrate knowledge of simple facts or specific details, concepts, elements, or phenomena	Understands the meaning of human rights (GENERAL, 2015) Knows the different expressions of the developmental differences (GE2, 2003)
B) Conceptual knowledge Ba) Knowledge of classifications and categories Bb) Knowledge of principles and generalizations Bc) Knowledge of theories, models and structures	Demonstrate knowledge of the causalities between concepts by connecting things Explain theories, models, structures, classifications, categories, principles, and generalizations with the help of examples	Is able to use the basic concepts of physical geography (GE1, 2003) Understands phenomena arising from the planetary nature of the globe (GE2, 2015) Is able to observe everyday environments and describe regional phenomena, structures, and interactions in nature and human activity (GENERAL, 2015)
C) Procedural knowledge Ca) Knowledge of subject-specific skills and algorithms Cb) Knowledge of subject-specific techniques and methods Cc) Knowledge of criteria for determining when to use appropriate procedures	Apply knowledge of the geographical methods and criteria in using the methods in certain situations Demonstrate an understanding of the grounds of the specific method and can use it in a real situation	Is able to use geomedia in preparing a geographical study, or in a participation and involvement project with a local, regional, or global dimension (GE4, 2015) Is able to visualize regional information in the form of maps, diagrams and photographs (GE4, 2003)
D) Metacognitive knowledge Da) Strategic knowledge Db) Knowledge about cognitive tasks, including appropriate contextual and conditional knowledge Dc) Self-knowledge	Knows how to read and memorize geographical texts Knows how to construct a geographical answer for different cognitive tasks (e.g. recalling versus critiquing) Displays knowledge of their own strengths and weaknesses	(not found in the 2003 or 2015 curricula) Is able to evaluate their own solutions to a geographical problem Is able to monitor their comprehension of geographical phenomena

We have also used revised Bloom’s taxonomy to analyze students’ answers, even though the taxonomy was originally developed to examine educational objectives and give “commonly understood meaning to objectives classified in one of its categories” (Krathwohl 2002, 218), because we are especially interested in students’ HOTS. To our knowledge, the SOLO-taxonomy (developed by Biggs and Collis [1982] to analyze students’ work and products) oversimplifies the higher cognitive processes. We suggest that revised Bloom’s taxonomy gives a better understanding of students’ thinking skills at the higher level of cognitive domains by differentiating three levels of thinking: analyzing, evaluating and creating. We have therefore sought to examine students’ answers from the perspective of whether they can demonstrate the required thinking skills in their answer, i.e. do students’ answers reflect higher-order thinking skills?

All researchers in this study are familiar with revised Bloom’s taxonomy. To attain consistency in the categorization of LOs and students’ answers, a preliminary framework based on revised Bloom’s taxonomy was developed by the first author (part of Tables 1 and 2). We then discussed the content of the tables and the principles of the categorization process together. The first author then made required changes to the tables to ensure all the authors agreed the categorization principles and process. Content analysis, which was performed utilizing this framework, was then used to revise and finalize Tables 1 and 2.

In this study (Table 1), LOTS consist of three domains. The first is remembering, which is a prerequisite for other cognitive processes, because the recognized and recalled scattered information is often used in more complex tasks (see Anderson et al. 2014, 66, 70). The second, largest, and very comprehensive category is understanding, which relates to the construction of connections between prior and new knowledge. In understanding, students mainly describe geographical phenomena by listing information and explaining concepts related to a given task. However, because of its comprehensive nature, understanding is an important part of geography education and has huge potential to enhance geography learning (see also Bijsterbosch, van der Schee, and Kuiper 2017, 18). The third is applying, in which students use their prior knowledge of geographical models, theories, or procedures (e.g. calculations and spatial data analysis) to solve familiar or unfamiliar exercises or problems (see Anderson et al. 2014, 77).

The more comprehensive skills, HOTS, include the domains of “analyze”, “evaluate” and “create”. Analyzing is a continuation of understanding, because it requires students to reorganize knowledge from different sources, recognize causalities, and decide the suitable information to be used in a given situation (see Anderson et al. 2014, 79). Whereas evaluation is an extension of analyzing, because students are expected to use “standards or performance with clearly defined criteria” (Anderson et al. 2014, 83) to make justifiable arguments and draw a firm conclusion, based on the analysis they have done beforehand. In other words, when students are evaluating, they draw conclusions and judgments based on the evaluation they have done by checking or critiquing something from a certain perspective, i.e. critical thinking is required. The most comprehensive domain is “create”, in which students solve a given problem by planning how to do it, generating different outcomes or producing a real solution for it (Anderson et al. 2014, 85). Creating requires a student to use deep understanding and synthesize scattered material to produce an organized whole. Creative and holistic thinking and synthesis are part of creating knowledge.

The knowledge domain (Table 2) of revised Bloom’s taxonomy is divided into four dimensions: factual; conceptual; procedural; and metacognitive. Factual knowledge consists of knowledge (e.g. terms, facts, concepts) that are separate parts of information, whereas in the conceptual domain, knowledge forms a larger entity, in which connections (for example between concepts) are seen, and learners’ everyday experiences are transferred (see Anderson et al. 2014, 42). When students use knowledge of how to do something, they are using procedural knowledge that is the use of subject- or discipline-specific skills, methods, and techniques, whereas metacognitive knowledge refers to students’ own knowledge of their strengths and weaknesses when studying geography, and to the broader skills of learning and problem solving (see Anderson et al. 2014, 52–60).

Content analysis of the learning objectives and students’ answers to the geography tests

The empirical part of this study consists of two different data sets, which were analyzed using the previously mentioned framework as a basis for the analysis. The geography LOs of the Finnish National Core Curriculum for General Upper Secondary Schools published in 2003 and 2015 (FNBE 2003, 2016) were analyzed between January and February 2020. The students’ answers to HOTS-type geography questions from the Finnish M.E. in both paper-based and digital forms were analyzed between January and March 2018. The data of the students’ answers was obtained from the Finnish Matriculation Examination Board (later FMEB). We used a qualitative approach and theory-driven content analysis to systematically analyze the data. Additionally, researcher triangulation was used in the analysis process. The reliability of our analysis was further strengthened by dialogue between the three researchers during the research process.

The analysis process of the LOs of the geography curricula published in 2003 and 2015 was started by the first author, who collected the LOs from the curricula. She then repeatedly read the LOs, later constructing a table presenting all the LOs, categorized into revised Blooms’ taxonomy according to the previously mentioned framework. The second and third author read this preliminary categorization individually. We later discussed the categorization at a joint meeting, ensuring that we had a common understanding of the categorization criteria (see Tables 1 and 2). Finally, the first author produced the final categorization of the LOs, which was then read and approved by the other two authors. The consistency of the produced framework was checked repeatedly during the research process.

There was a total of 76 LOs in the two analyzed curricula. These were further divided into 107 smaller LOs, because some LOs included two or more objectives. Additionally, we found ten LOs that we could not categorize at any level of revised Bloom’s taxonomy, because they did not reflect any parts of the taxonomy. Therefore, among the 107 LOs found in the curricula, 97 were categorized in one or more category of the taxonomy table, because the categories were hierarchical but also overlapped (Krathwohl 2002, 215). There are therefore a total of 174 categorizations (67 LOs in the 2003 curriculum and 107 LOs in the 2015 curriculum).

The data consisting of 200 students’ answers to four examinations, altogether 800 students taking part in the geography test in the Finnish M.E. between the autumn of 2015 and spring of 2017 was received from the FMEB. Thus, the data consisted of 400 students’ answers to paper-based (tests in the autumn of 2015 and spring of 2016) as well as 400 students’ answers to digital (tests in the autumn of 2016 and spring of 2017) forms. We limited the analysis of students’ answers to the HOTS-type questions (analyze, evaluate, or create conceptual or procedural knowledge) of the Finnish M.E. (analysis found in Virranmäki, Valta-Hulkkonen, and Pellikka 2020), because we were interested in whether students were capable of demonstrating their HOTS and conceptual or procedural knowledge. The analysis is therefore based on students’ answers to 33 HOTS-type questions.

The first author was responsible for communication with the FMEB, and did a preliminary categorization (according to the preliminary framework, Tables 1 and 2) of a sample of the students’ answers from a digital test in the autumn of 2016. Several joint meetings were then organized, and the results of the preliminary categorization were discussed and evaluated by all authors. These meetings ensured that all the authors agreed to the categorization principles. Finally, the first author conducted the final analysis process.

For every question (n = 33), we analyzed a random sample of 50 students’ answers from the data of 200 students’ answers. However, in some cases, there were not 50 answers out of 200 students’ answers in the research material. In these cases only some of the 200 students had answered the particular questions and the rest had not answered (as was the case for the digital test in the autumn of 2016: in assignment 7 A, for which there were only 35 answers of 200 students’ answers and 7B for which there were only 35 answers of 200 students’ answers, and in assignment 9B, for which there were only 19 answers of 200 students’ answers; and the digital test in the spring 2017 test: in assignment 5 C, for which there were only 46 answers of 200 students’ answers). From the paper-based tests, there was a total of nine assignments, consisting of 13 HOTS-type questions. Therefore, 650 students’ answers were analyzed. Meanwhile, in the digital test, there were 12 assignments with 20 HOTS-type questions, and 935 analyzed students’ answers. We therefore categorized a total of 1,585 answers in the content analysis.

Research findings

The thinking skills and geographical knowledge emphasized in the geography LOs in the 2003 and 2015 curricula

We analyzed the geography LOs of the Finnish National Core Curricula for General Upper Secondary Schools produced in 2003 and 2015. Figure 1 presents the percentages of the thinking skills and the geographical knowledge found in the analyzed LOs. We found that the 2015 curriculum’s LOs were slightly more demanding in terms of thinking skills than the 2003 curriculum’s LOs. In the 2015 curriculum, 61 percent of the LOs required lower-order thinking (remember, understand, apply); 39 percent required higher-order thinking (analyze, evaluate, create). In the 2003 curriculum, the percentages were 69 percent and 31 percent respectively. However, when the LOs requiring the use of higher-order thinking were examined more closely, only analytical thinking increased, from 6 percent to 14 percent, while the percentages of LOs emphasizing evaluating or creating remained almost the same. This means that understanding the causal relationships within and between human and physical geography phenomena has become more numerous.

Figure 1. The distribution (in percentages) of the geography LOs in the 2003 (n = 67 LO) and 2015 curricula (n = 107 LO) according to the cognitive and knowledge dimensions.

When examining the knowledge dimension of the LOs of the two geography curricula we found that most of the LOs in both curricula emphasized conceptual knowledge (60% of the 2003 curriculum’s LOs and 54% of the 2015 curriculum’s LOs), which means knowledge of classifications, categories, principles, generalizations, theories, models, and structures in geography. Second, the LOs emphasized the use of procedural knowledge (24% and 27% of LOs respectively), in other words, the knowledge of how to do geography, subject-specific skills, methods, and techniques. The LOs emphasized factual knowledge least (16% and 19% of LOs respectively). That is knowledge of geographical terms and specific details. Metacognitive knowledge was completely lacking in the geography LOs.

In examining the distribution (percentages) of the different higher-order thinking skills and knowledge dimensions between the general LOs of geography and the course-specific LOs (Tables 3 and 4) in more depth, we found that HOTS were distributed more evenly between the different courses in the 2015 curriculum. Analytical and evaluative thinking, as well as conceptual and procedural knowledge, were pursued in all geography courses and in general objectives. However, in both curricula, the highest level of thinking, creating, was pursued only in the general and course 4’s LOs. Additionally, we found that the only compulsory course (GE1) of the 2015 curriculum completely lacked factual knowledge and creative thinking. Sixty percent of the LOs in this course emphasized remembering and understanding conceptual knowledge, and 90 percent of the LOs were categorized as conceptual knowledge in this course.

Table 3. The distribution (in percentages) of the different higher-order thinking skills and knowledge dimensions between general and course-specific objectives in the 2003 curriculum.

Geography objectives	Higher-order thinking skills			Knowledge dimension
	Analyze	Evaluate	Create	Factual	Conceptual	Procedural
General	25	45	17	45	33	31
GE1 The blue planet	0	0	0	18	18	6
GE2 A common world	50	18	0	36	20	0
GE3 The world of risks	0	36	0	0	25	6
GE4 Regional research	25	0	83	0	5	56
Total	100	100	100	100	100	100
N	4	11	6	11	40	16

Table 4. The distribution (in percentages) of the different higher-order thinking skills and knowledge dimensions between general and course-specific objectives in the 2015 curriculum.

Geography objectives	Higher-order thinking skills			Knowledge dimension
	Analyze	Evaluate	Create	Factual	Conceptual	Procedural
General	27	23	20	25	29	28
GE1 The world in change	7	12	0	0	16	3
GE2 The blue planet	20	18	0	25	17	17
GE3 A common world	27	29	0	25	19	17
GE4 Geomedia	20	18	80	25	19	34
Total	100	100	100	100	100	100
N	15	17	10	20	58	29

Additionally, we found ten LOs in the geography curricula (3 out of fifty LOs in the 2003 curriculum, and 7 out of 57 LOs in the 2015 curriculum) that were categorized as value-based LOs, because they were incapable of reflecting any of the levels of revised Bloom’s taxonomy. These included tolerance and respect for cultural diversity and human rights, acting as an active global citizen promoting sustainable development, and taking a stance on local and global issues, and gathering experiences and interest in geography, and how geography examines the world.

Students’ higher-order thinking skills and geographical knowledge in the paper-based and digital tests

The examination of students’ ability to demonstrate their higher-order thinking skills and geographical knowledge in their answers to geography tests in the paper-based (Tables 5 and 7) and digital (Tables 6 and 8) forms are provided. In examining the cognitive dimension of the students’ answers (Tables 5 and 6), we found (Table 5) that students were able to show (54% of answers) and use analytical thinking skills in their answers in the paper-based tests. This means students could select relevant information and organize it coherently, making causal relationships visible. However, analytical thinking skills were seen only in 35 percent of the students’ answers in the digital tests (Table 6) and in the majority of answers (53%), students mainly listed and explained concepts. This means that they understood the given geographical phenomenon. Overall, students had difficulties analyzing diagrams about climate change, the potential of wind energy, and geographical information system methods. Students performed well when the assignments required them to analyze maps about free-time living in Finland, the regional characteristics of world population growth, and the environmental impact of fishing and aquaculture.

Table 5. Distribution of students’ answers (n = 650) as a percentage (in numbers) at different levels of cognitive skills when the HOTS-type question (n = 13) required analyzing, evaluating, or creating in the paper-based form of the Finnish Matriculation Examination.

	Cognitive dimension of the student answer
Cognitive dimension of the question	Remember	Understand	Apply	Analyze	Evaluate	Create	Total
Analyze	3.5 (16)	37 (165)		54 (245)	5.5 (24)		100 (450)
Evaluate	4 (2)	24 (12)		38 (19)	34 (17)		100 (50)
Create		7 (10)		17 (26)	1 (2)	75 (112)	100 (150)

Table 6. Distribution of students’ answers (n = 935) as a percentage (in numbers) at different levels of cognitive skills, when the HOTS-type question (n = 20) required analyzing, evaluating, or creating in the digital form of the Finnish Matriculation Examination.

	Cognitive dimension of the student answer
Cognitive dimension of the question	Remember	Understand	Apply	Analyze	Evaluate	Create	Total
Analyze	9 (60)	53 (361)		35 (238)	3 (22)		100 (681)
Evaluate	4.5 (7)	34 (51)		46.5 (70)	15 (22)		100 (150)
Create	17.5 (18)	13.5(14)		15 (16)		54 (56)	100 (104)

Table 7. Distribution of students’ answers (n = 650) as a percentage (in numbers) at different levels of geographical knowledge, when the HOTS-type question (n = 13) required conceptual or procedural knowledge in the paper-based form of the Finnish Matriculation Examination.

	Knowledge dimension of the student answer
Knowledge dimension of the question	Factual	Conceptual	Procedural	Total
Conceptual		100 (550)		100 (550)
Procedural		27 (27)	73 (73)	100 (100)

Table 8. Distribution of students’ answers (n = 935) as a percentage (in numbers) at different levels of geographical knowledge, when the HOTS-type question (n = 20) required conceptual or procedural knowledge in the digital form of the Finnish Matriculation Examination.

	Knowledge dimension of the student answer
Knowledge dimension of the question	Factual	Conceptual	Procedural	Total
Conceptual	10 (76)	90 (705)		100 (781)
Procedural	12 (18)	52 (80)	36 (56)	100 (154)

Our analysis revealed students had difficulties when questions required them to evaluate. In the paper-based tests, only 34 percent, and in the digital tests only 15 percent, of the students’ answers were structured in such a way that evaluative thinking was visible, i.e. conclusions and judgments were formed and justified. Most of the students’ answers were grouped either in the category of “analyze” (38% and 46.5% of answers respectively) or in the category of “understand” (24% and 34% of answers respectively). Therefore, most of the students’ answers lacked critical thinking. In the category of “evaluate”, there were assignments that required students to evaluate errors in geographical information systems or to evaluate the pros and cons of the mining industry, for example.

Students seemed to perform well when answering questions requiring them to create. Creative and holistic thinking were evident in 75 percent of the students’ answers in the paper-based tests, and in 54 percent of the students’ answers in the digital tests. In this category, the questions required students to create a map representing the larger geographical area and plan a geographical study, for example. However, students had difficulties in hypothesizing what impacts the mining industry might have in a certain area or how to improve the world’s food production. In the paper-based tests, 17 percent, and in the digital tests 15 percent, of their answers mainly showed analytical thinking, so pondering possible improvements was difficult for some students.

In researching the knowledge dimension of the students’ answers (Tables 7 and 8), we reported that almost all the students (100% of the students’ answers in the paper-based tests, and 90% of the students’ answers in the digital tests) were capable of answering questions requiring them to use conceptual knowledge, using geographical concepts, theories, models, and generalizations, and connecting them. However, in the digital tests, 10 percent of the students’ answers showed only factual knowledge of simple facts or specific details. Additionally, procedural knowledge was evident in 73 per cent of the students’ answers in the paper-based tests. Students were able to use geographical skills, techniques, and methods related to geographical information systems especially were well known, but students had difficulties when they were required to draw a map. However, in digital tests, only 36 percent of the students’ answers showed procedural knowledge. There were difficulties in using knowledge of methods about geographical information systems, planning a geographical research plan, or producing an altitude profile.

Discussion and conclusion

In this research, we analyzed Finnish upper secondary school geography LOs of the 2003 and 2015 curricula through the framework of revised Bloom’s taxonomy to examine whether and to what extent the geography curricula emphasized students’ higher-order thinking skills. Additionally, we analyzed students’ performance in answering the HOTS-type questions of the Finnish geography M.E. between the autumn of 2015 and spring of 2017 in both paper-based and digital forms.

Our results show that geography has the potential to enhance students’ higher-order thinking skills. We found that the current 2015 curriculum is slightly more demanding and 39 percent of LOs emphasize HOTS. However, this is mainly because of the increased requirement of analytical thinking skills (and the decreased requirement of understanding and remembering), whereas evaluative and creative thinking remained almost the same during the research period. When we examined the knowledge requirements of the geography LOs, we found that in both curricula, conceptual knowledge was the most emphasized knowledge, while procedural knowledge was the second most emphasized, and factual was the least valued knowledge type.

Furthermore, we found that the distribution of the LOs of the 2015 geography curriculum between the different courses did not support the learning of higher-order thinking skills. The highest level of thinking, creating, is found only in the general objectives, and in course 4. The curriculum reform resulted in only one compulsory geography course, which is quite one-sided in terms of cognitive and knowledge demands. However, the curriculum reform improved the structure of the curriculum so that analytical and evaluative thinking, as well as conceptual and procedural knowledge, were pursued in all geography courses, and HOTS are distributed more evenly between the different courses. We also identified value-based LOs, which increased during the curriculum reform. Teaching values, like sustainable development and diversity, are an important part of geography teaching and learning (see e.g. Uhlenwinkel et al. 2017). The value-based LOs included in the curriculum reflect geography’s ability to teach values that are important in helping students to appreciate a diverse society, in which everyone has the opportunity to participate actively (see also Bednarz 2019). The value-based LOs must therefore be included in the curriculum.

Regarding our second research question, our analysis of the students’ answers shows that students have difficulties demonstrating HOTS, especially in the digital tests. The results support the research of Stes et al. (2012) (see also Anderson et al. 2014) in concluding that not all students produce answers at the required HOTS-level. However, contrary to our findings, it has been suggested previously that digital technologies could be suitable for enhancing students HOTS (see e.g. Collins 2018; De Miguel González and De Lázaro Torres 2020; Favier and van der Schee 2014; Liu et al. 2010; Palladino and Goodchild 1993). However, the students’ answers lacked evaluative and creative, as well as holistic, thinking in both test forms, and analytical thinking in the digital tests. According to our interpretation of revised Bloom’s taxonomy (Tables 1 and 2), this means that although the questions required students to understand causalities and form a coherent conclusion, make judgments, and justify their views (i.e. be critical) or hypothesize possible changes, they were incapable of showing these thinking skills in their answers. However, these are all an important part of geographical learning (see e.g. Bednarz 2019; Béneker and Van Der Vaart 2020). Additionally, in the digital tests, students had difficulties when required to use procedural knowledge, i.e. knowledge of how to make a geographical research plan or draw an altitude profile or map.

The reason for these problems in the digital tests may be the tests’ transformation from the paper-based to digital format. We analyzed the last two paper-based tests and the first two digital geography tests, and the newness of the digital format may have affected the students’ answers, because they may not have become accustomed to digital studying yet. However, the M.E. in geography will be in digital form in the future and therefore, it is substantial to gain more knowledge on how well students can demonstrate their knowledge and thinking skills in digital geography tests. This study provides important new knowledge on this issue.

Relying on our results and also on previous research, which states that to enhance meaningful learning, teaching and learning should focus on HOTS and metacognitive knowledge (Airasian and Miranda 2002; Bijsterbosch, van der Schee, and Kuiper 2017; Krathwohl 2002), we suggest three development aspects for consideration. First, we need a careful rethinking of the desired thinking skills and knowledge dimensions of LOs in geography, and a revaluation of the distribution of the HOTS-type LOs between the different geography courses. The LOs requiring evaluative and creative thinking skills, as well as the use of procedural knowledge, should clearly be reconsidered, because students have difficulties when using these skills. Second, we suggest that there is a need to engage students to planning teaching and learning geography i.e. they should know what skills and knowledge they are required to use when learning geography. In this way students can “learn to use knowledge” (Béneker and Van Der Vaart 2020, 9), and “be conscious and mindful about their thinking processes” (Bednarz 2019, 525) i.e. they become aware of the metacognitive knowledge possibilities in their learning. This might empower youth to be critical thinkers and take responsibility for their own learning and doing. This requires more training for teachers and students about thinking skills. Clear instructions and careful practice in the classrooms could help students to perform better when using the HOTS. Students especially need practice in their evaluative thinking skills, i.e. how to make firm conclusions based on given material and known criteria, and to justify their views. Third, we need more discussion between teachers and students about LOs in geography to increase students’ metacognitive knowledge (see Anderson et al. 2014, 51), which is completely lacking in the current geography curriculum.

We have created a framework by applying a revised Bloom’s taxonomy. This reflects our understanding of it; it should be said that the interpretation of the taxonomy is always a matter of subjective statement (see Anderson et al. 2014, 33). However, we have attempted to describe the produced framework as accurately as possible so that other researchers can evaluate and use it in their own research. Using Finnish geography curriculum LOs and students’ answers to the M.E. as an example, we have shown that geography is much more than learning simple facts, it is about doing geography (see e.g. Chang and Kidman 2019, 2; Favier and van der Schee 2012, 666; van der Schee, Notté, and Zwartjes 2010, 7). Geography has the potential to enhance students HOTS. Geography’s position in the curriculum varies across the globe, but it is acknowledged to have some common knowledge and skills that make our findings about the Finnish context applicable elsewhere. Awareness of thinking skills and knowledge dimensions in geography curricula and students’ answers provides some insights for educators and policymakers in designing LOs and assessment questions that are more inspirational and engaging for students, and improve geography teaching and learning. Yet, researchers should also study them in different educational contexts to be able to communicate more effectively the “key ideas of our discipline and its distinct thinking processes and perspectives” (Bednarz 2019, 523) to the wider public.

Acknowledgment

We would like to thank the Finnish Matriculation Examination Board for giving us a research license for the use of the data of students' answers.

Additional information

Funding

This work was supported by LUMA (STEM) Centre of the University of Oulu.

Notes on contributors

Eerika Virranmäki

Eerika Virranmäki (MSc, Doctoral Researcher) is doing her PhD study in the Geography Research Unit at the University of Oulu. Her research interests are in the field of geography education primarily in the upper secondary schools, geographical knowledge and changing pedagogies in geography education and assessment.

Kirsi Valta-Hulkkonen

Kirsi Valta-Hulkkonen (PhD) is working as a lecturer in the Teacher Training School in the Faculty of Education at the University of Oulu. Her research interests are in the fields of geography and biology education.

Anne Pellikka

Anne Pellikka (MSc, Doctoral Researcher) is a university teacher in the didactics of biology, geography and health education in the Faculty of Education at the University of Oulu. Her research interests are in the fields of biology, geography and environmental studies in the context of teacher education.

Appendix

Appendix A. Content of school geography in Finnish upper secondary schools according to national core curricula of 2003 and 2015.

2015 curriculum	2003 curriculum
GE1. The world in change	GE3. The world of risks
Geography as a field of science	Hazard geography
Key global risks areas related to the system of nature	Hazards related to system of nature and to natural resources and the environment
Key global risks areas related to natural resources and the environment	Hazards related to humankind
Global risk areas and essential development questions of the humankind	Technical hazards
GE2. The blue planet	GE1. The blue planet
Geographical thinking related to physical geography	Geographical thinking
Planetary movements of the Earth and phenomena caused by these	Planetary movements
The atmosphere and hydrosphere in motion	Atmosphere
The structure and variable topography of the Earth	Hydrosphere
The use of physical geographic data in the society and daily life	Weather and climate
	Topography of the Earth
	Vegetation zones of the Earth
	Interpretation of the natural landscapes
GE3. A Common world	GE2. A common world
Geographical thinking related to human geography	Human geography
Population and settlements	Population and settlements
Primary production and the environment	Natural resources
Industry and energy	Primary production and the environment
Services, movement and interaction	Industry and energy
The regional structure of human activity	Movement and interaction
The use of human geographic data in the society and daily life	The regional structure of human activity
	Development control and sustainable development
GE4. Geomedia – explore, participate, and get involved	GE4. Regional research
The use of geomedia in daily life, the world of work, and the promotion of sustainable development	Cartography and geographical research materials
Geomedia and geographic research skills	Geographic information systems
Development control and sustainable development	A geographic study
A geographic study or a participation and involvement project