Alignment between biology curriculum standards and five textbook editions: a content analysis

ABSTRACT

This study investigates the alignment between curriculum standards and textbooks, based on a standards-based science education context. The analysis sample includes the High School Biology Curriculum Standards and five editions of biology textbooks in China. Porter's alignment model is used to construct a two-dimensional (content areas and cognitive levels) matrix of curriculum standards and textbooks to calculate alignment indices and marginal discrepancies between the two dimensions. The results show that 1) alignment between curriculum standards and textbooks has not been achieved; 2) textbooks are highly consistent and statistically significant, but independent of curriculum standards; 3) the distribution of curriculum standards and textbooks across various core concepts and cognitive levels is unbalanced; 4) both curriculum standards and textbooks overemphasise the cognitive levels of remembering and understanding, while minimally representing the applying, analysing, evaluating, and creating levels. This study examines the development of curriculum alignment in the context of current worldwide standards-based science education reforms and provides a framework for alignment research based on content analysis.

KEYWORDS:

• Alignment evaluation
• textbook
• content analysis

Introduction

Standards-based science education reform emerged as a global phenomenon in the 1990s and remains ongoing (Kahveci, 2010; Liu & Fulmer, 2008). This reform is characterised by attempts at consistency among various educational elements, including content standards, tests, textbooks, and even classroom instruction as experienced by teachers and students (Porter et al., 2007). According to the enactment curriculum model (Remillard & Heck, 2014), consistency between standards and tests only exists within the internal mechanism of official curriculum; however, of import is the match between official and operational curricula. Instructional materials, especially textbooks, play an important role in transforming curricular goals into instructional plans, and in supporting teachers to enact these plans (Remillard, 2005; Remillard et al., 2014).

Against the background of the new round of Chinese biology fundamental education reform, this research aims to reveal the alignment between Chinese biology curriculum standards and the five textbook editions as well as to provide a text-based analysis framework for examining the alignment between curriculum standards and textbooks. The findings can be used for comparative studies with different countries and may provide insights to improve science education from an international perspective.

Research background

Curriculum model: the role of curriculum standards and textbooks

In this study, we draw upon a conceptualisation of the enacted curriculum model, which situates curriculum policy, design, and enactment systems (Remillard & Heck, 2014). This model reveals the interrelations and important functions of various curriculum elements. According to the model, consistency between curriculum standards and textbooks is essentially the most important objective component of the consistency between official and operational curricula, and it can reveal fairness in education (You et al., 2019).

Curriculum standards are policy instruments used to articulate the vision, or framework, of a subject-matter discipline to its educational system (Schmidt et al., 2005) and through which the knowledge and skills students are expected to acquire in school are specified (La Marca et al., 2000). The compilation of textbooks should consider curriculum standards as a baseline (Ministry of Education of China, 2020b); therefore, for the dimensions of content and cognition, each textbook should be highly consistent with the curriculum standards to facilitate the basic consistency and fairness of general education. However, curriculum standards encourage textbook originality (Ministry of Education of China, 2020a), and so, there can be differences between textbooks that show their respective characteristics (Sun & Li, 2021; You et al., 2019).

Textbooks are among the most common forms of instructional materials used worldwide, and they continue to play a critical role in national education systems (DiGiuseppe, 2014; Dogan, 2021; Lee & Wan, 2022; Morris et al., 2015; Vojíř & Rusek, 2019). Textbooks as artefacts highlight the potential for representing ideas, conveying practices, reinforcing cultural norms, and influencing teachers (BouJaoude & Noureddine, 2020; Brown, 2009). Science textbooks are designed to translate the abstractions of curriculum standards into operations that teachers and students can carry out before, during, and after instruction (Stein et al., 2007; Valverde et al., 2002; You et al., 2019). At all levels of schooling, science textbooks are often used as the primary organisers of the subject matter that students are expected to master, and they provide detailed explanations of topics to be taught (Chiappetta & Fillman, 2007), as well as how to learn and teach.

Given their impact on learning, textbooks could affect student learning in both direct and indirect ways. In a direct way, textbooks are among the primary sources from which students obtain knowledge. The indirect influence occurs when teachers utilise textbooks to organise their teaching (Aydin & Tortumlu, 2015; Irez, 2009; Pepin & Haggarty, 2001). Previous studies have found that science textbooks support the development of students’ scientific literacy by presenting a storyline for how the content can be communicated and learned in the classroom (Chiappetta & Fillman, 2007); providing an inquiry-based learning environment (McDonald, 2016); frequently encouraging students to investigate, directing them to think about a phenomenon or situation, respond to questions, or gather information (Chiappetta & Fillman, 2007); and improving students’ reading ability in the field of science (Fang & Wei, 2010). Additionally, textbooks hold a central position in providing students with high-quality opportunities to learn (Silver, 2009). In underdeveloped countries (or regions), in areas that lack internet access, and for poor families, textbooks may be the most important learning materials owned by students.

Considering their impact on teaching, textbooks influence both what is taught and how it is taught (Chiappetta & Fillman, 2007; Nirode & Boyd, 2021). There are three perspectives on how teachers use textbooks: they implement textbooks as fidelity, recreate textbooks as designers, and adopt textbooks according to specific instruction contexts (Arias et al., 2016). Even viewing themselves as using the materials with fidelity, teachers enact different curricula in their classrooms and do not adhere rigidly to their textbooks but, instead, offer modified activities, choose what is important, and skip parts of the curricula as they see fit (Remillard & Bryans, 2004; Thompson, 2022).

Teachers’ interactions with textbooks can be grouped into five categories: selecting materials, interpreting materials, reconciling intended goals with their own, accommodating students’ conditions (e.g. their interests, experiences, and limitations), and departing from the intended plan by adding embellishments, modifying existing structures, and omitting unsatisfactory parts (Brown, 2009). In daily teaching, teachers decide which textbooks (or which sections of a textbook) to use, when and where a textbook is to be used, the sequencing of topics in a textbook, the ways in which pupils engage with the text, and the level and type of teacher intervention between pupil and text (Pepin & Haggarty, 2001). Teachers draw on their own resources and capacities to read, make sense of, evaluate, adopt, adapt, and replace the offerings in textbooks (Remillard, 2005). A study conducted in Namibian science classrooms indicated that teacher-dominated textbook use, low frequency and restricted range of textbook references per lesson, and textbooks being used primarily in class for diagrams, data, and to verify factual information (Lubben et al., 2003). Well-written textbooks could support teachers by pointing out common misconceptions, explaining the correct science, and providing appropriate tasks to promote meaningful learning (Tshuma & Sanders, 2015). Additionally, textbooks are useful for supporting substitute, beginning, and non-specialist science teachers, in addition to facilitating continuity of programming and staff support in schools with high staff turnover (McDonald, 2016).

However, textbooks have also been criticised for merely emphasising the descriptions of facts and concepts, containing unscientific statements, and not accurately representing the nature of science (McDonald & Abd-El-Khalick, 2017; Tshuma & Sanders, 2015). Textbooks appear to provide more systematic opportunities to learn definitions and practice demonstrated skills but fewer opportunities to learn when and how to apply those skills (Stein et al., 2007). Science practices reflected in the textbooks were predominantly teacher-directed, with only limited opportunities for student engagement in science practices (Ndumanya et al., 2021). Regardless of the criticisms mentioned above, textbooks are no longer the only instructional materials; still, they are one of the mediums most familiar to students and teachers.

Research on alignment studies

Alignment is the cornerstone of standards-based science education. When an education system is aligned, all the messages from the policy environment are consistent with each other (Porter, 2002). This alignment is reflected in both internal and external consistency, similar to official and operational curricula: one is expected, and the other is enacted (Remillard & Heck, 2014). Internal alignment refers to the original intention behind the compiling of textbooks (Polikoff et al., 2011) based on curriculum standards; external alignment refers to the perception and measurement of this alignment by third parties, such as textbook users, teaching participants, and assessment participants. Although external alignment is somewhat subjective, it is closely related to the enacted curriculum and serves as a useful, practical, and empirical measure of consistency.

In research focusing on curriculum standards and assessments, alignment between the two can be defined as the degree of agreement between a state's content standards for a specific subject area and the assessments used to measure student achievement of these standards (Bhola et al., 2003; Contino, 2013). Studies on alignment between curriculum standards and assessments are abundant (Contino, 2013; Edwards, 2010; Liang & Yuan, 2008; Liu et al., 2009; Liu & Fulmer, 2008; Lu & Liu, 2012; Polikoff et al., 2011; Porter, 2002). The probable reason for this is that standards-based science education reforms gained momentum in the United States when the No Child Left Behind Act was signed into law. This law required all states to develop rigorous content standards and standardised tests to measure students’ mastery of these content standards (Liu et al., 2009), and this law has, in turn, affected international studies and practices. However, most results have shown that curriculum standards are not aligned with assessments (Edwards, 2010; Liang & Yuan, 2008; Liu et al., 2009; Polikoff et al., 2011; Porter, 2002; Resnick et al., 2004). Research on the alignment between curriculum standards and assessments involves fairness of evaluation, while that on the alignment between curriculum standards and textbooks involves equal rights in a wider range of educational fields, which also affects the consistency between teaching and assessments.

There has, however, been little rigorous investigation of the alignment between curriculum standards and textbooks (Sun & Li, 2021; You et al., 2019). Nor has there been a common, replicable approach to doing so at scale (Polikoff et al., 2015). Content analysis of textbooks can be used to assess the breadth and depth of the ‘intended’ curriculum (Porter, 2002); however, significant differences exist between the analysis textbook units and assessment items. Comparably, considerable differences can be noted in the units that analyse the content of textbooks, such as chunks (Polikoff, 2015; Polikoff et al., 2015), learning activities (You et al., 2019), and teaching objectives (Sun & Li, 2021). Some studies focus on the alignment of a particular topic, with substantial differences in the analysis unit and comparison method observed therein. Others take the teaching objectives as the analysis unit and tend to analyse a textbook’s intent rather than its substance. Most of these studies have adopted Porter's alignment model, which is very useful if the goal is to study the enacted curriculum (Martone & Sireci, 2009). The results show that the alignment between curriculum standards and textbooks is relatively low (Polikoff, 2015; Sun & Li, 2021; You et al., 2019). The alignment between different editions of textbooks, and between old and new editions from the same publishing house, is higher than that between curriculum standards and textbooks (Polikoff, 2015). There is a great need for quality information on textbook alignment to standards (Polikoff et al., 2015) to facilitate an international comparison of standards-based curriculum alignment. In terms of alignment research methodology, a more operational analysis framework is needed for systematic analysis and horizontal comparison.

Curriculum standards and textbook changes in China

Nine general education curriculum reforms have taken place in China since the early 1950s (Lu & Liu, 2012). The latest editions of the High School Curriculum Framework and the High School Biology Curriculum Standards (HSBCS) were published by the Ministry of Education in May 2020. The key points of the new curriculum standards put forward the Core Competencies and Values for Chinese Student Development (CCVCSD), which are divided into three aspects: cultural foundation, independent development, and social participation (Lin, 2017). Simultaneously, they also mention the Domain-Specific Key Competencies (DSKC) of each discipline; the DSKC of biology curriculum includes the ideas of life, scientific thinking, scientific inquiry, and social responsibility (Ministry of Education of China, 2020a). Instructions, teaching materials, and assessments need to be aligned with the new curriculum standards, and the main objectives of training for CCVCSD and DSKC need to be highlighted as well (Ministry of Education of China, 2020b).

Five new editions of high school biology textbooks have been compiled according to the HSBCS and approved by the National Textbook Council. Our research samples are the HSBCS and these five textbook editions – selected because they are one of the most recent samples of standards-based science education reform.

Research questions

This study investigates the alignment between the HSBCS and five editions of biology textbooks in China to answer the following questions:

RQ1: How is the alignment between the HSBCS and biology textbooks?

RQ2: What is the extent of alignment between the HSBCS and biology textbooks in terms of content areas?

RQ3: What is the extent of alignment between the HSBCS and biology textbooks in terms of cognitive level?

Methods

Alignment measures: porter's alignment model

Many alignment models exist, which produce different measures (La Marca et al., 2000; Martone & Sireci, 2009; Porter, 2002; Webb, 1997). Generally, they can be divided into three categories based on complexity: low, moderate, and high. The more complex a model, the less likely that a match will be found (Bhola et al., 2003). In the present study, we chose the moderately complex model of Porter (2002) to measure the alignment between curriculum standards and textbooks for several reasons. First, the use of a common language and the quantitative alignment index produced in Porter's model allows the degree of alignment to be measured and compared among any standard documents, instructional practices, and assessments (Liang & Yuan, 2008; Lu & Liu, 2012). Second, the statistical data of Porter's alignment model are based on a two-dimensional matrix composed of content areas and cognitive levels, which can be used for content analyses of instructional materials (Porter, 2002). In the HSBCS, the rows and columns of the two-dimensional matrix are clearly marked by the concept level and behavioural verbs – classified based on the revised Bloom's taxonomy. Hence, this matrix can be used as the gold standard for textbook analysis. Third, marginal discrepancies can be calculated, and sub-totals for rows and columns provide valuable information about the differences in content areas and cognitive levels (Liu & Fulmer, 2008).

We chose sentences as coding units according to Porter’s (2002) suggestion. Curriculum standards and textbooks are instrumental texts, which are different from ordinary literary works. Therefore, their sentence expressions are normative, complete, and clear, especially in the Chinese context. We constructed a two-dimensional matrix according to the content areas and cognitive levels of the revised Bloom's taxonomy. By coding the analysis units one by one, we summarised the data into the matrix. Whenever two or more knowledge types were involved, the coding was based on the knowledge type and the highest cognitive level (Wei & Ou, 2019). The HSBCS and textbooks were then established in the matrix; the numbers in each cell represented the number of analysis units appearing together in two dimensions. Finally, we performed the data standardisation calculation, which divides the number of cells in each matrix by the total number of matrices, and then adjusts the frequency proportionally. Therefore, the sum of the rows and columns was 1, and all matrices were unified and comparable. To elaborate the process, we constructed a simple X matrix representing curriculum standards and a Y matrix representing textbooks, with rows representing concepts and columns representing cognitive levels (Figure 1).

Figure 1. Examples of the content areas and cognitive levels matrix. Note: The number in the cell indicates the count, the number in parentheses indicates the proportion, and the letters and numbers in square brackets indicate the location of cells.

Porter's alignment indices (P) were calculated based on the pairing between matrices; its operational definition is (Liu et al., 2009; Porter, 2002) (1) $P=1-{\displaystyle \frac{{\sum }_{i=1}^n\left|X_i-Y_i\right|}{2}}$(1)

In the above equation, n denotes the number of cells in the matrix, and i is related to one particular cell ranging from 1 to n; X_i represents the ith cell’s ratio value of one matrix, and Y_i represents the ith cell’s ratio value of the other matrix; and X_i－Y_i represents the discrepancy between the two corresponding cells in the two matrices (Lu & Liu, 2012). The possible values of this index range from 0 to 1, with 1 indicating perfect alignment (Porter, 2002). Substituting the data in the matrix of Figure 1 into the formula, the alignment index was calculated as 0.87. The process is as follows: $\begin{array}{rl}& P=1-{\displaystyle \frac{\left|X_1-Y_1\right|+\left|X_2-Y_2\right|+\left|X_3-Y_3\right|+\left|X_4-Y_4\right|+\left|X_5-Y_5\right|+\left|X_6-Y_6\right|}{2}}\\ & =1-{\displaystyle \frac{\left|0.27-0.31\right|+\left|0.09-0.08\right|+\left|0.09-0.00\right|+\left|0.36-0.38\right|+\left|0.18-0.15\right|+\left|0.00-0.08\right|}{2}}\end{array}$

Porter's alignment indices can be compared with each other, but they cannot be judged for statistical significance. It is essential to assess how far an observed alignment index is from the centre of the distribution of indices that could occur by chance (Fulmer, 2011). As cell values of two random tables may result in a certain degree of alignment, it is necessary to find out how high an alignment index needs to be to claim its statistical significance (Li et al., 2020). MATLAB (2014b) was used to perform this assessment, and the function used was unidrnd. As the subtotal of the coding matrices of curriculum standards and textbooks was different, random number matrix pairs were generated respectively. Through 20 000 random operations (Lu & Liu, 2012), the normal distribution of the alignment index was obtained, and the significance reference value, ‘p’ (p = Mean+2SD), of the alignment index at the level of 0.05 was calculated. If P > p, the alignment index is significant; otherwise, it is not significant. The results of the 20 and 40 000 trial calculations were generally consistent. Additional iterations do not noticeably alter the distributions of the simulated alignment indices by cell (Fulmer, 2011). Therefore, in this study, we adopted the calculation results obtained after 20 000 random operations.

Content-area and cognitive-level categories

To construct the matrix, we chose the suggested content areas and cognitive levels as the two analysis dimensions (Liu & Fulmer, 2008). The content dimension of this study adopted the conceptual system of the HSBCS as a basis of classification. The HSBCS comprises four big ideas, 11 core concepts, and 45 third-level concepts. We chose the 45 third-level concepts as the basis of coding because their expression is clearer and more specific. Moreover, we chose the core concepts as the basic unit of data to achieve a balance between simplicity and effectiveness. This involved adapting the 11 core concepts as the rows of the two-dimensional matrix and using the serial numbers of the 11 core concepts in the curriculum standard as the codes. The specific content of each core concept is presented in Table 1.

Table 1. Four big ideas and 11 core concepts, with the HSBCS codes.

Big Idea	Code	Core Concept
A cell is the basic unit of an organism; cell structure and vital activity.	1–1	Cells are composed of various molecules, including water, inorganic salts, sugars, lipids, protein, and nucleic acids. Of these, protein and nucleic acids are the most important biological macromolecules.
	1–2	The cell's components separate and co-operate with each other to perform all the vital activities of the cell together.
	1–3	All cells have similar basic structures, but they are different in morphology and function.
Cells need energy and nutrients to survive, and they proliferate by division.	2–1	Substances enter and leave cells, via passive and active transportation, to maintain the normal metabolic activity of cells.
	2–2	Most of the functions of cells are based on chemical reactions, which occur in specific areas of cells.
	2–3	Cells will undergo life processes, such as growth, proliferation, differentiation, ageing, and death.
Genetic information controls biological traits and is passed down from generation to generation.	3–1	The genetic information transmitted from parental generation to filial generation is mainly encoded on DNA molecules.
	3–2	The segregation and recombination of genes in sexual reproduction lead to many possible gene combinations between parents and offspring.
	3–3	The variation caused by gene mutation, chromosome variation, and gene recombination can be inherited.
Biological diversity and adaptability are the result of evolution.	4–1	The species living on the earth are rich and diverse, and they come from common ancestors.
	4–2	Adaptation is the result of natural selection.

Categories of cognitive levels were based on the revision of Bloom's Taxonomy of Educational Objectives (Anderson et al., 2001), which has been widely used in related alignment studies (Edwards, 2010; Fulmer & Polikoff, 2014; Liang & Yuan, 2008; Liu et al., 2009; Liu & Fulmer, 2008; Lu & Liu, 2012). This taxonomy’s six dimensions, Remember, Understand, Apply, Analyse, Evaluate, and Create, were used to construct the columns of the matrix. Moreover, considering the differences between textbooks and educational objectives, more detailed coding tools with keywords, explanations, cases, and coding experiences were applied to the specific coding process.

Therefore, the coding result was composed of an 11×6 matrix: 11 categories of core concept content by six cognitive levels. By graphically converting the matrix, topographs (content maps) – in which specific topics are displayed as lines of latitude and cognitive levels are displayed as lines of longitude (Polikoff et al., 2011) – are used to represent relative emphases among different content areas and cognitive levels, which can help visualise where the distribution of focus is (Liu et al., 2009; Lu & Liu, 2012).

High school biology curriculum standards

The content of the HSBCS used for coding was the conceptual system of ‘content requirements’ in chapter 4, ‘course content’, which included big ideas, core concepts, and third-level concepts.

In this study, two researchers trained using the coding manual and experienced in biology teaching and research, carried out the coding independently, and then compared the coding results. To facilitate the coding process, keywords were used to classify analysis units into the different cognitive levels (Edwards, 2010). Each third-level concept provided an obvious behavioural verb as a symbol of Bloom's cognitive dimension; hence, the two coders’ coding consistency of Cohens’ kappa value was 0.808 (p = 0.000). The disputed codes were arbitrated by another trained coder, and all disputes were settled through negotiation and arbitration.

Five editions of high school biology textbooks

Ten copies of the latest high school biology textbooks (compulsory I and compulsory II from each of the five editions) were used for analysis. Compulsory textbooks represent the basic fundamental requirements of the national biology curriculum, which should be studied by all the students. The publishers of the five editions are People's Education Press, Jiangsu Phoenix Education Press, Zhejiang Science and Technology Press, Shanghai Science and Technology Education Press, and Beijing Normal University Press. We used the codes PA, PB, PC, PD, and PE to represent the above textbook editions. We encoded the main body of each textbook because it is the largest component and can effectively represent the textbook. We recognise that the introduction, supplementary information box, exercises, and pictures in textbooks are also integral, but they are not included in the present study and will be analysed in future research. To ensure accuracy, interrogative forms in the text were transformed into the declarative form, according to their specific content and their answers in the teacher’s solution books. Many of the obvious transitional sentences were excluded from the analysis because they served only linguistic functions.

The same two coders handled the textbook coding process. Taking two chapters as a unit, five rounds of trial coding were conducted. The coding results from each round were discussed, and the coding rules were jointly revised based on consensus. Sentence analysis units in textbooks could be clearly defined on two dimensions most of the time. The main content of some sentences with multiple meanings were selected as the coding basis. In contrast, some sentences with specific content were difficult to determine, so they were comprehensively analysed by combining the context (Knain, 2001). The two coders independently completed all the coding work, and the Cohens’ kappa value of coding consistency was 0.811 (p = 0.000). The disputed codes were arbitrated by another trained coder, and all disputes were settled through negotiation and arbitration.

Results

Coding results for curriculum standards and textbooks

The total number of codes in the curriculum standards was 45, and the valid number of codes in the textbooks was 6881 after deleting 1074 invalid analysis units. The number of codes for each edition of the textbook is 1179 (PA), 1268 (PB), 1619 (PC), 1734 (PD), and 1081 (PE). All the codes were counted into the 11×6 matrix, and the proportion was calculated according to the above-mentioned method (refer to supplementary materials for detailed data information).

Overall alignment

Table 2 presents the overall alignment indices in pairs. The unshaded data in the upper right represent the alignment indices (P) between pairs, and the shaded data in the lower left represent the significance-level reference values (p) of each pair, as calculated by MATLAB. According to the matching combinations of the HSBCS and textbooks, the alignment indices of the HSBCS and PA (P = 0.58, p = 0.60, P < p), HSBCS and PB (P = 0.54, p = 0.59, P < p), and HSBCS and PC (P = 0.54, p = 0.60, P < p) were lower and statistically insignificant. The alignment indices of the HSBCS and PD (P = 0.61, p = 0.57, P > p) and HSBCS and PE (P = 0.63, p = 0.60, P > p) were higher and statistically significant. However, the absolute values were not high. In brief, the alignment indices between the HSBCS and textbooks were low (mean = 0.58).

Table 2. Overall alignment indices between the HSBCS and five textbook editions (PA-PE).

	HSBCS	PA	PB	PC	PD	PE
HSBCS	1.00	0.58	0.54	0.54	0.61*	0.63*
PA	0.60	1.00	0.79*	0.80*	0.82*	0.88*
PB	0.59	0.71	1.00	0.83*	0.77*	0.73*
PC	0.60	0.71	0.70	1.00	0.80*	0.80*
PD	0.57	0.72	0.74	0.73	1.00	0.86*
PE	0.60	0.68	0.69	0.69	0.69	1.00

Note: The unshaded data represent alignment indices; shaded data represent the significance reference value of the alignment indices. * indicates statistically significant data.

In contrast, the alignment indices among the five textbook editions were high (mean = 0.81) and statistically significant. Figure 2 presents the ranks of the alignment indices. The thickness of the line represents the rank, and the actual or false value of the line represents statistical significance, or lack thereof. PA and PE have the highest alignment index (P = 0.88), while the HSBCS has the lowest alignment index (P = 0.54) with PB and PC. The radiation of the HSBCS is thin and dotted, indicating low alignment indices between the HSBCS and the five biology textbook editions. The radiation of PE is thick and solid, indicating that PE has high alignment indices with the HSBCS and other textbooks (mean = 0.78).

Figure 2. Overall alignment relationship between the HSBCS and five textbook editions (PA-PE) based on rank. Note: The thickness of the solid line represents the rank of alignment indices. The thicker the solid line, the stronger the alignment with statistical significance.

Figure 3 displays the emphasis distribution of the HSBCS and five textbooks for both content areas and cognitive levels. The horizontal axis of the topographs represents the core concepts of the content areas, and the vertical axis represents the six cognitive levels. The colour depth at the focus of the horizontal and vertical axes represents different proportional distributions. We use red to represent the HSBCS and black to represent textbooks, and we incorporate the red curriculum standard pattern in the topographs of textbooks for comparison. For instance, in the HSBCS, the focus of the understanding level in content 1–1 is crimson, indicating that the proportion of the understanding level in core concept 1–1 is between 0.08 and 0.11. The proportion represented by the lightest grey is 0.00–0.03 (refer to the legend).

Figure 3. Topographs of the HSBCS and five textbook editions (PA-PE) with content areas (x-axis) and cognitive levels (y-axis).

Examining each topograph separately, the distribution of curriculum standards and textbooks based on the various core concepts and cognitive levels is unbalanced. Both curriculum standards and textbooks overemphasise the cognitive levels of remembering and understanding, while only minimally focusing on the other levels. Simultaneously, in the content areas, core concept 1–3 of the HSBSC and textbooks is generally ignored. From the comparison of topographs, large differences are noted in the distribution of emphasis between the curriculum standards and textbooks as well as between the different textbooks, although the overall alignment index between the textbooks is high (mean = 0.81). The ideal topograph distribution should resemble a wheat field, not an abstract painting; at the very least, the topographs of curriculum standards and textbooks should be similar, with subtle differences between the two representing an ideal situation for alignment studies.

The overall alignment results indicate that curriculum standards and textbooks present two systems, and textbooks as a system have high consistency with each other. This implies a disjointed relationship between curriculum standards and textbooks. More specific difference analyses can be conducted from two aspects: content area and cognitive level.

Alignment by content area

Figure 4 presents the subtotals of the discrepancies by content area, with their direction compared with the HSBCS. The excessively over-represented core concepts include 2-2, 2-3, 3-1, and 3-2, whose main contents are cell functions, cell life processes, DNA, and gene. The excessively under-represented core concepts include 1-1, 1-3, 2-1, and 4-1, which elaborate upon the molecular composition of cells, similar structures but different morphologies and functions of cells, way in which substances enter and exit cells, and evolution theory. Over-or-under-representation of core concepts 1-2, 3-3, and 4–2 vary among the different textbooks, but these three concepts are more consistent with the HSBCS; specifically, the average absolute value of core concept 1–2 is 0.009, indicating its greater consistency with the HSBCS. Compared with the curriculum standard, the most over-represented core concept is 3–2 (mean = 0.05), with values of 0.07, 0.01, 0.05, 0.05, and 0.08 for PA, PB, PC, PD, and PE, respectively. The most under-represented core concept is 1–1 (mean = −0.04), with values of −0.01, −0.05, −0.09, −0.06, and −0.01 for PA, PB, PC, PD, and PE, respectively. Compared with the HSBCS, the maximum and minimum absolute values of misalignment for the dimension of content are for PC (0.37) and PE (0.26), respectively; the order from the maximum to minimum values is: PC > PA > PD > PB > PE.

Figure 4. Subtotals of discrepancies, by content area, with direction of textbooks (PA-PE) compared to the HSBCS.

Alignment by cognitive level

Figure 5 presents the subtotals of the discrepancies by cognitive level, with their direction compared with the HSBCS. The five textbook editions display a strong consistency. All textbooks over-represent the cognitive level of remembering (mean = 0.28), while under-representing the cognitive level of understanding (mean = −0.31). Textbooks represent the cognitive levels of applying and analysing to some extent, but the proportion is low. Textbooks do not present any content for the cognitive levels of evaluating and creating, which is highly consistent with the curriculum standards. Additionally, the biggest difference in the cognitive levels is between PB and the HSBCS, whether it is over-representation at the remembering level (0.34) or under-representation at the understanding level (−0.36). The levels of applying and analysing appear to be more consistent with the HSBCS. However, this is not the case, because the proportion of curriculum standards in these two cognitive levels is 0%; hence, even a small proportion shows a big difference between the textbooks and curriculum standards.

Figure 5. Subtotals of discrepancies, by cognitive level, with direction of textbooks (PA-PE) compared to the HSBCS.

Figure 6 shows the proportion distribution of the HSBCS and textbooks for each cognitive level. There are obvious differences between curriculum standards and the textbooks, and there is a high consistency between the textbooks. The proportion of the cognitive level of understanding in the curriculum standards is 87%, while it is approximately two-thirds of this level in the textbooks (mean = 56%). PB has the highest proportion of remembering (47%), and PE has the highest proportion of understanding (60%). PA is the only textbook with content distributed across the six cognitive levels: remembering (37.91%), understanding (56.49%), applying (0.85%), analysing (4.41%), evaluating (0.09%), and creating (0.25%). However, both the evaluating and creating levels account for less than 1% in PA. The cognitive proportions of PB, PC, PD, and PE for the evaluating level are 0%. For the creating level, the cognitive proportion of PB is 0.09%, while that of PC, PD, and PE is 0%.

Figure 6. Distribution proportion of cognitive levels in the HSBCS and five textbook editions (PA-PE).

Discussion

This study indicates that the required level of alignment between curriculum standards and textbooks is far from met. The alignment indices between the curriculum standards and textbooks (0.54–0.63, mean = 0.58) in this study were lower than in Sun and Li’s (2021) study, which showed an alignment index (0.76) between third-grade primary school science textbooks and curriculum standards. This misalignment between curriculum standards and textbooks has also been confirmed in other related research (see Polikoff, 2015; Sun & Li, 2021; Yang et al., 2019; You et al., 2019; Zhuang et al., 2021).

The larger the matrix, the harder it is to achieve consistency. As the number of cells increases, there is more room for discrepancy between the ratios, and values for the index are likely to be lower (Fulmer, 2011). Consistent with Polikoff’s (2015) study, our findings displayed a high and statistically significant alignment (0.73-0.88, mean = 0.81) among textbooks, but they did not fit the requirements of curriculum standards. This indicates that the textbooks are not covering what they should and not embodying their own distinctiveness. Compared with curriculum standards, textbooks are obviously self-contained. The current textbooks (People’s Education Press edition) in China can be traced back to the early 1950s. Therefore, the order in which textbooks and curriculum standards have appeared has resulted in the textbooks becoming a system, and the contents of the textbooks being relatively stable and dynamically fine-tuned. Particularly, China faces the problem of ‘teaching for exams’. Therefore, the examinations have a deeper and wider influence on textbooks than the curriculum standards. The study results indicated a distance between the textbooks and curriculum standards, possibly due to the historical inertia of textbooks. With the current development, studies of alignment measure the match, or quality of the relationship; to achieve alignment, either the curriculum standards or textbooks, or both, need to be revised.

In terms of content areas, the textbooks cover the conceptual system of the curriculum standards well. However, significant differences were observed in weight. These differences were not indicated or implied in the curriculum standards, showing that the compilation of textbooks was knowledge-based or subject-based (Ou & Wen, 2016). The differences in weight indicated a greater focus on the discipline standards of biology than on students’ learning and development. The weight depends more on the position and difficulty of knowledge in the discipline, just as the law of free combination of genes accounts for the highest proportion (mean = 0.14). However, if the weight relies more on the cultivation of students’ DSKC, the evolution content (mean = 0.09) should be strengthened. Concepts should be treated fairly, and the benchmark of such fair treatment should be based on developing the CCVCSD.

The imbalance in the cognitive levels is more obvious. All the contents in the curriculum standards focus on remembering and understanding, while applying, analysing, evaluating, and creating have 0% representation. On average, 97% of the analysed text in the five textbook editions focus on remembering and understanding, with the other four levels accounting for no more than 3% in total. It is strange that, even though teachers, students, society, the development of CCVCSD, and the 4C learning and innovation skills all call for great attention to higher-order cognitive ability, studies have shown that curriculum standards, textbooks, and examinations still lack exact descriptions of these skills and requirements for their implementation in K-12 education. Therefore, the reform of curriculum standards, textbooks, and examinations should further emphasise higher-order cognitive ability. Teachers can build their own teaching matrix (see Porter, 2006), constantly evaluate the differences between teaching and standards, and balance the distribution of different dimensions by filling in the blanks.

Implications

The study results have several implications. First, the alignment between curriculum standards and textbooks is generally insufficient, and the curriculum standards and textbooks form their own systems. Misalignment is widespread in all content categories and cognition levels. Under the background of standards-based science education, if there is serious misalignment, fairness and quality of education will be difficult to guarantee. Hence, further reforms of the curriculum standards and textbooks should focus on reducing this difference. Second, although the curriculum standards define the content and cognitive dimensions of the third-level concepts, detailed attention is warranted on how curriculum ideals can be realised in practice (Rogan, 2007). Thus, alignment based on a solid foundation can be achieved, rather than consistency based on an abstract foundation. Third, Chinese biology teachers can directly use the curriculum standard matrix constructed in this study to compare their actual teaching from the two dimensions of content and cognition, or use the textbook matrix to better select the resources in textbooks, or compensate for the imbalance of teaching concerns (such as higher-order cognitive ability) by checking the matrix distribution. Teachers in other countries can try to use the alignment matrix method to construct a local curriculum standard matrix, a textbook matrix, and their own teaching matrix, and then compare, select, and check them. Finally, textbooks are no longer single entities available to teachers (Chiappetta & Fillman, 2007); especially after the pandemic, online learning has become the norm, and various web-based materials are more convenient, timely, and useful, but they may not have been strictly screened and examined. Therefore, the evaluation method based on the alignment of the model adopted in this study becomes more important; the first step is not only to analyse textbooks, but more importantly, to evaluate various instructional materials by applying a standard matrix. If the misalignment with the standard exists at the higher level of cognition, it will promote development, and if it exists at the lower level, it will help to identify problems. Education administrators, developers of curriculum standards, developers of textbooks, teachers, and students can use the alignment model to evaluate teaching, learning, and instructional materials, and through constant comparison and evaluation, can be more pragmatic in promoting standards-based education and constantly promoting curriculum reform in the same context (alignment model).

Porter’s alignment model is adopted in this study. However, each model has its strengths and limitations, and investigating tools for examining alignment are both necessary and timely (Newton & Kasten, 2013). In this study, the unit of analysis is more objective when it constitutes sentences; yet, the integrity of text expression may be fragmented, and context may be ignored in such a case. Although in the process of coding, we asked the coders to consider these two aspects, the mechanism of sentence analysis, and its inherent limitations, still exist.

This study helps examine the development of alignment in the context of current worldwide standards-based science education reforms, and it provides a framework for alignment research based on content analysis. Further studies on the alignment among curriculum standards, textbooks, classroom instruction, assessments, and professional development for teachers should be undertaken; interdisciplinary and multi-year studies can provide valuable information in this regard. The influence of the alignment or misalignment of the education system on teaching and learning should be tested.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability

The data that support the findings of this study are available from the supplementary materials.

Additional information

Funding

This work was supported by the Research Program Funds of the collaborative Innovation Center of Assessment toward Basic Education Quality at Beijing Normal University under Grant number 2018-05-012-BZPK01; Fundamental Research Funds For the Central Universities under Grant number 2019TS058; and Fundamental Research Funds For the Central Universities under Grant number 2020TS049.