Student groups evaluating their group work and learning of critical online reading

Abstract

The study examined student groups’ (n = 72) self-evaluations of their group work and their learning of critical online reading during an inquiry task. The analyses focused on aspects of critical online reading, describing group work practices, and evaluating them. For learning critical online reading, the most often mentioned aspects were sources, perspectives, and author; corroboration and evidence were mentioned the least. About half of the groups mentioned 0–2 aspects which implies low diversity in learning critical online reading. The most often mentioned aspect in describing group work was division of work. In evaluating group work, member contributions were reflected most often. A majority of the groups mentioned four or five aspects of group work practices or evaluations which implies a moderate ability to reflect on group work. The results suggest that the students’ learning of critical online reading and reflecting on group work jointly were not very extensive.

Keywords:

• Collaborative learning
• critical online reading
• group work practices
• self-evaluation
• upper secondary school

Introduction

There is an urgent need for students to acquire critical online reading skills to differentiate between reliable and inaccurate information—or even deliberately produced disinformation and fake news (Jeong, Cho, & Hwang, 2012; Mihailidis & Viotty, 2017; Pennycook & Rand, 2021). This is evidenced, for example, by the disinformation campaigns supporting Donald Trump during the 2016 US presidential election (Bennett & Livingston, 2018), the spread of false information concerning the COVID-19 pandemic (Posetti & Bontcheva, 2020), and most recently, the massive spread of disinformation and fake news about the war in Ukraine (Aguerri, Santisteban, & Miró-Llinares, 2022). The recent development of AI applications makes it even more difficult to distinguish false arguments from correct ones (Benzie & Montasari, 2022; Kertysova, 2018). Disinformation can spread through all types of media, but it spreads most effectively through the Internet and social media channels (Zubiaga, Aker, Bontcheva, Liakata, & Procter, 2018). Since young people rely on social media and Internet resources particularly often, they face the risk of making important decisions based on inaccurate or flawed information (Horrigan & Rainie, 2006) or even falling victim to disinformation (Tandoc, Lim, & Ling, 2020). Jones-Jang, Mortensen, & Liu (2021) found that those with greater information literacy (i.e., the ability to identify, locate, understand, evaluate, and use information) were more likely to identify fake news. Previous studies (Bronstein, Pennycook, Bear, Rand, & Cannon, 2019; Pennycook & Rand, 2019, 2021) have also shown that people who are analytical and reflective are less prone to trust disinformation. These results support the importance of teaching critical reading and information literacy when combating the harmful effects of false and inaccurate information.

This poses a challenge for educators to develop appropriate pedagogical tools grounded in twenty-first century education that is both learner-centered and community-centered, without forgetting the role of assessment in learning (Griffin & Care, 2015). Students need to be engaged in collaborative knowledge-centered processes in order to develop their competencies and skills for acting in a responsible manner in a globalized and digitalized world. To be constructive, these processes require both cognitive and meta-cognitive skills (e.g., critical thinking, learning to learn, and monitoring and reflecting on learning progress), social and emotional skills (e.g., self-efficacy and working collaboratively and constructively with others), and practical and physical skills (e.g., using new information and digital devices) (Griffin & Care, 2015; OECD, 2023a).

To address these issues, we examine a novel pedagogical approach focusing on upper secondary school student groups’ self-evaluations of their group work practices and their learning of critical online reading during an online inquiry task.

Theoretical background

Online inquiry

Reading and writing skills are fundamental skills essential for learning and various advanced educational activities, such as online inquiry. Mullis and Martin (2019) defined reading literacy in the PIRLS (Progress in International Reading Literacy Study) assessment framework as “the ability to understand and use those written language forms required by society and/or valued by the individual. Readers can construct meaning from texts in a variety of forms. They read to learn, to participate in communities of readers in school and everyday life, and for enjoyment.” (p. 6). Besides the basic reading skills, this definition includes the reading of a variety of text forms and the social aspects of reading. In addition, reading is an active process of constructing meanings. Multimodal technology makes the reading process more demanding because different resources are combined, and multimodal texts require a new, multimodal approach to reading. Students might need support to navigate these various resources and understand their possibilities and challenges (Danielsson & Selander, 2016).

According to the online research and comprehension framework (Leu, Kinzer, Coiro, Castek, & Henry, 2013), online inquiry can be defined as a process in which students solve a problem or learn about a specific topic by examining multiple online texts. During this inquiry process, students engage in several online reading practices: identifying a problem and specifying an information need, searching for, evaluating, and synthesizing information, and communicating results to others (Leu et al., 2013). Online reading requires critical thinking skills and a critical attitude toward information, for example, to identify accurate and useful information, to consider the authors’ purposes, and to make inferences between the texts (cf. Cervetti, Pardales, & Damico, 2001). The term critical online reading is used to refer to readers’ abilities to consider, evaluate, and synthesize the source information and content of multiple online texts during online inquiry (Hämäläinen, 2023). Critical online reading is important in all phases of online inquiry, but it takes place particularly during the evaluation of information, which can be conducted based on the connected tiers of information—namely, context, source, and content (Forzani, 2018, 2020). When evaluating the context in which the online text is presented, attention is paid to elements of online resources, such as URL type (e.g., commercial or organization), text genre (e.g., blog or scientific article), and the date when the information was created. Evaluating the source refers to assessing the author or publisher of the online resource and their expertise, point of view, and intention for writing or publishing the text. When evaluating the content of online texts—a cognitively challenging element of evaluation and an important means to bring about learning—students assess the accuracy of ideas presented through argumentation (claims, evidence, and reasoning) and explanation. In this tier, students also evaluate whether the information presented is comprehensive (i.e., whether multiple perspectives are represented) and whether it corroborates their prior knowledge and other texts on similar topics. Corroboration requires students to compare and connect information sources and evaluate whether they support or oppose each other (Britt, Rouet, & Durik, 2018; Kohnen & Mertens, 2019; Wineburg, 1991). Within and across all these tiers and texts, students construct their topic understanding during the online inquiry.

Several studies have shown that students struggle with evaluating online information. A study by the Stanford History Education Group (2016) showed that students’ skills in judging the credibility of information delivered through social media channels were insufficient. Hämäläinen, Kiili, Räikkönen, & Marttunen (2021) found that when upper secondary school students selected and evaluated health-related online texts, they frequently used author, venue, and evidence but quite rarely used intention and corroboration as information evaluation criteria. Furthermore, Coiro, Coscarelli, Maykel, & Forzani (2015) found that seventh graders were unable to justify whether the author of a website was an expert in the field the website represented. International student assessments also show an alarming decrease in reading comprehension skills in Finland (Mullis et al., 2023; OECD, 2023b). In the digital environment, only 3% of Finnish adolescents were shown to demonstrate sufficient critical evaluation skills (Fraillon, Ainley, Schulz, Friedman, & Duckworth, 2020).

At several class levels, students have been found to have difficulties recognizing and evaluating biased and misleading information. For example, sixth graders (Kiili, Leu, Marttunen, Hautala, & Leppänen, 2018) and upper secondary school students (Marttunen, Salminen, & Utriainen, 2021) were found to possess limited abilities to recognize biased online sources. In addition, in a study by Breakstone et al. (2021), high school students had deficiencies in evaluating whether the evidence presented on a biased website was trustworthy and whether the website was created by an unbiased organization or author.

Group work practices

Research has demonstrated the positive effects of collaboration on different aspects of learning (Van Leeuwen & Janssen, 2019). In educational settings, the role of collaborative activities can be viewed from two perspectives: they can foster the learning of the topics under study (“collaborating to learn”) and they can promote the acquisition of collaborative skills (“learning to collaborate”) among students (e.g., Häkkinen et al., 2017).

For collaborating to learn, it has been acknowledged that under favorable conditions, collaborating with others can be effective for individuals to learn new knowledge and skills (Andrews & Rapp, 2015; Nokes-Malach, Richey, & Gadgil, 2015). For example, Kiili, Coiro, & Räikkönen (2019) reported that completing online inquiry tasks in groups was a promising practice for learning online inquiry skills among high school students. However, the benefits of collaboration on learning depend widely on the way group work is organized in the classroom (Huber & Huber, 2008) and on students’ ability to engage in high-quality group work (Leopold & Smith, 2019). Therefore, it is important to pay attention to how student groups work when collaborative learning is applied in schools.

For learning to collaborate effectively, collaboration skills are considered to be one of the key competencies students need in their future studies, work careers, and everyday lives (Griffin & Care, 2015; Ilomäki, Lakkala, Kallunki, et al., 2023; OECD, 2023a). Furthermore, students should learn to collaborate because the origin of knowledge lies in collaboration; knowledge is a result of the communication between various actors and is affected by the changing relations among the knowledge producers (Fuller, 1988, xiii-3). Thus, students at various educational levels should have opportunities to participate in collaborative activities throughout their studies, integrated with appropriate assignments. According to Earnest, Williams, & Aagaard (2017), good teamwork pedagogy includes goal-oriented group assignments, as well as explicit teaching and practicing of teamwork skills. In addition, reflecting on the collaborative activity is central both for developing group work practices and for individual skill learning (Gutwin & Greenberg, 2004). According to Scardamalia (2002), group self-reflection sessions at different stages of the working process increase participants’ understanding of collective knowledge building.

The features that characterize effective and successful group work include, for example, shared coordination of working strategies, shared responsibility, equal participation, active communication, constructive interaction, openly sharing information, and members giving and receiving help (Fransen, Weinberger, & Kirschner, 2013; Hoegl & Parboteeah, 2007; Taggar, 2002). A distinction is often made between mere cooperation, wherein the group members complete the task by dividing it into individually conducted parts, and collaboration, which includes coordinated working together to create shared understanding and outcomes (Baker, 2015). According to Hoegl & Parboteeah (2007), teams with high teamwork quality use multiple coworking strategies and combine individual and collective sequences in different phases of the process. Sormunen, Tanni, Alamettälä, & Heinström (2014) interviewed student groups in a secondary classroom about their group work strategies in a source-based writing assignment. The authors found that many groups used a combination of strategies during their working process, from more cooperative delegation or division of work to more collaborative pair or group work; however, about half of the groups failed to complete the writing assignment as a mutually coordinated effort. According to Ross (2008), the quality of group discussions in primary and secondary classrooms is often inadequate for enabling shared knowledge construction.

In the present study, these viewpoints on group work and collaboration had an effect on our design of the intervention task as a collaborative online inquiry that included group reflection on the inquiry process after each phase and on group work practices at the end.

Aims and research questions

Many studies on online reading have applied targeted measures to assess students’ skills in various elements of online reading skills at the individual level, such as justification skills (Hämäläinen et al., 2021), sourcing (Bråten, Stadtler, & Salmerón, 2018), evaluation of credibility (Kiili et al., 2018; Marttunen et al., 2021) and argumentation (Marttunen et al., 2021), as well as synthesizing information during online reading (Kiili & Leu, 2019). Students’ learning of online inquiry has also been the focus of several intervention studies (e.g., Brante & Strømsø, 2018; Hämäläinen et al., 2020, 2023; McGrew & Byrne, 2020). However, self-evaluations of studying and learning, particularly through working in groups, are scarce. Group work with elaborative interaction among students has proved efficient for the learning of complex conceptual knowledge in general (Van Boxtel, Van der Linden, & Kanselaar, 2000) and of online inquiry skills in particular (Kiili et al., 2019). Furthermore, students’ ability to engage in high-quality group work has been found to affect the success of collaborative learning (Leopold & Smith, 2019). Thus, there is a need for research about student groups’ self-evaluations when online inquiry skills are practiced through collaborative group work. This study aims to examine general upper secondary school student groups’ (n = 72) self-evaluations of their group work practices and their learning of critical online reading during an online inquiry task carried out in a classroom. The research questions (RQs) were as follows:

1. How did the student groups evaluate their learning of critical online reading during the online inquiry task?

   1. What was the focus of the students’ perceived learning?

   2. How diversely did the student groups describe their learning?

2. How did the student groups reflect on their group work during the online inquiry task?

   1. What was the focus of the students’ group work reflections?

   2. How diversely did the groups reflect on their group work?

3. What are the associations among student groups’ evaluations of their learning of critical online reading and their reflections on group work, and the background variables?

Method

Study context

The study was part of a larger research project, Argumentative Online Inquiry in Building Students’ Knowledge Work Competences, in which we investigated and developed a new pedagogical design for teaching online inquiry competences (Kiili et al., 2022). The study was conducted in Finnish language courses in nine classrooms with five voluntary teachers and their students. The researchers designed the working schedule and the materials (lecture slides, task instructions, and the online working document template), which were subsequently improved based on the teachers’ feedback in a joint workshop. The teachers were responsible for the classroom work, following the joint decisions made with the researchers about the content, the working process (structured in a template) as well as organizing the group work. Students worked in face-to-face groups in classrooms with online tools. Although the general guidelines were the same for all students, teachers guided the groups with a few minor issues, e.g., if a student had been absent from some lessons, or organizing the final presentations of the groups. This study reports the findings on student groups’ reflections on the online inquiry process and their group work. The group reflection questions presented to the students were implemented in the teaching process so that they also supported the pedagogical aims of the course.

Online inquiry task

The online inquiry task used in this study was created to enhance students’ online inquiry competencies and included four 75-minute lessons each designed to support one of the phases of online inquiry (Kiili et al., 2022): (1) the planning and implementing the search, (2) evaluating information, (3) synthesizing information, and (4) communicating information. The students were offered four alternative topics on a controversial health issue: cell phone radiation, food additives, sun and health, and sleeping pills. First, the students chose the topic, and after that, the groups were formed based on the chosen topics. At the beginning of the lessons, the teacher briefly introduced one of the phases of online inquiry. Next, the student groups followed an online working document in which the various phases of online inquiry and the main aspects of critical evaluation of online information (e.g., author, venue, intentions) were structured and guided, learning of which were the goals of the pedagogical intervention (Kiili et al., 2022). At the end of each lesson, the groups reflected on their outcomes and work. The reflection was prompted by the questions related to each phase of online inquiry. For example, one question for the phase “Evaluating information” was “How well did we consider different source features when evaluating the quality of online texts?”. At the end of the last lesson, the student groups evaluated their learning and working during the whole online inquiry task, and these final evaluations were used as data in the present study.

Participants

The participants (232 students in total; 44.8% females, 39.2% males, and 15.9% not answered; age range 16.7 to 18.9 years; M age 17.4 years) were recruited from four Finnish general upper secondary schools (one semi-urban school and three city schools; two city schools located in the metropolitan area and one city school and the semi-urban school located in the Häme region). The students worked in 72 small groups. Three groups consisted of two students, 50 groups of three students, and 19 groups of four students.

Informed consent was asked from all students and if a student was underage, consent was also requested from guardian(s). The research was approved by all the participating upper secondary schools and included no risk for participants. Thus, a statement by the Ethical Board of the University of Jyväskylä was not required.

Data

The data comprised the student groups’ answers to four self-evaluation questions presented as a part of the working document at the end of the online inquiry task. Two of the questions focused on learning critical online reading, and two questions focused on group work.

The self-evaluation questions on critical online reading (RQ1) were as follows: (1) What did we learn about critical online reading and its importance? and (2) How did the project (online inquiry task and related teaching) affect our attitudes toward information found on the Internet in the future? Correspondingly, the self-evaluation questions on students’ group work practices (RQ2) were: (3) How was group work planned and coordinated? What was successful? What was challenging and difficult? and (4) How did the group members participate in the joint work? How were the responsibilities and tasks divided between the group members?

Data analysis

Self-evaluations of critical online reading

Data for the analyses of RQ1 consisted of student groups’ responses to self-evaluation questions 1 and 2. The responses to these two questions were connected for the analyses, as they together shed light on what students thought they had learned about the important aspects of critical online reading that should be taken into account when reading online information. The final data consisted of 72 responses (3,059 words in total; average length = 42.5 words; SD = 28.2 words).

The data analysis encompassed both a theory-driven and a data-driven approach so that all student groups’ responses were analyzed based on two categories (1 “yes”; 0 “no”) depending on whether the particular aspect of critical online reading was mentioned in the response or not. The theory-driven aspects of critical online reading were author, venue, intentions, evidence, corroboration, and perspectives (Forzani, 2020; Hämäläinen et al., 2021; Leu et al., 2013; Marttunen et al., 2021). Furthermore, two analysis aspects (information search and sources) were defined based on data. The analysis aspects and related data examples are described in Table 1.

Table 1. Aspects of critical online reading identified in the analysis of student groups’ responses.

Aspect of critical online reading	Description	Examples of student responses that included the selected aspect
Author	The attributes (e.g., expertise, education, or professional experience) of the author of the online text	In the future, we will pay more attention to whether the author has expertise on the topic (Group 30). We learned to be more critical in terms of the occupation of the author (G 59).
Venue	The venue (e.g., site, publisher, or forum) of the source	We will familiarize ourselves with the background of the text publisher (G 55). It is necessary to take into account the publisher, place of publication, and date (G 54).
Intentions	The intentions or motives of the writer of the text	In the future, we will pay more attention to why the text was written (G 57). If only one source is used, a particular motive or purpose may affect how information is emphasized (G 38).
Evidence	The arguments (e.g., grounds, evidence, and reasons) for the claims or opinions presented in the text	We will pay more attention to how the information is justified (G 30). Some of these opinions may be based on the experiences of one individual, which is generally not enough to refute research performed by a widely recognized expert body (G 28).
Corroboration	Searching for other sources to verify the information presented in the text	In the future, we will check whether other sources support the initial source (G 4). We learned to consult several sources to verify the issue (G 37).
Perspectives	Perspectives (e.g., their number and diversity) presented in the sources	We will remember to compare sources and take into account sources with different positions (G 59). The given issue must be examined from various perspectives and from many sources (G 76).
Information search	Information search process (e.g., formulating search queries and search terms or scanning search results)	One should not lean on the first search result (G 5). Evaluating the reliability of sources goes well if one knows where to begin the search (G 27). We learned how important it is to use the correct search terms to find reliable sources (G 56).
Sources	Sources (e.g., their number and diversity or differences between sources)	We learned that it is important to examine different sources (G 45). In the future, we will be more critical of various sources (G 56).

To examine the breadth of students’ learning of critical online reading according to their own evaluations, the number of aspects of critical online reading mentioned in the student groups’ responses was counted and a three-category variable, diversity in learning critical online reading (low, moderate, or high) was formed.

Self-evaluations of group work practices

The data used for answering RQ2 consisted of student groups’ responses to self-evaluation questions 3 and 4. These responses were analyzed together, as the student groups reflected on various aspects of group work when answering both questions. The combined responses revealed how comprehensively each group reflected on their group work practices. The final data consisted of 71 responses (3,072 words in total; average length = 42.7 words; SD = 23.4 words). One group did not answer the questions about group work practices.

We analyzed the group work reflection responses following an abductive strategy (Timmermans & Tavory, 2012) in the content analysis, moving back and forth between theory and data and taking into account aspects of successful teamwork identified in previous research (Baker, 2015; Fransen et al., 2013; Hoegl & Parboteeah, 2007) and aspects found in the group responses. The responses included both neutral descriptions of group work practices and evaluations of the successes and challenges of group work, as asked in the self-evaluation questions. Similarly, as in the analysis of learning critical online reading, all student responses were analyzed based on two categories (1 “yes”; 0 “no”) depending on whether the particular aspect of group work was mentioned in the response or not. For responses describing group work practices, the aspects used in the analysis were coordination, division of work, joint working, co-construction, joint discussions, and helping each other. For responses evaluating group work, the aspects used in the analysis were evaluating members’ contributions, evaluating coordination, evaluating inquiry task success, evaluating collaboration, and evaluating general success. The aspects and related examples of the analyses are described in Table 2.

Table 2. Aspects of group work reflection identified in the analysis of student groups’ responses.

Aspects of group work reflection		Description	Examples of student responses that included the selected aspect
Aspects of describing group work	Coordination	Joint planning for organizing the process and responsibilities	We discussed in the group how to construct each section (Group 33). We had agreed on the parts to be presented in advance (G 43).
	Division of work	Dividing the tasks between group members and/or the method of division	We distributed the tasks equally to each group member (G 33). We basically agreed on the fly that you take care of this, and I take care of that (G 45).
	Joint working	Doing the tasks together and/or without explicit sharing of work	In the end, everyone worked on all the sources (G 31). Everyone was jointly responsible; no one was given individual responsibility (G 36).
	Joint discussions	Joint discussions and sharing of information when performing the inquiry task	We discussed the questions together (G 4). We thought together about the similarities and differences in the sources by discussing (G 53).
	Co-construction	Writing and constructing jointly the texts and/or outcomes of the inquiry tasks	We pondered the answers to all points together (G 20); We formed a common stance based on all the information we found (G 30).
	Helping each other	Giving or receiving help and/or asking for help during the inquiry task	We asked each other for advice if we did not know something (G 20). We helped each other when needed (G 41).
Aspects of evaluating group work	Evaluating coordination	Evaluating the organization of group work (e.g., planning and keeping the schedules)	Sometimes it was challenging to get everything done during the lessons, because in some sections, we pondered the answer for too long (G 14). Planning the work was easy (G 22).
	Evaluating members’ contributions	Evaluating members’ activity in group work (e.g., active participation and completing the tasks)	The division of tasks could have been a little more evenly distributed (G 5). Everyone participated actively, both in examining the sources and in the seminar (G 38).
	Evaluating collaboration	Evaluating working together in the group (e.g., joint working and discussing)	We should have done more work together (G 9). We worked well together all the time, so that one wrote while the other two dictated what to write (G 64).
	Evaluating inquiry task success	Evaluating the successes and/or challenges in the inquiry tasks	The most challenging was the selection of sources (G 37). The presentation of the project went well (G 43).
	Evaluating general success	Evaluating how the group coped with the task in general	The length of the project reduced the accuracy of working (G 50). In our opinion, the project as a whole was successful (G 53).

The number of group work aspects mentioned in the responses was counted, and a three-category variable, diversity in reflecting on group work (low, moderate, or high) was formed in order to examine the students’ ability to reflect on their group work. In addition, the sections of responses that included evaluative statements of group work were further analyzed to examine the types of successes and challenges the groups experienced during the project. This analysis was conducted in a data-driven manner through thematic analysis (Braun & Clarke, 2006), and the result was used descriptively to provide a better understanding of the project assignment from the student groups’ perspectives.

Reliability of the qualitative analyses

The final categories for the qualitative analyses were constructed iteratively, starting from a theoretical understanding of the phenomena—critical online reading and group work practices—and integrating it with the data-driven analysis of the student groups’ answers, combining categories or creating new ones according to a deeper understanding of the data (Braun & Clarke, 2006). Two author pairs carried out the initial coding of each phenomenon, which was then examined collectively by all the authors. Disagreements were discussed in multiple analysis sessions, and changes were made if needed. The idea of investigator triangulation was followed to guarantee the reliability and quality of the analysis procedures (Creswell & Miller, 2000). The main categories were based on previous research on critical online reading and collaboration described in the theoretical background section. The sub-categories identified through the analysis of group work reflection were diverse, and each had relatively low frequencies; thus, calculating interrater reliability was not considered relevant.

Statistical analyses

To answer RQ3, chi-square tests were performed to examine associations between the nominal-scale independent and dependent variables and to calculate intercorrelations (Pearson’s r) for interval-scale independent and dependent variables (Table 3).

Table 3. Variables used in the statistical analyses.

Variable name	Definition	Scale
Independent variables
Mother tongue grade	Average of the previous mother tongue grades of the group members	Interval
Health education grade	Average of the previous health education grades of the group members	Interval
School	School of the group members	Nominal
Topic	Topic (mobile radiation, food additives, sunlight and health, or sleeping pills) for the online inquiry task assigned to the group	Nominal
Group size	Number of members in the student group	Interval
Dependent variables
Learning critical online reading	Number of aspects mentioned by the student groups when evaluating their learning of critical online reading	Interval
Diversity in learning critical online reading	Classification of the groups (low, moderate, or high) based on the number of aspects mentioned in evaluating their learning of critical online reading	Nominal
Reflecting on group work	Number of aspects mentioned by the student groups when reflecting on their group work	Interval
Diversity in reflecting on group work	Classification of groups (low, moderate, or high) based on the number of aspects mentioned by the student groups when reflecting on their group work	Nominal

Results

Critical online reading: focus and diversity (RQ1)

To answer research question 1 (How did the student groups evaluate their learning of critical online reading during the online inquiry task?), the analyses were directed at both the focus and diversity in students’ perceived learning. To analyze the focus of students’ perceived learning, the aspects of critical online reading mentioned in the small groups’ responses were identified (RQ 1.1). The results showed that the most often mentioned aspects were sources (86.1% of the groups), perspectives (43.1%), and author (37.5%). The least frequently mentioned aspects were corroboration (15.3%) and evidence (9.7%). Roughly a quarter of the student groups mentioned information search (27.8%), intentions of the writer (26.4%), and venue of the source (23.6%). To analyze the diversity in students’ perceived learning (RQ 1.2), the number of different aspects related to critical online reading mentioned in the student groups’ responses was counted (M = 2.7; SD = 1.6), and three levels of diversity were formed (Table 4).

Table 4. Diversity in learning critical online reading in different groups based on the number of aspects mentioned in the responses.

Diversity in learning critical online reading	Number of aspects of critical online reading	f (groups)	%
Low	0–2	37	51.4
Moderate	3–4	25	34.7
High	5–6	10	13.9
Total		72	100

Note: The maximum number of different aspects of critical online reading was 8.

The majority (51.4%) of the student groups mentioned less than three aspects of critical online reading in their responses (Table 4), which implies a low diversity in learning critical online reading. High diversity in learning (five or six aspects) was reported by 13.9% of the student groups, and moderate diversity in learning (three or four aspects) by 34.7% of the groups. As a whole, the results regarding the students’ learning experiences on critical online reading suggest that the students’ learning was not extensive.

Appearances of the different aspects of critical online reading were also compared among the groups scoring high, moderate, and low in the diversity in learning critical online reading (Table 5). Due to the high skewness of the distribution for many of the individual aspect variables, statistical tests were not performed, and comparisons were made based on frequency.

Table 5. Aspects of critical online reading mentioned by groups with different levels of diversity in learning critical online reading.

Aspect of critical online reading	Diversity in learning critical online reading
	Low (37 groups)		Moderate (25 groups)		High (10 groups)
	f	%	f	%	f	%
Sources	30	81.1	22	88.0	10	100.0
Perspectives	7	18.9	15	60.0	9	90.0
Author	4	10.8	14	56.0	9	90.0
Information search	6	16.2	8	32.0	6	60.0
Intentions	1	2.7	11	44.0	7	70.0
Venue	1	2.7	7	28.0	9	90.0
Corroboration	2	5.4	5	20.0	4	40.0
Evidence	2	5.4	3	12.0	2	20.0

The aspects intentions and venue were mentioned proportionally more often in the group with high (70% and 90%, respectively) diversity in learning critical online reading compared to the groups with moderate (44% and 28%, respectively) and low (2.7% and 2.7%, respectively) diversity. Corroboration and evidence, which are cognitively demanding aspects of critical online reading, were seldom mentioned in all groups. However, they were proportionally more often mentioned in the group with high (40% and 20%, respectively) diversity in learning critical online reading compared to the groups with moderate (20% and 12%, respectively) and low (5.4% and 5.4%, respectively) diversity.

Group work (RQ2)

Answering research question 2 (How did the student groups reflect on their group work during the online inquiry task?) was approached by analyzing two aspects: first, the focus and diversity in students’ group work reflection and, second, the perceived successes and challenges of group work.

Focus and diversity of group work reflection

To analyze the focus of group work reflection, the aspects of describing and evaluating group work mentioned in the small groups’ responses were identified (RQ 2.1). When describing group work practices, the groups mentioned the division of work most often (79.2%). About one-third of the groups (34.7%) mentioned coordination, and about a quarter mentioned joint working (27.7%), co-construction (25.0%), and joint discussions (23.6%). Helping each other was mentioned least often (12.5%) when the groups described their group work practices. When evaluating group work, the student groups focused most often on the members’ contributions (79.2%). About half of the groups mentioned coordination (51.3%) or inquiry task success (50.0%), and a little less than a third mentioned collaboration (30.6%) or general success (27.8%) when evaluating their group work.

To analyze the diversity in group work reflection (RQ 2.2), the number of different aspects of group work identified in the responses was counted (M = 4.4; SD = 1.6), which led to distinguishing three levels of diversity. Table 6 presents the number of group work aspects mentioned in the responses, reflecting the groups’ ability to evaluate their group work.

Table 6. Diversity in reflecting on group work in different groups based on the number of aspects mentioned in the responses.

Diversity in reflecting on group work	Number of aspects of group work	f (groups)	%
Low	0–3	19	26.4
Moderate	4–5	38	52.8
High	6–9	15	20.8
Total		72	100

Note: The maximum number of different aspects of group work was 11.

A majority of groups (52.8%) mentioned four or five different aspects of group work in their responses, which implies a moderate ability to reflect on group work jointly. However, about one-fourth of the groups (26.4%) mentioned three aspects at most, even though the reflection questions already guided them by offering some viewpoints to consider. The groups with the most diverse reflections on group work (20.8%) mentioned six to nine aspects of group work in their responses.

To examine the qualitative differences between groups in their ability to reflect on group work, comparisons were made among the groups exhibiting high, moderate, and low diversity in reflecting on group work (Table 7). The skewness of the distribution for many of the individual aspect variables was high. Therefore, statistical tests were not performed, and comparisons were made based on frequency.

Table 7. Aspects of group work mentioned by groups with different levels of diversity in reflecting on group work.

Aspect of group work		Diversity in reflecting on group work
		Low (19 groups)		Moderate (38 groups)		High (15 groups)
		f	%	f	%	f	%
Aspects of describing group work	Division of work	10	52.6	32	84.2	15	100.0
	Coordination	3	15.8	13	34.2	9	60.0
	Joint working	3	15.8	12	31.6	5	33.3
	Co-construction	0	0.0	10	26.3	8	53.3
	Joint discussions	2	10.5	8	21.1	7	46.7
	Helping each other	0	0.0	2	5.3	7	46.7
Aspects of evaluating group work	Evaluating members’ contributions	11	57.9	32	84.2	14	93.3
	Evaluating coordination	7	36.8	18	47.4	12	80.0
	Evaluating inquiry task success	5	26.3	23	60.5	8	53.3
	Evaluating collaboration	2	10.5	12	31.6	8	53.3
	Evaluating general success	2	10.5	12	31.6	6	40.0

Most aspects were mentioned most often in the high-diversity groups and least often in the low-diversity groups, with the moderate groups falling in between. The moderate groups mentioned evaluating inquiry task success most often, and the high and moderate groups were very close to each other in terms of how often they mentioned joint working. Co-construction was mentioned by over half (53.3%) of the high groups and one-fourth (26.3%) of the moderate groups but none of the low groups. The aspect of helping each other was mentioned by almost half of the high groups (46.7%), but it was rarely mentioned by the moderate groups (5.3%) and not at all by the low groups.

Perceived successes and challenges in group work

The sections of the group work reflection responses that included the evaluation of group work were further analyzed to examine the types of successes and challenges the students experienced when completing the critical online inquiry task. The groups mentioned, on average, 2.7 successes or challenges, but five groups (6.9%) did not write any evaluative statements in their group work reflections.

In evaluating members’ contributions, the groups wrote mostly positively about group members’ commitment to the work (48.6% successes, 2.8% challenges); for example, “All members of the group completed their tasks responsibly and on time” (Group 56). Other mentions in this category related to members’ participation (37.0% successes, 5.6% challenges) and the distribution of workload (13.9% successes, 6.9% challenges).

In evaluating coordination, the groups mentioned the use of time (15.3%) and coordination of collaboration (2.8%) equally often as a success and a challenge. Other aspects mentioned under this category related to coordination in general (9.7% successes, 5.6% challenges), dividing tasks (8.3% successes, 5.6% challenges), and completing the tasks (2.8% successes, 4.2% challenges). The following is an example of a challenge in coordinating collaboration: “It was a bit challenging to get the opinion and perspective of all group members to be heard” (G 11).

In evaluating inquiry task success, the groups reported more challenges than successes in the following aspects: answering working document questions (8.3% successes, 18.1% challenges), selecting sources (2.8% successes, 4.2% challenges), evaluating sources (2.8% successes, 9.7% challenges), and making the synthesis (2.8% successes, 6.9% challenges). Other aspects of inquiry task success were related to searching for sources (12.5% successes, 6.9% challenges) and the seminar presentation (9.7% successes, 5.6% challenges). The following is an example of a challenge in answering working document questions: “In the task, it was difficult and challenging to find valid arguments” (G 52).

When evaluating collaboration, the groups described it mainly positively: working together (20.8% successes, 4.2% challenges), discussing (5.6% successes, 1.4% challenges), helping each other (2.8% successes), and atmosphere (2.8% successes). The successes achieved in working together were described by one group as follows: “We did the work well together all the time, so that one wrote while the other two dictated what to write” (G 64).

Evaluating general success was also described mainly in positive terms: overall success of the group in the inquiry task (15.3% successes), quality of working and outcomes (8.3% successes, 1.4% challenges), and the use of digital technology (1.4% successes, 1.4% challenges). Overall success was illustrated, for example, in the following way: “In our opinion, the project as a whole succeeded well” (G 53).

Associations between the variables (RQ3)

Chi-square tests revealed a significant association between the variables school and diversity in reflecting on group work (X² = 12.78; df = 6; p = .047). In addition, a weak correlation (Pearson’s r = .22, p = .061) was found between the variables learning critical online reading and reflecting group work. To clarify this association, the appearance of various aspects of group work in the groups’ responses was compared with that in groups with low, moderate, and high diversity in learning critical online reading. Chi-square tests showed statistically significant associations for coordination (X² = 6.22; df = 2; p = .045) and joint working (X² = 9.09; df = 2; p = .011). Coordination was mentioned by 21.6% of the low-diversity groups, 52.0% of the moderate-diversity groups, and 40.0% of the high-diversity groups, indicating a weaker ability of low-diversity groups to focus on coordinating their work. Furthermore, joint working was mentioned by 43.2% of the low-diversity groups, 12.0% of the moderate-diversity groups, and 10.0% of the high-diversity groups, which suggests that the groups scoring low on diversity in their reflections on group work performed the task by everyone contributing without explicit organization. Other correlations (Pearson’s r) between the independent and dependent variables were low, varying between $-$.13 to .17.

Discussion

In this study, we examined upper secondary school student groups’ self-evaluations of their group work practices and their learning of critical online reading during an online inquiry task. The administered learning task was challenging compared to ordinary upper secondary level tasks (Ilomäki, Lakkala, Muukkonen, Paavola, & Toom, 2023) because it was a large, open task that students had to solve together through joint discussions and decisions and that was not focused on a single subject but on more general phenomena. Students chose the topic based on their own interests, which was the basis for group formation. In general, the results suggest that neither the students’ learning of critical online reading nor their joint reflections on group work were very extensive.

Learning critical online reading

Although the online inquiry task included both lessons in skillful critical online reading and emphasized the evaluation of online sources, more than half of the student groups mentioned less than three aspects of critical online reading in their responses, indicating only a low diversity in learning (RQ1). The student groups did learn quite well to pay attention to different sources and the perspectives the sources represented, as well as to the different attributes of the authors when seeking reliable online information for the learning task. However, corroboration and evidence were mentioned least often as aspects of critical online reading. This result is understandable, as these aspects are cognitively highly demanding (Forzani, 2020; Kohnen & Mertens, 2019), but also somewhat alarming, as both corroboration and evidence are important aspects of skillful critical online reading (Hämäläinen et al., 2021; Nygren & Guath, 2022). The ability to verify the information in one source by comparing it to other sources (Britt et al., 2018; Kohnen & Mertens, 2019) and the ability to evaluate the evidence behind arguments are both highly needed to identify and tackle disinformation and flawed argumentation and to recognize fake news (Bronstein et al., 2019; Pennycook & Rand, 2019, 2021). Therefore, the online inquiry task should be further developed to more effectively incorporate corroboration and reasoning as information evaluation strategies. It seems that students need both more teacher guidance and specific prompts to support these cognitively demanding aspects of critical online reading.

Group work reflection

In this study, students completed the inquiry task in small groups and wrote a group self-evaluation about their group work, in addition to evaluating their learning of critical online reading. We examined the groups’ answers to understand how the student groups reflected on their group work: which aspects of group work they focused on and how diversely they described their ways of working (RQ2). By far, the most frequently mentioned aspects of group work were associated with the ways of dividing tasks or the workload between group members and with the members’ contributions to the work.

Learning groups often complete tasks by dividing them among members: each member completes their own part, and the outcomes are put together at the end. This type of cooperation is easier to organize than close collaboration between members, and it helps ensure fair division of the workload (McWhaw, Schnackenberg, Sclater, & Abrami, 2003). However, it does not provide the benefits of joint knowledge creation by merging members’ efforts (Ross, 2008). Based on our analyses, the groups that described their group practices in a more versatile way also mentioned more advanced collaboration practices, such as co-construction and helping each other. The inquiry task included both work phases that could be naturally divided (i.e., everyone examining their own information source) and phases that required joint development (i.e., the comparison of sources or conducting the synthesis), and some groups even managed to combine these strategies, as in Sormunen et al. (2014) study.

Student groups commented on member contributions mostly in positive terms, describing that everyone participated in joint work and, surprisingly, that there were no major commitment issues. According to previous research (Ilomäki, Vasileva, & Stefanova, 2020; Le, Janssen, & Wubbels, 2018), uneven participation between members in group work and the free rider effect are problems that students often mention when they are asked about group work experiences in educational settings. In the present case, the inquiry task was quite clearly structured through defined phases and included explicit guidelines, as well as a joint working document with prompts and group self-evaluation questions after each phase, which might have supported the coordination of group practices and helped avoid collaboration problems (Kiili et al., 2022; Lakkala & Ilomäki, 2015).

Group work strategies were not explicitly taught in the present study besides structuring the task; the focus was on teaching critical online reading. Our results showed that it was difficult for the groups to separate collaboration practices from the inquiry task: half of the groups mentioned successes and challenges in the inquiry process when they were asked to reflect on their group’s performance. To reflect on their group practices afterwards, students should be aware of group strategies and be able to explain them using the appropriate concepts. Receiving training in effective group work strategies before engaging in collaborative activities can promote the effective functioning of groups and students’ ability to evaluate the group process (Snyder, 2008). One possible topic for future research is to include the teaching of group work strategies in the pedagogical intervention and to examine how it affects student groups’ working process and self-evaluation.

Associations among variables

We also examined the associations among student groups’ evaluations of their learning of critical online reading and reflections of group work, and background variables (RQ3). Some interdependence seems to exist between the groups’ self-reported learning of critical online reading and their description of group work practices, which is in line with previous research indicating a relationship between collaborative methods and learning (Andrews & Rapp, 2015; Nokes-Malach et al., 2015; Van Leeuwen & Janssen, 2019). However, the identified correlation between learning critical online reading and reflecting on group work was weak, which suggests that these practices require different skills. This means that a student group’s proficiency in one skill may not translate into proficiency in another, warranting separate attention to each. The groups that received higher scores in learning critical online reading mentioned coordination, co-construction, and helping each other more frequently in their group work evaluations than the other groups. These strategies are considered advanced and effective in teamwork (Hoegl & Parboteeah, 2007; Sormunen et al., 2014). The results suggest that paying attention to the coordination of group work is related to learning, as observed in Barron’s Barron (2003) study.

We chose students’ group work as a pedagogical solution because previous studies (Kiili et al., 2019) have shown it to be effective for learning online inquiry skills. The present study confirmed this finding: the individual background variables were not associated with the level of the groups’ outcomes. Because the groups were formed based on students’ own interest in a topic, we may expect that this motivated students and influenced the groups because the teachers did not decide the group members. This might have taught students to collaborate with less familiar peers as well.

According to the analyses, there was a significant association between the school and the student groups’ ability to reflect on their group work. This result might indicate that in some schools, students have more opportunities to practice group work and develop their competencies in reflecting on their collaborative performance. However, a more plausible explanation for the observed difference is the variation in the teachers conducting the experiments in the classroom. While the teachers were provided with the same instructional materials for teaching critical online reading, the guidance of group work relied more on the teachers’ individual competencies. Although the task structures were designed to facilitate group work, the teacher guidelines did not explicitly involve methods to guide groups, such as explaining group work strategies, providing guidance to groups, or monitoring student engagement in evaluating their group work. This finding suggests that the ready-made instructional materials aided teachers in teaching critical online reading and that similar materials could have been beneficial for supporting students’ group work practices and reflection.

Limitations

This study has three main limitations. The first limitation is that the findings of the study are based solely on the student groups’ self-evaluations of the online inquiry process and their group work practices. Self-reports of learning are not only reflections of a group’s learning of critical online reading but also describe the participants’ metacognitive competence and their ability to articulate their understanding about their learning. One might criticize the use of self-reports for the lack of objectivity as a research method. To obtain a more comprehensive understanding of students’ learning and group work, more objective methods, such as valid tests to assess students’ learning of critical online reading or observations to assess group work, could have been used to strengthen the study.

The second limitation relates to the possible lack of motivation among students. The large variation in length in the student groups’ responses to the self-evaluation questions might indicate that the groups’ commitment to performing the entire task also varied. For this reason, it is possible that not all groups performed the self-evaluation task equally carefully. One reason for this may be that this kind of group reflection task was somewhat unfamiliar to the students. Teaching and practicing group work strategies were also not included in this study.

The study offered a well-designed approach to teaching critical online reading based on student groups’ self-reflections on their learning and working as a group. However, performing group-level research—instead of investigating individuals—is methodologically challenging, which can be considered the third limitation of the study. When the target of analysis consists of responses collaboratively written by groups, as in this study, it is difficult to determine the contributions of individual group members. Furthermore, as our analyses focused on students’ self-evaluations of their learning at the group level, we lacked information about the amount of variation in individual students’ perceived learning. Previous pre-posttest research on the same students as in this study (Hämäläinen et al., 2023) showed that students with initially low sourcing skills benefited the most from the teaching intervention. The same concerns may extend to the students’ perceived learning of critical online reading in this study as well. However, data on students’ interactions during their group work might have shed more light on the individual students’ performance.

Conclusion

This study revealed the ongoing need to teach students critical online reading and group work skills. In this study, the teachers applied the same instructional design to teach online inquiry, which helped avoid any significant teacher-specific differences. The design also proved to be a workable method for examining the diversity of critical online reading. However, although the students worked in groups, they did not extensively discuss the most cognitively demanding aspects of critical online reading, such as corroboration and evidence. To further develop the design, the instruction and practice of group work strategies can be added to ensure the positive effect of group work on learning. Furthermore, schools should encourage the use of assignments that embed collaborative and reflective working practices and argumentative discussions to stimulate high-quality cognitive interactions among students.

In this study, online inquiry was examined and practiced through a teaching intervention in which all plans, materials, task assignments, group work organization, working documents, reflection tasks, and the overall study design were carefully and systematically planned in collaboration with teachers. This kind of systematic collaboration between teachers and researchers in planning teaching arrangements can be regarded as somewhat exceptional, and all participating teachers greatly appreciated it. Although we did not achieve strong and convincing results on students’ learning, this does not negate the importance of the intervention. Rather, it emphasizes the ongoing need for the development of the teaching and learning of online inquiry, which should be further practiced in the future. Nonetheless, the study showed that a single project aimed at cultivating online inquiry skills is insufficient and that such complex skills can be acquired only through long-term, many-sided, and repeated practice in a variety of ways throughout school.

Acknowledgments

The authors would like to thank the students and the five teachers who participated in the study, Justiina Pihlajamaa for her help in the data analysis, and Carita Kiili, Julie Coiro, Auli Toom, Elina Hämäläinen, Eero Sormunen, Jukka Utriainen, Hanni Muukkonen, Tuulikki Alamettälä, and Jannica Heinström, who participated in planning and implementing the intervention.

Disclosure statement

The authors report that there are no competing interests to declare.

Additional information

Funding

This research was funded by the Academy of Finland (Grant numbers 285817 and 285806).