A Review of the Use of Investigative Projects in Statistics and Data Science Courses

Abstract

An investigative project can engage the learner in all aspects of a statistical investigation, including developing a question or issue of interest, gathering needed information, exploring the data, and communicating the results. This article summarizes the available literature regarding the implementation of investigative projects, including the potential benefits of projects, how projects have been used in a variety of settings, advice for those looking to implement projects, and future research avenues.

KEYWORDS:

• Active learning
• Literature review
• Statistical analysis investigative cycle
• Statistics education

1 Introduction and Purpose

Statistics educators have been writing about implementing investigative projects in their classes since at least 1976 (Griffiths and Evans 1976; Scott 1976). However, the floodgates were opened in the early 1990s with numerous calls to change the traditional way of teaching statistics (e.g., Hogg 1991; Cobb 1992; Snee 1993). Educators were encouraged to place emphasis on asking appropriate questions, collecting data, and exploring and interpreting data.

In response to these and many subsequent calls for change, the American Statistical Association (ASA) endorsed the Guidelines for Assessment and Instruction in Statistics Education (GAISE) College Report (2016) and the GAISE II (Bargagliotti et al. 2020) report. Both reports recommend implementing investigative projects. From a comprehensive survey of the available literature regarding the use of investigative projects within a classroom setting, this article draws out important educationally-beneficial aspects of using investigative projects in statistics and data science courses. Although over 250 peer-reviewed articles, book chapters, and International Association for Statistical Education (IASE) conference proceedings were reviewed for this article, it is noted that many instructors are implementing projects but have not written about their experience in such settings. Thus, this article may not be representative of all classroom projects being implemented, nor does it include projects outside the classroom setting (e.g., REUs, internships, and project competitions).

The projects discussed throughout this article involve students choosing a question or issue to explore, collecting and considering appropriate data, analyzing the data, and drawing conclusions in context (Bargagliotti et al. 2020). For many decades, statisticians and statistics educators have referred to this process as the “statistical investigation process” and similar terms (GAISE College Report 2016; Lee et al. 2022) with reference to the practice of statistics (Kenett and Thyregod 2006).

Despite the variety of ways projects have been implemented and the different labels given to the assignment, projects are always a way for students to experience the entire cycle of a statistical investigation process. Henceforth in this article, the term “project” refers to investigative projects, which incorporate the essential characteristics of statistical data investigations with at least some student-driven contributions.

This article summarizes potential benefits of projects in Section 2 and provides citations for the interested reader to find example projects implemented in specific contexts in Section 3. Section 4 reviews the findings of several case studies investigating the impact of projects on student attitudes toward statistics and performance on select statistical concepts. Section 5 provides advice collected from the literature and Section 6 describes areas for future work.

2 Potential Benefits of Projects

The literature includes many potential benefits of projects which can serve as valuable learning opportunities for both the student and the instructor (Biajone 2006; Forster and MacGillivray 2010). Projects can offer elements of student choice, which can be particularly motivating to a student and interesting to the instructor. Projects can provide students with application opportunities to deepen understanding of statistical concepts, arguably above and beyond other means of assessment (Yesilcay 2000). The skills gained through participating in projects can be marketable in future employment pursuits. The following subsections expand further on the potential benefits of projects in these areas.

2.1 Students Practice the Entire Cycle of a Statistical Investigation Process

Projects allow students to be involved with and practice the entire cycle of a statistical investigation. There is universal acceptance among statistics educators of the value of having students participate in the entire data investigation cycle (Forster and MacGillivray 2010), with some asserting that projects serve as the best way of introducing students to the entire process of statistical inquiry (Yesilcay 2000; Chance 2002). In particular, projects help students model the way statisticians work and recognize how data and statistics are used to investigate questions (Garfield and Ben-Zvi 2008). George Cobb (2015, p. 277) draws the following similarity between statistics and other disciplines:

In the Humanities, students in a first course engage with original sources. You don’t just prepare your students to read Austen; they read Austen. You do not just prepare students to hear Bach; they hear Bach. Our statistics curriculum should follow those examples: Our job is not to just prepare students to use data to answer a question that matters; our job is to help them use data to answer a question that matters. In short, teach through research.

Projects can be referred to as “wicked” problems that do not always have a clear solution (Rittel and Webber 1973). Students apply knowledge and make decisions that are not explicitly laid out by an instructor. Each stage of the statistical investigation allows students to apply and develop different skills. The following subsections outline potential benefits of specific stages of an investigative project (Lee et al. 2022):

• Frame a problem

• Consider and gather data

• Explore and visualize data, and consider models

• Communicate and propose possible actions

2.1.1 Student-Selected Projects Can Provide Motivation and Interest

The first phase in the data investigation process is to frame a problem. This can be completely determined by the instructor; however, students may find it more interesting and motivating if they exercise choice in posing investigative questions within the project framework. When students are allowed to select their own project topic, they take an active role in shaping their learning experience. Students can choose a topic of particular interest to them, and are motivated to see how statistics can be applied to real world problems (Lee Lee n.d.; Sylwester and Mee 1993). Cobb (2015, p. 277) asserts that “nothing motivates students like choosing their own questions and being the first to offer an answer.”

Not only do students find their own self-selected projects interesting (Sevin 1995; Terán 2020; Lee Lee n.d.), they also enjoy other student projects (Wardrop 1999). Wardrop (“A New Approach to Introductory Statistics”, n.d., p. 5) states that “students know better than professors what interests them. Hence, they find projects performed by other students to be more interesting than many of the examples selected by the instructor.”

Student motivation and interest benefit more than just the student. Instructors have also expressed enjoyment from learning from students (e.g., Fillebrown 1994; Wardrop 1999; Forster and MacGillivray 2010). Specifically, Fillebrown (1994) states that even though it was more work than other tasks, she enjoyed the process, found she looked forward to reading the reports, and was genuinely interested in the students’ results.

2.1.2 Students Can Experience the Complexities of considering and Collecting Data

After framing a problem, one must consider and gather data. Statisticians benefit by taking part in the data collection process in order to fully understand the context of the data and properly respond to the client (Kenett and Thyregod 2006). Similarly, having students collect their own data can expose them to some of the principles of experimental design, some of the practical struggles of collecting data, and some incidental decisions needed for data gathering (Gelman and Nolan 2017). Students are able to see first-hand how difficult it can be to obtain good data and to discover the importance of inspecting and cleaning data before further analysis is done (Sole and Weinberg 2017). Further, by developing a data collection plan, students come to understand that statistics starts even before data have been delivered (Ledolter 1995).

Some of the challenges with already-collected data involve a lack of explanatory information provided by data sources, which can be frustrating to students (Mackisack 1994). Some datasets may involve issues beyond the immediate grasp of undergraduates. Although using real data from research journals is beneficial, “students can almost never know enough about the subject matter to know what the answers mean” (Mackisack 1994, p. 6). If students have knowledge of the subject matter, they have a better understanding of the sources of variation affecting the response as well as a better understanding of the data itself. Some instructors may dismiss the problems of data collection in simple contexts as technically trivial, “but if the students are interested in, understand, and learn from the context, its lack of importance to the instructor seems irrelevant” (Mackisack 1994, p. 6).

In contrast, by sourcing existing datasets, students can be exposed to much larger datasets with added complexities. With these larger datasets, students have an opportunity to explore more complex analysis concepts. Additionally, sourced data may be less time intensive than manually collecting data and less computationally intensive than scraping data from online sources. Even when students are sourcing existing datasets, they can still gain exposure to evaluating the quality of data, exploring data for unusual values, and gathering additional data as needed (Gould 2021). Sourced data still needs to be interrogated Bargagliotti et al. (2020) as the process for which data were obtained is often hidden (Rubin 2021). Rubin (2021) describes having students take part in data journalism, asking about the data production process using question words (e.g., who, when, where, why, how) and inquiring “can we measure what we want to measure?” (p. S24).

It is important, regardless of whether students are collecting or sourcing data, that they have a clear understanding of where the data come from, in order to understand the scope of the conclusions that can be drawn from these data.

2.1.3 Students Gain a Deeper Understanding of Material by Deciding Appropriate Ways to Explore and Analyze Data

Students learn best if they are allowed to identify appropriate means of exploring and analyzing data, as opposed to some lab settings where the analysis is presented to them (Guyot, Walker, and White 2018; Lee Lee n.d.). This construction of material, rather than the memorization of statistical concepts, promotes learning (Silva and Pinto 2014) as it bridges the gap between abstract thought and practical application (Ghinis, Chadjipantelis, and Bersimis 2005).

When students are given the opportunity to decide appropriate ways to explore their data, they model the way statisticians work. They come to recognize how data and statistics are used to investigate questions (Garfield and Ben-Zvi 2008), are exposed to the messy side of data analysis (Groth 2006), and learn that statistics is more than just summarizing numbers (Ledolter 1995). It is possible for students to investigate more complex questions with projects than is possible in a single afternoon lab session (Cobb 1993).

The application of statistical concepts to student projects reinforces key concepts and deepens students’ understanding of statistical techniques (Forster and MacGillivray 2010).

2.1.4 Students Communicate and Reflect on Statistical Findings

Projects can provide students opportunities to communicate both orally and in writing to a general audience (Mackisack 1994; Kettenring 1995; Ledolter 1995; Bryce et al. 2001; Ritter, Starbuck, and Hogg 2001; MacGillivray and Pereira-Mendoza 2011; GAISE College Report 2016; Guyot, Walker, and White 2018; Lasser et al. 2021). By expressing their ideas both orally and in writing, students become more involved in their own learning (Garfield 1995). There is a need for improved training of statistics students in both written and oral communication skills to promote effective interaction with stakeholder audiences (CATS 1994). Halvorsen and Moore (2001, p. 27) state that “few students receive instruction specifically focused on writing about numbers, and the statistics research project offers a unique opportunity to do this.”

In the communication phase, students craft a data story to convey insights, justify claims with evidence from data, and address uncertainty and potential bias (Lee et al. 2022). Students can also reflect on the various phases of their project, providing a chance to critically think about what worked well and what could be improved upon. This process aims to promote links between statistical topics and a reflection on statistical concepts in practice (Silva and Pinto 2014). Communicating both orally and in writing can be classified as a metacognitive activity and may provide ways for students to develop conceptual understanding (Lipson and Kokonis 2005).

2.2 Students Learn Skills Applicable and Marketable to Future Careers

Investigative projects can grant students an opportunity to practice skills attractive to employers no matter the eventual career (Anderson and Loynes 1987). Projects encourage creative thinking (Griffiths and Evans 1976), build confidence (Sylwester and Mee 1993), and strengthen students’ ability to become life-long learners (Anderson and Loynes 1987). Examples of marketable career skills that can be developed through such projects include:

• Working effectively in teams (Joiner 1985; Mackisack 1994; Bryce et al. 2001; Halvorsen and Moore 2001; Ritter, Starbuck, and Hogg 2001; Guyot, Walker, and White 2018)

• Developing problem solving and decision making skills (Mackisack 1994; Ritter, Starbuck, and Hogg 2001; Forster and MacGillivray 2010)

• Meeting multiple deadlines (Guyot, Walker, and White 2018)

• Developing oral, written, and graphical communication skills (Ritter, Starbuck, and Hogg 2001)

Sisto (2009) cites students receiving recognition from outside departments proclaiming how prepared students were for project work in their courses. Some students have attributed their success in obtaining jobs directly to projects completed in their coursework (Kaus 2008).

2.3 Students Can See the Value of Statistics

The literature demonstrates that completing projects allows students to discover the value of statistics (Snee 1993). Kettenring (in Moore 1997, p. 153) observes that students build “an appreciation for what it means to manage data: how to collect it, how to sample it, how to view it, how to model it, how to draw inferences, how to assess uncertainty, and how to integrate statistical analyses into the larger context of the business problem.” In addition to accounts of students seeing the value of statistics through investigative projects, several instructors recall students’ identifying projects as the most useful or valuable learning experiences of the course (e.g., Cobb 1992; Vardeman 1996; Halvorsen and Moore 2001; Chance 2005). Even when students began the semester with negative attitudes toward statistics, their views dramatically changed to positive ones after completing projects and understanding how statistical concepts relate to their careers (Biajone 2006).

Mackisack (1994) describes implementing projects in a course for second year undergraduate students specializing in mathematics, noting high marks on evaluations with many students continuing into the third year of their institution’s statistics program. Graduate students who completed projects through the CAPSULES program appreciated the real-world applications of statistics and found the course helpful in preparing for their thesis or dissertation research (Thompson 2009). Through the use of student projects, student attitudes can transform and lead to a student population that looks favorably on statistics as a field (Bingham 2010).

2.4 Students Are Afforded Authentic Assessment Opportunities

Projects can be used as an authentic assessment tool for assessing student’s understanding of statistical concepts (Zeleke 2006). A few authors discuss the benefits of projects over exams specifically. For example, some argue that a test is designed to measure learning after it has occurred, and is not a learning experience in itself. Projects on the other hand, span a much larger portion of the course, and are not completed in a single sitting as exams typically are (Cobb 1993). Students appreciate being able to take their time and not feel as rushed on the projects compared to in-class assignments and tests (Bingham 2010). Projects have also been shown to decrease test anxieties (Zieffler et al. 2008).

Projects provide opportunities for cooperation between teacher and student, as opposed to exams which are more adversarial (Roberts 1992). The opportunity for close student and instructor interaction can inspire a great deal of growth for both parties (Vardeman 1996). Instructors are afforded a much better understanding of what students comprehend than is revealed by traditional exams. Further, students find doing projects more interesting than other assessment tasks (e.g., Anderson and Loynes 1987; Lee 2005) and have been found to remember projects months after the conclusion of a course (Bingham 2010).

Students are not the only ones to find projects more enjoyable than exams. Roberts (1992) states that for faculty, grading projects is much more interesting than grading exams and that students rarely argue about their grade on projects (whereas this is more common with exams). Roberts argues that students “live up to their potential” (p. 114) on projects, and felt so strongly about the benefits of projects that he replaced all exams with projects in his courses.

3 Project Implementation Logistics

The following subsections outline some of the contexts for which projects have been implemented as well as how projects have been structured. The tables included in each subsection provide references to articles so that educators can find examples similar to their own class experiences. These lists are not exhaustive, but do span a variety of settings and contexts for the interested reader.

3.1 Range of Class Audiences

Projects have been successfully implemented with students as young as 6 years (Du Feu 2005)! In this elementary school classroom, students collected flowering data from local parks and created data visualizations using Legos. As the literature reveals, educators have found ways of implementing projects with students at a variety of education levels, spanning from early elementary school through graduate school. Examples of such projects and the corresponding education level are summarized in Table 1.

Table 1 Selected articles describing investigative projects conducted in pre-college Classrooms.

Audience level	Source(s)
Elementary School	Du Feu (2005)
	Metz (2004)
	Cobb (2015)
	Paparistodemou and Meletiou-Mavrotheris (2008)
Middle School	Ghinis, Chadjipantelis, and Bersimis (2005) Santos and César (2006) Lavigne and Lajoie (2002)
Pre-college students in Ireland	Boland (1998)
High School	Dolan (1979)
Advanced Placement Statistics Class	Groth and Powell (2004)

A number of authors discuss the use of projects in a variety of college and graduate level courses. Examples are highlighted in Table 2. It is noted that the elementary statistics class referenced in Table 2 differs from the other audiences in Table 2 in that no statistical software package is used nor are students introduced to hypothesis testing.

Table 2 Selected articles describing investigative projects conducted at the college and graduate level.

Audience level	Source(s)
Elementary Statistics Class	Fillebrown (1994)
Introductory Statistics or Data Science Class	Baki and Baki (2014) Dierker et al. (2012) Halvorsen (2010) Guyot, Walker, and White (2018) Kaus (2008) Sylwester and Mee (1993) Wardrop (1999)
Community College	Brown (2006)
Second or Third Courses in a Statistical Sequence	Ledolter (1995) Mackisack (1994) Periasamy (2006) Shemyakin and Tiefenbruck (2009)
Design of Experiments	Hunter (1977)
Probability Courses	MacGillivray (2002) Sisto (2009)
Senior Capstone	Blaconá (2006) Griffiths and Evans (1976) Mackisack and Petocz (2002) and Martonosi and Williams (2016) Yesilcay (2000)
Graduate School CAPSULES Program	Thompson (2009)
Graduate School MBA program	Love (1998) Roberts (1992) Sisto (2009) Smith (2011)

Although the literature contains examples from a variety of student audiences, the most diverse audience may be described by Sisto (2009), using projects at a business school in Europe. The class typically contains 10–20 different student nationalities within the same class. At the time the article was published, the author describes having 40 different nationalities. At this school, English is the second language for most of the student body, and the author describes a restriction that no more than two same-language speakers may be in the same group for the projects conducted. Sisto reports that student feedback was overwhelmingly positive, that students now use statistical reasoning and analysis in their thesis before graduation, and that client disciplines have reported students being better prepared for project work in their courses. This is a strong indication of the ability for statistics projects to flourish in a diverse environment.

3.2 Projects in Large Class Sizes

Although it may be intimidating to consider the added workload of conducting projects in large classes, there are several examples of instructors of larger classes implementing projects, and having positive results. Specifically, Ledolter (1995, p. 365) claims that conducting projects in a 600-student class was “just as positive [as it was with the smaller class], confirming the fact that projects are useful at any level.” Table 3 summarizes examples of projects implemented where the class size was specified to be greater than 100 students per course.

Table 3 Selected articles describing projects implemented in large class settings.

Class size	Discipline	Source(s)
120	Engineering	Bryce (1993)
200	Engineering	MacGillivray (2002)
200	Business	Sylwester and Mee (1993)
250	Business	Zahn (1993)
250+	None specified	Ledolter (1995)
400+	Engineering	Forster and MacGillivray (2010)
600	None specified	Ledolter (1995)

3.3 Project Group Sizes

To manage the workload of both instructor and student, many instructors have opted to conduct projects using student groups. Using groups has the added benefit of allowing students to practice working as a team, a skill attractive to many educators and employers. Table 4 highlights some examples of specific group sizes used.

Table 4 Selected articles specifying group size, number of projects conducted per course, and corresponding class size.

Group	Projects	Class
size	per course	size	Source(s)
2–3	1	40–50	Halvorsen (2010)
	1	25–45	Mackisack (1994)
	2	24	Dolan (1979)
	2	10–25	Sisto (2009)
	6	30	Smith (1998)
2 – 4	1	100+	Bryce (1993)
	1	32	Kaus (2008)
3	1	∼300	Terán (2020)
	1		Yesilcay (2000)
3–4	1	large	Baki and Baki (2014)
	1	400	MacGillivray (1998)
3–5	1	∼35	Biajone (2006)
		small	Chance (1997)
4–5	1	15–25	Shemyakin and Tiefenbruck (2009)
5	1	∼300	Ledolter (1995)
	5		Davidson, Dewey, and Fleming (2019)

3.4 Collecting and Sourcing Data Strategies

As reviewed in Section 2.1.2, students benefit from taking an active role in collecting and sourcing appropriate data. The following sections provide a variety of examples of data collection and sourcing methods used throughout the literature.

3.4.1 Dataset Provided

Several authors have opted to provide datasets for student projects. When datasets are provided, students are saved time in the data collection process. Additionally, students may be exposed to more complex datasets than may easily be collected or scraped from online sources. By providing datasets, instructors can also ensure that the dataset and intended analysis for the project are an appropriate match.

Similarly, there are several examples of community-based projects where the partner client provides datasets to the students. In these examples, students get hands-on experience working with clients who are well-placed to add context and connection to the datasets. Table 5 provides some examples of projects where datasets were provided by the instructor or partnering client.

Table 5 Selected articles for which datasets are provided by the instructor or client.

Dataset provider	Source(s)
Instructor	Albert (2000) Biehler (2007) Boehm and Hanlon (2021) Dierker et al. (2012) Frischemeier (2020) Holcomb and Ruffer (2000) Terán (2020) Witmer (1992)
Client organization	Anderson and Sungur (1999) Lasser et al. (2021) Mackisack and Petocz (2002) Thompson (2009)

3.4.2 Observational Data Collection Methods

Students can collect data from the world around them through observation at parks, stores, and coffee shops. By collecting information in this way, students take an active role in the data collection process, and have a keen understanding of where the data came from. Table 6 summarizes some studies using observational data collection methods, several at the pre-college level.

Table 6 Selected articles for which data are collected by students from Physical spaces.

Audience	Data collection
level	location	Source(s)
Elementary school	Local park	Du Feu (2005)
	Themselves	Paparistodemou and Meletiou-Mavrotheris (2008)
High school	Local stores	Dolan (1979)
College	Daily routine	Chadjipadelis and Andreadis (2006)
	Local coffee shops	Sole and Weinberg (2017)

3.4.3 Online Data Collection

Data can be collected from online sources (e.g., likes on a social media post, movie ratings, career sports statistics). As the amount of information available online is vast, students have a great deal of flexibility in topic choice. Students can take an active role in the data planning stage, considering variables of interest, and methods of collecting a representative sample. Students can consider the bias of using more readily available Top 100 Lists as well as creative random generators (e.g., Random Lists 2013). When students take an active role in the data planning and collection process, they can better check for errors in the dataset and investigate outliers. Table 7 lists several authors that have encouraged students to collect data from online sources.

Table 7 Selected articles for which data are collected by students from online sources.

Level of students	Source(s)
Middle school	Lavigne and Lajoie (2002)
High school	Groth and Powell (2004)
Elementary statistics college course	Field (1985) Fillebrown (1994)
Introductory statistics college course	Halvorsen (2010) Sevin (1995) Smith (1998)

3.4.4 Survey Data Collection

Collecting data via survey is a way of collecting opinions and data from people. Students can be afforded a great deal of customization in the data they collect and practice the art of wording questions. Instructors and students can discuss difficulties of dealing with voluntary response, nonresponse bias, and missing data (Cobb 1992). Topics such as data ethics, informed consent, confidentiality, and their institution’s research ethics committee should be emphasized (Cobb 1992). Some examples which include use of a survey to collect data include a second course in college statistics in which students use surveys and phone interviews of local businesses and social networks (Ledolter 1995) and a collective class survey (Biajone 2006; Lovekamp, Soboroff, and Gillespie 2017).

3.4.5 Experiments

Several authors encourage students to design and perform experiments to collect data. In these instances, students get practice in experimental design and can investigate possible causal connections. Experiments can be a way to introduce randomness in the data collection process, which can allow students to practice making distinctions between random sampling and random assignment, specifically in terms of the effect on the conclusions that can be made. Several authors have listed past experiments or possible experiment topics. Table 8 lists several authors that have used experiments as a way for students to collect data, notes on the corresponding audience, and whether the article included a list of potential experiment topics.

Table 8 Selected articles for which data are collected from experiments.

	Included list
Audience	of topics	Source(s)
Engineering statistics course		Bryce (1993)
Users of The Island Project	Yes	Bulmer and Haladyn (2011)
Design of experiments course	Yes	Hunter (1977)
Engineering statistics course		MacGillivray (2002)
Second year math majors	Yes	Mackisack (1994)
Introductory statistics course	Yes	Wardrop (1999)

3.4.6 Simulated Data

Some authors describe students using simulated data for student projects. For example, Bulmer and Haladyn (2011) developed a simulated population they call The Island, where information about the residents is simulated based on past literature. Other instructors have students take advantage of The Island population to perform simulated experiments (Baglin, Bedford, and Bulmer 2013). Darius, Portier, and Schrevens (2007) describe having students run two java applets to collect data. In one applet, students collect data from an industrial plant while in anther applet, students collect data from greenhouse tomato plant growth. Davidson, Dewey, and Fleming (2019) encourage students to record their simulated data with various errors, arguing that inconsistencies and missing values make a dataset more realistic.

3.5 Amount of Time Students Are Given to Complete Projects

Because projects involve students’ performing an entire investigative cycle, many will span more than a single class period. The amount of time spent will depend greatly on the scope of the project. Generally speaking, younger audiences had shorter project durations while advanced college-level projects could take an entire semester or even a year. Table 9 summarizes examples of projects conducted at the pre-college level, and Table 10 highlights examples of projects conducted at the college level.

Table 9 Selected articles summarizing project duration with projects conducted at the pre-college level.

Project duration	Audience level	Source(s)
A few class periods	Elementary school	Du Feu (2005)
3–4 days	Middle school	Lavigne and Lajoie (2002)
2 weeks	High school	Landrum and Smith (2007)

Table 10 Selected articles summarizing project duration with projects conducted at the college level.

Project duration	Source(s)
2 weeks	Smith (1998)
3 weeks	Davidson, Dewey, and Fleming (2019) Hunter (1977)
5 weeks	Mackisack (1994)
1 semester	Anderson and Sungur (1999) Baki and Baki (2014) Biajone (2006) Bryce (1993) Chance (1997) Fillebrown (1994) Guyot, Walker, and White (2018) Halvorsen (2010) Kaus (2008) MacGillivray (1998) Roberts (1992) Sevin (1995) Terán (2020) Zahn (1993)
1 Year	Boehm and Hanlon (2021) Griffiths and Evans (1976) Mackisack and Petocz (2002) Yesilcay (2000)

3.6 Assigning Multiple Projects per Course

Although a single project can afford many benefits to students, there are advantages to having students complete more than one project in a given course (Anderson and Loynes 1987; Moore, Moore n.d.; Sisto 2007). If students can explore multiple topics or datasets, they have an opportunity to apply their knowledge in a variety of contexts, encounter different ways of obtaining data, and practice applying different exploratory methods.

There were several instances where instructors implemented more than one project per semester. In particular, Davidson, Dewey, and Fleming (2019) describes a year-long, two-course sequence of biostatistics courses where students conducted a total of five projects. Each project was completed within 3 weeks and students simulated data based on literature reviews. Smith (1998) conducts six projects, each lasting 2 weeks, at the college level. Students work in groups of 2–3, choose a topic of interest, develop a data collection plan to collect, and submit a final report. Bingham (2010) describes conducting 10 smaller projects within a course, and although topics are given to students, they develop their own data collection plan, collect data, and decide how they will go about producing results. Table 11 lists examples where authors specify having students conduct more than one project per course as well as the corresponding audience level.

Table 11 Selected articles where authors specify implementing more than one project per course.

Number of projects	Audience level	Source(s)
2	High school	Dolan (1979) Groth and Powell (2004)
	College	Spence, Bailey, and Sharp (2017) Sylwester and Mee (1993) Wardrop (1999)
3	Middle school	Lavigne and Lajoie (2002)
	College	Campos et al. (2010) Sisto (2007)
5	College	Davidson, Dewey, and Fleming (2019)
6	College	Smith (1998)
10 (smaller)	College	Bingham (2010)

Not all of the examples listed are conducted at the college level. There are cases at both the middle school (e.g., Lavigne and Lajoie 2002) and high school levels as well (e.g., Dolan 1979; Groth and Powell 2004).

3.7 Project Scaffolding and Resubmission Opportunities

Several authors recommend having organized stages of a project with intermediate due dates to help scaffold the project (Moore Moore n.d.; Sevin 1995; Sole and Weinberg 2017; Sylwester and Mee 1993). Students appreciate the additional opportunities for feedback on their progress (Sylwester and Mee 1993; Fillebrown 1994). These opportunities for feedback can be a way for early instructor intervention (Smith 2011), and can lead to meaningful mentoring (Vardeman 1996).

Some authors allow students to resubmit their initial proposal (Guyot, Walker, and White 2018; Terán 2020). Although this portion of the project is not always graded, it does provide an opportunity for instructor feedback before students begin data collection. Similarly, Short and Pigeon (1998) describe having students perform pilot studies as a small-scale rehearsal for a larger main study in order to learn more about the data acquisition process without investing large amounts of time and resources.

Roberts (1992) uses a “redo” grade for any students who would receive a “low” grade on the portion of the project, allowing students to resubmit their work. Ledolter (1995) allows students to resubmit their final report. Mackisack (1994) allows students to integrate feedback from their presentation, which is ungraded, and apply these changes to their final report which is graded. Table 12 summarizes how some instructors have segmented projects into a number of smaller assignments.

Table 12 Selected articles where authors specify number of sub-assignments of a project.

Number of sub-assignments	Source(s)
1	Hunter (1977) Ledolter (1995) McKenzie (1993) Smith (1998)
3	Roberts (1992) Sisto (2009) Sole and Weinberg (2017) Terán (2020) Zahn (1993)
4	Root and Thorme (2001)
5	Chance (1997)
6 (only 1 graded)	Sevin (1995)
6	Sylwester and Mee (1993)
11	Bryce (1993)

4 Studies Exploring the Impact of Projects

Much of the literature on the implementation of projects and their positive impact consists of case studies, personal anecdotes, and pedagogical justifications. For example, some authors note that projects played a significant role in predicting exam scores (Forster and MacGillivray 2010). Others state more specifically that student exam scores increased after implementing projects (Smith 1998). Bingham (2010) surveyed students four months after the course ended, and not only did students remember the projects, but students felt they understood the material, the material had relevance to their lives, and the students found the material interesting and fun!

There are also controlled studies that explore the impact of projects and specific student outcomes. This section summarizes six controlled studies exploring the impact of projects on student attitudes toward the field of statistics and on student statistical content knowledge.

Ghinis, Chadjipantelis, and Bersimis (2005) describe a longitudinal study with elementary children with ages ranging from 10 to 12 years old. The students completed projects including data collection, analysis of results, preparing written reports, and giving oral presentations. Three years later, the same students were assessed on statistical interpretation abilities. These students were then compared to a control group of students that were taught using “conventional methods.” Minimal details were provided regarding how the control group was assembled. The authors asked both groups of students a set of 14 questions to assess their statistical content knowledge. The authors concluded that students in the experimental group scored higher in statistical abilities than those in the control group, with significantly higher average scores in 5 of the 14 questions. The questions where significantly higher average scores were found included interpreting histograms, interpreting pie charts, creating plots, understanding means, and understanding percentages.

Spence, Bailey, and Sharp (2017) describe a study with introductory college students over multiple semesters, (although not with the same groups of students). Their study explored both student attitudes toward statistics and student content knowledge. The study included eight instructors that did not use projects in their courses and were selected from institutions across the United States, invited via email to participate in the study. The instructors were found via contacts and statistical networks of the authors. The eight instructors first taught a semester of their typical course (to serve as a control). They then attended a workshop to learn how to implement projects in their courses. Following the workshop, they taught the same course as they did pre-workshop, with the addition of projects. Due to various circumstances, not all instructors were able to complete both courses. Students’ content knowledge, as well as attitudes, were surveyed in each of the courses. Although the average scores of students’ content knowledge was statistically higher in the course where projects were implemented, the difference was viewed by the authors as minimal. Similarly, student belief in their ability to understand and use statistics was significantly higher in the course where projects were implemented, but again, the difference was viewed as rather minimal by the authors.

Silva and Pinto (2014) used projects in the second half of a course. On the end-of-the-year evaluations, students’ views about learning were collected via closed form and open ended questions. The closed-form questions asked students to rate the difficulty of each portion of the project on a scale from 0 (very difficult) to 10 (very easy). Authors found that presentation of data summaries, choice of topic/theme, and working in groups accounted for 71% of the variability in overall performance of the project.

Ramirez and Bond (2014) describe two courses similar in content and structure with a project requirement in one course, while the other had an online homework component. The authors used the Survey of Attitudes Toward Statistics – 36 (Schau et al. 1995) to measure students’ attitudes before and after each course. The authors found that cognitive competence in the project-based course dropped about .4 points between the pre and post tests, while the hybrid course had a negligible increase. In both courses it appeared as though attitudes had gotten worse over the course of the semester, and students from the project-based course had the worse mean change score for Interest in the subject at –1.13. The authors conclude that perhaps it is not enough to merely conduct one project per semester for projects to have a positive impact on attitudes.

Carnell (2008) also implemented the Survey of Attitudes Toward Statistics–36 in two introductory classes where one class completed a project and the other did not. The survey was administered at the beginning and end of the courses. The students that completed a project did not exhibit more positive responses on the six subscales than the students that did not complete a project. The author also concludes that perhaps a single project was not enough to positively impact attitudes.

Chadjipadelis and Andreadis (2006) used projects in an introductory statistics course at the Department of Political Science of Aristotle, University of Thessaloniki. Similar to the previous two studies, the authors used the Survey of Attitudes Toward Statistics to measure the influence of teaching statistics with projects on the student’s attitudes toward statistics. Authors found that those with lower mathematical background had a significant difference in how easy they felt it was to learn statistics; the students in the project group, feeling statistics was easier to learn than students in the control group. However, there was not as large a difference between the project and control groups regarding their feelings of how difficult it was to learn statistics for those students that had higher mathematical abilities. The authors conclude from the case study that students in the project group have more positive feelings toward statistics, scored better on cognitive competence, showed greater interest in statistics, and tried harder to learn statistics.

5 Tips and Advice for Implementing Projects

For those inspired to implement projects (whether for the first time or to improve a well-established process), the following sections are a collection of advice from the literature.

5.1 Consider Adjusting Class Logistics

Instructors should be aware that in order to implement projects within some courses, changes might need to be made. Adjustments may include a rebalancing of workload for both faculty and students, or a resequencing of topics to align with project scaffolding and goals.

Regarding the workload, MacGillivray (2010) acknowledges that the time commitment for both student and instructor is a valid concern, therefore, care must be taken in constructing and managing student and staff support, particularly in large classes. Smith (1998) describes cutting back on weekly exercises (going from 10 to 5), and removing a mid-semester term paper in order to implement six projects per semester.

Examples of resequencing material include Fruiht (2018) mentioning the resequencing of a project-based course, emphasizing not only tools and topics, but application. The resequencing of topics introduces statistical concepts to students “in a way more in line with the process followed by scientists” (Fruiht 2018, p. 1). The resequencing allows statistical inference to be introduced early in the semester, and revisited in a variety of contexts throughout the semester. Similarly, Chance and Rossman (2001) also consider resequencing topics to mirror the practice of statistics and allow students to begin their own data collection early in the course.

5.2 Leverage Groups and Peer Feedback to Minimize Workload

In order to minimize workload of projects, instructors can use group projects instead of individual projects, leverage peer review to provide feedback (Forster and MacGillivray 2010), and use discussion sections for student presentations (GAISE 2016). Additionally, as outlined in Section 3.7, valuable feedback and guidance can be provided to students on various portions of the project without also needing to assign a grade to those same portions (e.g., Roberts 1992; Mackisack 1994; Smith 2011). For example, some offer feedback throughout the project, but only assign a grade for the final report (Mackisack 1994; Chance 1997).

Feedback from peers saves the instructor from having to provide all feedback while still giving students the opportunity to receive guidance on their projects. Not only does peer review save instructor grading time, students find it less intimidating than instructor feedback (Sevin 1995) and peer review can be even more valuable to students than instructor input (Chance 1997; Sisto 2007).

5.3 Guide Students on How to Work in Groups

If students are to work in groups, team management is essential, and one must teach students how to work in teams (Roberts 1992). It may be tempting for students to split up the workload, but everyone should take part in every step of the project to practice all aspects of the statistical analysis process (Sylwester and Mee 1993).

A common concern regarding group work is what many authors refer to as “freeloading” or “free riding” where one or more group participants make notably less effort than their peers. To take a proactive stance against freeloading, some instructors have students decide what they will do if someone is not pulling their weight (Cobb 1993). Zahn (1993) went so far as to have students develop a Team Working Agreement. Several authors have students report on the contributions of team members on the project and encourage them to resolve conflicts on their own (e.g., Smith 1998; Holcomb and Ruffer 2000). Although each group will likely have one submission for each portion of the project, instructors can have one portion of a student’s grade be the group grade and one portion be an individual grade (e.g., Chance 1997).

5.4 Have a Clear Setup for Students

Clear and detailed instructions are important prerequisites for implementing projects (Sylwester and Mee 1993; Sevin 1995). Sisto (2009, p. 6) argues “the more specific and clear the criteria are, the higher the quality of the projects.” Additionally, the grade percentage allotted for the project should be consistent with the time and effort of the student (Sylwester and Mee 1993) and should reflect the importance of the project (Holmes 1997).

5.5 Provide Example Papers

It can be challenging for students to develop research questions on their own, before they learn and experience what type of questions can be answered through statistical analysis (Guyot, Walker, and White 2018). Thus, it is beneficial for students to be given example papers or topics to start brainstorming their own research questions. Similarly, Wardrop (1999, p. 2) writes “few students are creative without models of good projects, but almost every student is creative when provided models of good projects.”

5.6 Have Students Submit Proposals

Several authors suggest students pick their own topic; however, a few go further and suggest that students submit a project proposal prior to collecting data. Some topics may be more complex than students can handle (Field 1985) and a proposal ensures the project is attainable within the scope of the class (Guyot, Walker, and White 2018). Short & Pigeon recommend performing pilot studies to have students “learn more about the data acquisition process without investing large amounts of time and resources” (p. 6). Instructor feedback on proposals can prevent potential pitfalls and set students up for success. Because projects are exploratory by nature, students must articulate a theoretical rationale for the inclusion of variables (Smith 2011). Further, when conducting studies involving data collection from human (or animal) subjects, there should be a discussion with students regarding ethics and a consultation with the relevant institutional research ethics committee for which the study is being conducted should occur (Pileggi, Çetinkaya-Rundel, and Stangl 2014).

5.7 Use Student Projects for Examples and/or Exam Questions

Students find other student projects interesting. Many authors take advantage of past projects to use as examples in class or as exam questions. Having students provide their raw data (e.g., csv files) with clear descriptions of the data collection method and definitions for each variable is highly beneficial for instructors that want to use student work (Ledolter 1995).

5.8 Devote Class Time to the Project

Class time can be used to emphasize the importance of the project and allocate time for students to work on specific parts of the project (Halvorsen and Moore 2001). Class time can show students not only “how” to do something with data, but “what” to do with data (Fillebrown 1994). Students can produce better projects when instructors use class time to proactively address common pitfalls (Smith 2011).

5.9 View Projects as an Opportunity to Learn with Students

Projects can be used as a great opportunity to learn with and from students (Biajone 2006). Wardrop (1999, p. 2) says “I must check my ego at the door—frequently one of my students finds a clear and creative answer that I missed.”

6 Future Directions

As vast as the literature on projects is, there remain areas for future research. The sections that follow outline areas to consider.

6.1 Project Assessment

Several authors advise having clear rubrics for students (e.g., Sylwester and Mee 1993; Sevin 1995; Zeleke 2006); however, little is written regarding how exactly to construct such a rubric. Further, although many authors provide example rubrics, clarification can be added to specify how exactly the criterion assessed in the rubric matches current statistics and data science recommendations. In response to concerns regarding the time-consuming nature of grading projects, it may be of interest to have insight on the amount of time saved using certain rubrics or grading approaches. In order for a rubric to be useful in a variety of settings, it would be beneficial to show that a rubric can assess a diverse number of project topics, analysis types, and project frameworks. Analysis of the validity and intra-rater reliability of such rubrics would be beneficial, and is lacking in the current literature.

With many authors proclaiming the benefits of feedback, it is of interest to explore how a student interacts with and responds to a rubric. For example, are students able to predict their grade based on a certain rubric? Is the rubric clear enough for students to use in assessing other student projects? Is there a balance in the amount of information provided so students have clarity, but not so much information that they gloss over the list? Are students motivated by the feedback received from the rubric, or are they merely checking boxes? This information would help to guide instructors in the most effective use of their time and simultaneously provide the most useful feedback to students.

6.2 Controlled Studies

The controlled studies that explore the impact of projects focus heavily on student attitudes and statistical content knowledge. Although attitudes and content knowledge may be byproducts of completing projects, the main purpose of implementing projects is to provide students a unique opportunity to do statistics. For example, a study exploring statistical content knowledge may assess whether a student can identify a random sample, given a proper description. This same study may not be able to assess whether a student can successfully collect or source a random sample and create a tidy dataset suitable for exploring with statistical software. Future research can therefore explore quantitatively how well students are able to conduct specific elements of a statistical process as a result of completing projects.

It has been noted that future research could explore the impact of conducting more than one project in a given term (Carnell 2008; Ramirez and Bond 2014). Several authors have successfully implemented multiple projects within a course. It therefore seems feasible to explore the relationship between the number of projects conducted and variables such as student attitudes, statistical content knowledge, and performing stages of a statistical analysis process.

6.3 Survey of What is Being Done Currently

Although many authors have written about using projects within their class, there are undoubtedly other instructors implementing projects that have not written about their experiences. Garfield et al. (2002) conducted a survey of introductory statistics classes to gauge the reform efforts of the prior decade. With the advances in technology, continued educational reform, and accessibility of data, a more current survey could be conducted to discover the kinds of projects that are being implemented in statistics and data science classes. This exploration would better gauge the continuing transformation of the field of statistics and data science education. Of particular interest would be how instructors are incorporating current data gathering, analysis, and visualization techniques in such projects. This survey could be rather expansive as statistics and data science topics are introduced at young ages and projects are being used beyond the traditional classroom.

6.4 Adapting Projects to Different Settings Including Data Science Courses

As discussed throughout this article, there are many examples of projects being successfully implemented in a variety of contexts. An interested reader can now use Section 3 to locate specific citations that match a particular setting of interest. Although these articles provide useful information, they do not also explore how the project could be altered to different settings. Regardless of how well the setting within the written article matches that of the interested reader, the reader may still desire to make adjustments within the project implementation process.

It would be beneficial to have a comprehensive body of work that not only details a recommended structure of an investigative project, but also advises how to tailor such a project to different student skill levels, audience ages, depth of statistical knowledge, and class sizes. Additionally, as data science is a newer field still being defined (Lee et al. 2022), future work to develop guidance for incorporating projects into data science courses would be beneficial. Further, it may be of interest to detail how recommendations may differ between data science courses and more traditional statistics courses.

Acknowledgments

The author would like to send special thanks to Jonathan Davidson and Richard Glejzer for providing valuable feedback and insight throughout the initial construction of this article. Additional thanks to Jessica Denke and the Muhlenberg College library staff for tracking down several hard-to-find articles to make this review of project work even more robust. The author would also like to thank the editor, associate editor, and anonymous reviewers for their valuable comments and suggestions.

Disclosure Statement

No potential conflict of interest was reported by the author(s).