Taking Stock: Existing Resources for Assessing a New Vision of Science Learning

ABSTRACT

A new vision of science learning described in the Next Generation Science Standards—particularly the science and engineering practices and their integration with content—pose significant challenges for large-scale assessment. This article explores what might be learned from advances in large-scale science assessment and accountability systems, and in our understanding of students’ engagement with the science and engineering practices, to inform such efforts. We highlight promising examples from current large-scale assessment programs, an alternative national accountability model, and cutting-edge research and offer several cautions to the field.

KEYWORDS:

• computer-based assessment
• large-scale assessment
• performance assessment
• science and engineering practices
• science standards

Funding

An earlier version of the analysis of large-scale assessment frameworks and the New Zealand accountability system appeared in a paper commissioned by the K–12 Center at ETS. We are grateful to Nancy Doorey for her assistance in facilitating discussions with representatives of these assessment programs. Analysis of research on students’ engagement in science and engineering practices, revisions to the original paper, and further conceptual work on large-scale assessment systems were funded by the Gordon and Betty Moore Foundation through Grant GBMF4244 to the first author.

Notes

1. In this article, we focus on consideration of large-scale assessment for accountability/monitoring purposes. In doing so, we do not discount the very important role that classroom-level assessment plays in a comprehensive assessment system and in the implementation of the NGSS. Indeed, efforts are already underway to develop assessments for classroom use (e.g. Achieve, 2014b). However, we argue that the impact of these resources is reduced if large-scale assessments are not also aligned with the new vision of science learning.

2. There are some important differences between learning progressions as defined in this chapter and the progressions defined in the NRC Framework and NGSS, such that not all aspects of the chapter are relevant to assessing the new vision of science teaching; however, the basic challenges entailed in a progression approach are explored in this chapter.

3. As explored in more detail below, the new frameworks for AP® examinations in biology, physics, and chemistry also take this approach.

4. The 2014 NRC report on assessing the NGSS included consideration of state-level accountability assessments, with discussion of their potential pitfalls. Here, we focus on national- and international-level assessments, in part because, consistent with the 2015 NRC report on implementing the NGSS, we think that the work of developing monitoring assessments for the NGSS is likely to require partnerships extending beyond single states.

5. Despite this choice, it should be noted that the TIMSS science framework includes some components of the science and engineering practices that are not well represented in the other assessment frameworks. Thus, examination of specific items from the TIMSS science assessment also may be warranted.

6. We also tried other keywords, such as “practice,” “scientific practices,” assess,” and “engineering,” but the resulting searches were too broad to identify relevant projects for our purposes.

7. The full task is available at http://nationsreportcard.gov/science_2009/hot_g8_scoring.asp.

8. The full task is available at http://nces.ed.gov/nationsreportcard/tel/wells_item.aspx. An associated scoring guide is not available for this task.

9. This component of defining problems appears to be partially represented in the NAEP TEL task. Students are selecting questions from a set of choices, rather than asking their own questions. These questions do help to refine the problem (in terms of determining whether the problem is with the aquifer or with the pump).

10. This component of designing solutions appears to be partially represented in the NAEP TEL task. Consistent with the NAEP TEL framework, students are not required to apply scientific knowledge in order to solve the engineering problem in this task.

Additional information

Funding

An earlier version of the analysis of large-scale assessment frameworks and the New Zealand accountability system appeared in a paper commissioned by the K–12 Center at ETS. We are grateful to Nancy Doorey for her assistance in facilitating discussions with representatives of these assessment programs. Analysis of research on students’ engagement in science and engineering practices, revisions to the original paper, and further conceptual work on large-scale assessment systems were funded by the Gordon and Betty Moore Foundation through Grant GBMF4244 to the first author.