Balancing the Books: Analyzing the Impact of a Federal Budget Deliberative Simulation on Student Learning and Opinion

Abstract

The federal budget and the rising national debt are crucial concerns in American politics. Yet, they are issues about which average citizens, and particularly young citizens, are presumed to have limited knowledge and very little to say. They are also topics that are not generally seen as engaging to students in introductory political science courses. This article describes the implementation and assessment of a deliberative simulation of the budget process that aimed to inform students about the federal budget through active learning. We use an experimental design to test the impact of the deliberative exercise on two factors: (1) students’ retention of knowledge about the federal budget and (2) the effect of public debate on students’ substantive policy decisions about the budget. As the deliberative exercise we employ mirrors one conducted nationally among the general public by the organization AmericaSpeaks, we also compare students’ responses with those of citizens more generally.

Keywords:

• deliberation
• federal budget
• knowledge
• opinion
• simulations

Notes

As quoted in Shlaes (2011).

This is evidenced by the omission of the policy chapters in many of the “brief” editions of popular American government introductory texts. In addition, the budget process cuts across many institutions that are generally taught using a sequential approach (Congress, the Presidency, the bureaucracy, etc.) in introductory courses. This is likely an additional reason why the budget is not generally a significant element of introductory courses. Daniel Stroup and William Garriott (1997) argue that the segmented approach to American government is flawed as students find it difficult to integrate their learning and to comprehend how institutions interact in the process of policymaking. They advocate a historical “era-” driven approach for introductory courses that pushes students to see the political process as an interaction between the economic and social context and political institutions. Their argument is compelling, yet it does not appear to have drawn wide support nor led to significant changes in textbook design. Adding a segment on the national budget to introductory courses may achieve some of their objectives but does not call for a wholesale redesign of courses.

The state of Texas mandates that all college graduates take two courses in government for just this purpose.

We recognize that simulations may have additional benefits in enhancing students’ political efficacy and promoting political participation but this is not our focus here.

See also Raymond and Usherwood (2013) on the topic of building assessment of student-learning outcomes into simulations.

Individual essays focus on the civic role of colleges in promoting financial literacy (Yarrow and Orphan 2010), a campus effort to incorporate budget topics into general education core courses (Galatas and Pressley 2010), the application of concepts learned in a political theory course to budget policy (J. W. Martin 2010), the use of intergenerational discussions on the federal debt in economics courses (Catlett 2010) and exemplars of cocurricular activities promoting knowledge about the federal debt (Stollman 2010).

Public Agenda is a national, nonprofit, nonpartisan organization dedicated to strengthening democracy and improving people's lives. The American Democracy Project (ADP) is a multicampus initiative of the American Association of State Colleges and Universities (AASCU), focused on higher education's role in preparing the next generation of informed, engaged citizens for our democracy. For information about their joint project, see Public Agenda (2011).

Students were asked seven questions about the size of the federal budget and deficit and the proportions of the budget devoted to social security, Medicare, defense, and foreign aid. The average number answered correctly was between one and two questions on both the pre- and posttest.

Comprehensive reviews of theoretical concerns are provided by Bohman (1998), Chambers (2003), and Mansbridge et al. (2010), while empirical implications are explored in Thompson (2008) and Smith (2009), and changes in policy preferences are addressed in Barabas (2004) and Fishkin (2002).

See for example, James Fishkin (2009).

The organization is nonpartisan in approach and its mission is to “reinvigorate American Democracy by engaging citizens in the public decision-making that most impacts their lives.” AmericaSpeaks provides extensive information about the organization's history, mission, financial contributors, projects, and staff on its Web site (AmericaSpeaks 2010a).

Comprehensive details about the national project are provided in Esterling et al. (2010) and are available at the project Web site (AmericaSpeaks 2010d).

The advisory committee for the project includes individuals from a diverse array of organizations ranging from Economic Policy Institute to the Heritage Foundation. A complete list is provided in both AmericaSpeaks (2010b) and (2010c).

The budget primer and options workbook are available to download free of charge at the project Web site; http://usabudgetdiscussion.org.

Some, mostly liberal, observers felt that the materials “stacked the deck” and pushed readers toward favoring reducing entitlement programs such as Medicare and especially social security (Baker 2010; Eskow 2010; Hickey 2010; Page and Jacobs 2010). These observers also questioned the neutrality of AmericaSpeaks primarily because the organization receives some of its funding from the Peter G. Peterson Foundation, a nonprofit that has been active in promoting deficit reduction (Baker 2010; Eskow 2010; Hickey 2010; Page and Jacobs 2010). In response to this public criticism, AmericaSpeaks highlighted the fact that the OBOE project also received funding from the W. K. Kellog Foundation and the MacArthur Foundation that are generally characterized as more liberal in orientation; and in addition AmericaSpeaks emphasized the careful steps they had taken in preparing the educational materials and selecting participants for their deliberative forums (Fung 2010; Lukensmeyer 2010). Public criticism quelled once the polls measuring the policy preferences of participants in the OBOE deliberation were released. The results were consistent with a number of public opinion polls on similar questions derived from various polling formats (Fung 2010).

Upon closer review of the criticisms leveled at AmericaSpeaks and the OBOE issue forum revolved not around the materials prepared for participants but due to the “agenda-setting” role of the issue forum. A crucial objection was that the deliberations revolved around policies for reducing the national debt when many detractors believed that unemployment was the most crucial economic problem facing the nation. Thus, they were critical that the project did not gauge public opinion regarding whether deficit reduction should be a priority but rather simply engaged the public in strategies for deficit reduction. See especially Page and Jacobs (2010).

In order to attain an ample sample size, the deliberative issue forum was conducted in two midsize classes of about 40 students each; one in the spring semester and one in the fall. The control group was a large lecture class of about 80 students held in the spring semester. Students self-selected into the courses, but they had no awareness that the experiment was taking place, and no reason to opt for one section of the course over another beyond the convenience of scheduling. We recognize that it would have been ideal to have gathered all of our data in one semester; however, the ability to gain a reasonable sample size outweighs the negative aspects of spreading the experiment over two semesters. The economic and media context regarding budget issues did not alter significantly over the course of the year, thus it is unlikely that the fact that some students participated in the deliberative forum in the fall semester while others and the control group participated in the spring should not impact our results.

Student identification numbers rather than names were used on these surveys to reassure students that their responses were not readily identifiable.

To ensure all students were prepared to discuss the material, they were required to bring to class a statement identifying their preferred policy option for the week's topic as well as reasons for their preference. These brief two paragraph statements were factored into their participation grade for the course.

See Esterling, Fung, and Lee (2010).

The questions were presented in multiple-choice format as part of the larger survey instrument.

It should be noted that these knowledge gains are not attributable to students being asked to study the material for the purposes of the test; the students did not have any idea that they would be questioned on the material related to the budget at the end of the semester. Hence, the gains should be seen as resulting from student's exposure to the material either through the lecture or simulation.

In chi-square tests, statistical significance was achieved at the .10 level for debt among the control group, entitlements for the simulation group and at the .05 level for entitlements for the control group and deficits for the simulation group (n = 131 for both groups).

Both statistically significant at the .05 level for the simulation group. Significant at .10 for the control group for defense (n = 131 for both groups).

Improvement is statistically significant at the .001 level for both the simulation and control groups. Corporate tax is significant at the .05 level for the control group (n = 131 for both groups).

These results were significant at the p < .05 and p < .01 levels, respectively (n = 131 for both groups).

The p-value is < .10 for foreign aid for simulation, p < .01 for all other conditions (n = 131 for both groups).

Thus, a student who scored a 5 on the pretest but then shifted to a 4 on the posttest would have a change score of 1, indicating a shift to a more supportive position. Likewise a student who initially selected 1 on the pretest but then moved to a 3 would receive a − 2 score for change—thereby indicating a movement away from supporting the policy position.

Chi-square results comparing lecture and simulation groups on measures of change were not significant for any of the policy questions (n = 131 for both groups).

In a few instances, our relatively small sample size may account for the lack of statistical significance. However, as the graphs illustrate, on many policy dimensions, the control and simulation groups had very similar policy shifts, thus pointing to the fact that it was the receipt of information more than the deliberation that accounted for the shifts.