Promoting Political Competence and Engagement in College Students: An Empirical Study

Abstract

Political participation is critical for the legitimacy of democracy, yet we know surprisingly little about how political competencies develop and can be promoted in young adults. Many studies show low levels of political activity among young Americans, including college students and recent graduates. Although this is widely recognized as a problem, there is little research on specific experiences and practices that show promise for increasing political understanding and involvement among young people. In addition, much existing research on political life focuses on a narrow set of activities, especially voting, rather than on the multiple dimensions of responsible political engagement, dimensions such as the understanding, skills, and motivations that support and enhance many forms of active democratic citizenship. This article describes and shares initial results from a pre- and postsurvey used in The Political Engagement Project, a study of the effects of 21 different courses and programs on a diverse group of undergraduates at a range of colleges and universities across the United States.

Analyses of variance performed on surveys of 481 students conducted at the beginning and end of courses and programs show that educational interventions with a focus on political engagement can significantly boost many dimensions of democratic participation, including expectations for future political activity. On average, all students participating in the twenty-one interventions increased their political knowledge and skills. Other effects of courses and programs differed for students who began their programs with higher versus lower levels of political interest. For the group with lower initial interest, the interventions significantly increased students' sense of identity as politically engaged persons and their expectations for participating in a range of political activities, with small to medium effect sizes (Cohen's d = .24–.66). The group with higher initial interest experienced smaller but significant gains in understanding and skills (d = .12–.19). The results support understanding political engagement as a multidimensional set of inclinations, competencies, and behaviors. The results from this research also provide evidence that well-designed courses and programs can effectively promote these four key dimensions of political engagement in a diverse range of undergraduates without significantly changing students' party identifications or their positions on a liberal-conservative continuum.

Keywords:

• civic education
• civic engagement
• democratic citizenship
• political learning

Acknowledgments

We thank our colleagues and partners in this project, who generously allowed us to study their courses, programs, and students. We particularly acknowledge invaluable research assistance from Jeff Greene, who helped perform the survey data analysis reported here, and from Joshua Corngold, Shubha Dathatri, and Marivic Dizon, who assisted with interviews and other qualitative research. We are also grateful for past research assistance from Jason Stephens, Hahrie Hahn, and John Bullock. This research was generously supported by grants from the Atlantic Philanthropies, Carnegie Corporation, CIRCLE, Ford Foundation, the Walter and Elise Hewlett Foundation, as well as support from the Carnegie Foundation for the Advancement of Teaching.

Notes

ⁱStudents were asked to respond about their sense of identity and to rate their knowledge, skills expectations of behavior on 6-point scales (where 6 is high and 1 is low). Each mean is based on a scale developed through exploratory and confirmatory factor analysis. Cronbach's alphas for scales reported here range from .74 to .87.

Repeated-measure analysis of variance with one within-subject effect tested (presurvey compared to postsurvey) and one within-subject effect tested (low initial political interest compared to high initial political interest). All two-way interactions were significant, indicating a difference between the interest groups in the amount of change (with the low initial interest group changing more than the high initial interest group). Probability level adopted for judging the significance of all analysis was p < .05.

We derived our scales through a two-stage process, first using exploratory factor analysis to create preliminary scales and then using confirmatory factor analysis to examine and further refine those scales. EFA was performed in SPSS using principle factor analysis, also known as principle axis factoring. We used Promax rotation (Kappa value = 4), allowing us to see both an orthogonal solution (which assumes latent factors are independent or uncorrelated and yields a pattern matrix showing factor loadings) and an oblique solution (which allows for latent correlated factors, which we believed were likely, and yields a structure matrix showing correlations between factors and variables). We determined the number of factors through a combination of Kaiser criteria, examining only Eigenvalues ≥ = 1, looking at “elbows” of scree plots of Eigenvalues, and relying on our theoretical assumptions about relationships between and among variables and factors. Factor loadings less than .30 were eliminated. To determine whether the factor solutions produced by the EFAs were acceptable, we followed standard social science practice: we checked KMO statistics (all were greater than .70), Bartlett's test of sphericity (all were significant), communalities (all were greater than .30), correlations (all of which were higher than .30) and highly significant (p < .001). We also relied on the comprehensibility of the factors produced and how well the factors fit with our initial theories about relationships among items and latent constructs.

The EFA scales were further refined through confirmatory factor analysis (CFA), using structural equation modeling in EQS. For the CFAs, we used Hu and Bentler's (1999) criteria that a model has good fit is retained if it has a Standardized Root Mean Squared Residual (SRMR) greater than or equal to .09 (SRMR), and either an acceptable Comparative Fit Index (CFI) (greater than or equal to .96) or Root Mean Squared Error of Approximation (RMSEA) value less than or equal to .06 with a 90% confidence interval whose upper bound is below .06 (Thompson 2004). Simulation studies have shown that the above combinations best meet the dual criteria of rejecting those models that do not fit while also retaining those models that do. The final scales reported here all had alphas above .74. With the exception of the expected participation in political voice activity scale developed by Keeter et al. (2002) all of the scales we report here met criteria for “good fit” in confirmatory factor analysis.

ⁱⁱEffect size (ES) refers to a family of indices that measure the magnitude of a treatment effect and, unlike significance tests, are independent of sample size. There are various methods of computing ES when the two groups are dependent, as in our repeated measures design. We report ES as the standardized differences between the pre- and post-test means, using pooled standard deviation: Cohen's d = M₂–M₁/√[(σ₁ ² + σ₂ ²)/2] (Rosnow and Rosenthal 1996). To interpret the magnitude of ES, we rely on Cohen's (1988) tentative recommendation that d = .20 is small, d = .50 is medium, d = .80 is large. Effect size can also be interpreted in terms of percent of nonoverlap of posttest scores with pretest scores. An ES of 0.0 indicates that the distribution of scores for the posttest overlaps completely with the distribution of scores for the pretest, or 0% of nonoverlap; an ES of .1 indicates nonoverlap of 8% for distribution of scores; an ES of .2 indicates nonoverlap of 15%; an ES of .3 indicates nonoverlap of 21%; an ES of .4 indicates nonoverlap of 27%; an ES of .5 indicates nonoverlap of 33%; an ES of .6 indicates nonoverlap of 38%; and an ES of .7 indicates nonoverlap of 43% in the two distributions (Cohen 1988, 21–23).

1. Heartening news from the 2004 presidential election showed that voter turnout among those under age 25, the least politically active group, climbed 11 points, from 36% in 2000 to 47% percent. However, that age group still trailed the next generation of voters (25 to 34) by nearly 10 points, the next (35 to 44) by 17 points, and the next (45 to 54) by 22 points. (CIRCLE 2005; U.S. Census Bureau 2005). A number of scholars show that this pattern of generational declines cannot be dismissed as life-cycle effects that will fully resolve themselves as young Americans mature—finishing one's education, finding a mate, establishing a career, “settling down,” and developing community ties (See, for example, Delli Carpini and Keeter 1996; Highton and Wolfinger 2001; Schlozman, Brady, Verba, Erkulwater, and Elms 1999; Vanishing Voter 2000; Putnam 2000).

2. The belief that citizens of participatory republics need values and habits suited to the principles of their states is deeply rooted in the canon of political thought, including Aristotle's Politics and Montesquieu's Spirit of the Laws. Similar themes weave through a variety of American political thought, from Thomas Jefferson to John Dewey [See, for example, Jefferson's Notes on the State of Virginia (1794), Dewey's The Public and its Problems (1927) and Democracy and Education (1916)]. More recent scholarship similarly emphasizes that not only does the legitimacy of participatory democracies require the consent of the governed, but that citizens should have the capacities as well as the rights needed to participate in collective self-governance (See, for example, Pateman 1970; Barber 1984, 1992; Gutmann 1987; Gutmann and Thompson 2000; Macedo 2000; Galston 2001; Schlozman 1999).

3. There is considerable research on the role of various costs and benefits in predicting and promoting political activity, such as voluntary organizations that increase civic skills (Verba et al. 1995); registration procedures that can raise or lower the level of initiative required for voting (Wolfinger and Rosenstone 1980); and, more recently, the role of face-to-face contact in increasing voter turnout (Gerber and Green 2000). We have found, however, no large-scale pre- and poststudies of the specific processes involved in developing greater political engagement or the role educational experiences might play in promoting engagement.

4. See, especially, Galston 2001; Perry and Katula 2001. The Harvard Institute of Politics survey included questions about what kinds of incentives and programs college students perceive as being most effective for increasing their participation, but their research is not designed to study the effects of participating in or being exposed to any of the incentives or programs students endorse (Harvard Institute of Politics 2000).

5. Here, as in Educating Citizens, we seek a middle ground between overly narrow and overly inclusive definitions of political activity. We define political involvement as “activities intended to influence social and political institutions, beliefs, or practices and to affect processes and policies related to community welfare, whether that community is local, state, or national or international” (Colby, Ehrlich, Beaumont, and Stephens 2003, 18–19).

6. An early report from the “Baccalaureate and Beyond” study, for example, found that students' college courses, particularly the number of social science courses taken, was significantly related to graduates' political participation (Nie and Hillygus 2001). Moreover, recent longitudinal research shows that both community service and service-learning in academic courses can enhance significantly a range of civic values and behaviors (Astin et al. 2000).

7. The content areas of the interventions were quite varied, with one-fifth of students studying introductory American politics or American national politics and another fifth studying community citizenship and organizing. Thirteen percent studied comparative political and economic systems, 12% studied state politics and history, and the same share studied urban politics. The others were split among democratic theory (5%), leadership (7%), international politics (4%), single-topic public policy (5%); and an issue-advocacy extracurricular program (3%).

8. Nearly half (42%) of the 481 students participated in a one-semester academic course and another 7% were in academic courses lasting one-quarter. One-fifth were in credit-bearing summer programs, and 13% were enrolled in intensive one-semester academic programs, which typically involved multiple courses or an intensive cocurricular component connected to a course. The remainder participated in two-semester intensive academic programs (15%) and an extracurricular program (3%).

9. Eleven interventions were small courses and programs (8–20 students), another seven were medium-sized (21–50 students), and four were large (more than 50 students). The large group included an introductory American politics course of 280 students. To prevent skewing the data too much in favor of the effect of one intervention, a random sample of 60 students was drawn from that course for this analysis.

10. The full text of the question, which comes from the National Election Study, was “Some people seem to follow what's going on in government and public affairs most of the time, whether there's an election going on or not. Others aren't that interested. How often would you say you follow what's going on in government and public affairs?”

11. Work in comparative politics tends to emphasize national, ethnic, ideological, or class aspects of political identity within or across countries, and work in American politics often focuses on Americans' party identification, political ideology and attitudes, and, increasingly, “identity politics” related to racial or ethnic identity.

12. These included the Americorps/Corporation for National Service Assessment (see Aguirre International 2002); the Walt Whitman Center's Measuring Citizenship Project (see Barber, Smith, Ballou, Higgens, Dedrick, and Downing 1997); the IEA international survey (see Amadeo et al. 2002), and the Pew Foundations' Project 540 (see http://www.gse.upenn.edu/cssc/project540/pdfs/executivesummary.pdf).

13. In the Pew national survey, conducted by random-digit-dial, slightly more than one-third of the undergraduates identified themselves as liberal or very liberal, as compared with slightly more than half in our sample. Thirty two percent of college students in the Pew study identified as Democrats, 24% identified as Republicans, 20% were Independent, 14% were “Other” and 11% didn't know. (Data derived from Keeter et al., National Civic Engagement Survey I, Spring 2002).

14. Two (8%) cells had an expected count less than 5. The minimum expected count is 4.55. Although chi-square is considered less robust if expected values are less than 5, the small-expectation cells are rare in our contingency table. Cochran (1954) gives the following rule for contingency tables with more than one degree of freedom: If relatively few expectations are less than 5 (such as 1 cell out of 5, or 2 cells out of 10 or more), a minimum expectation of 1 is allowable in computing χ².