Student Performance in Research Methods Classes: Assessing the Predictive Validity of the DRAMA Scale

Abstract

A recent study appearing in this journal argued that student anxiety concerning research methods classes in criminology and criminal justice is characterized by three separate but related concepts: Disinterest, Relevance Argumentation, and Math Anxiety (DRAMA). The current study extends the previous work by presenting a modified version of the DRAMA scale and assessing its ability to predict student performance among a sample of students enrolled in an introductory research methods class at a university located in the southeastern United States. In support of the main hypothesis, the composite DRAMA scale was a significant, negative predictor of student grades. Subsequent analysis indicated that the effect of DRAMA was largely the result of the Math Anxiety subscale.

Notes

1. The research methods course is required of all students majoring in criminology at the study university. The course description in the undergraduate catalog explains that the course addresses methodological and statistical issues in criminology, and syllabi for the course, both past a present, is available online for students to review prior to enrolling in the course. The availability of this information, coupled with the fact that students regularly speak with one another about past and future required courses, leads to the belief that students generally have a good idea of what the course entails.

2. There were an additional 13 cases that were identified using DFITS and Cook’s D as having an unusual degree of influence. Further investigation into these cases revealed that they had very low grade percentages due to the failure to turn in assignments or take tests. The timing of the missing assignments and tests strongly suggests that these were students who dropped the course mid-semester, but did not file official paperwork. An informal polling of instructors at the study institution suggests that this is not uncommon. For these students, final grade percentage is not an indicator of performance in the class, but rather an indicator of having dropped the course. It might be worthwhile to examine the effect of DRAMA on dropping out of a research methods course, but since we do not have sufficient data to address this research question, we chose to exclude these 13 cases from the analysis, bringing the final sample size to 118. It is worth noting that these cases were also not significantly different from the remainder of the sample in their levels of DRAMA and its subscales (all p-values >0.20).

3. Six of the ten items in Briggs et al. (2009) math anxiety scale were randomly chosen to be in the math anxiety scale in the current study, and small changes were made to the language of each item for the purposes of simplification and/or clarification. For example, in Briggs et al. (2009), the first item for math anxiety read, “I usually have been at ease in math classes,” whereas in the current study the item read, “I usually feel at ease in math classes.” Similar changes were made to items contained within the relevance argumentation and research disinterest scales. In some instances the wording used in the current study was the same as the wording used in Briggs et al. (2009). Readers are referred to Briggs et al. (2009:221) to make a direct comparison of the items.

4. Principal component analysis indicated that the 18 items all loaded highly on a first factor (eigen value = 5.83; average loading = .56) and with varimax rotation three factors emerged with eigen values greater than 2, with each factor corresponding to the 6 items for each of the three subscales. These analyses are available on request.

1Item was reverse-coded.

5. Two sections of the class were taught by professors and two sections were taught by graduate students. As an alternative to including this control variable in the analysis, we created dummy variables for each of the class sections and included all but one of the section dummy variables in the analysis. The results presented in the paper using the ‘professor’ variable were substantively identical to the results that emerged when using section dummy variables.

6. No freshman enrolled in the class.

*p < .05

7. We examined the VIF and tolerance statistics for model 2 given the strong bivariate correlation between the relevance argumentation and student disinterest scales (.67). The VIF for the two scales both were less than 2.0, suggesting that multicollinearity is not likely to be a problem. We ran additional models (available on request) that included only one subscale at a time. The results of these step-wise models revealed the same conclusions that can be gleaned from model 2 presented in Table 3 and are excluded.

8. It is also important to note that the composite DRAMA scale was no more predictive of student performance than the math anxiety subscale. The standardized effect sizes were identical (B = −.18). This further illustrates the idea that math anxiety is the driving force behind student levels of DRAMA.

9. The finding that students who are further along in their academic program (seniors) are less likely to perform well in the research methods class merits brief discussion. On potential explanation for this interesting finding may be that lower-performing students may be more likely to put off taking the research methods class and that the more ambitious and motivated students may take the class earlier in their academic program.

10. The coefficient of variation (SD/mean) for math anxiety was .34, while for the other scales it was only .20 and .23.

*p < .10.

**p < .05.

1Correlations with student performance calculated using the within-section standardized grade value.