Behavior Monitoring in the Middle Grades: Evaluation of the Classroom Performance Survey

ABSTRACT

Middle level school teachers commonly measure and monitor students’ academic performance, but such is not the case for students’ classroom behavior. This is unfortunate, given the rise in students’ classroom behavior problems in middle school, especially for students who experience emotional or behavioral disorders. The Classroom Performance Survey (CPS) is a brief behavior rating scale that has shown promise for use in elementary school settings, but no studies have investigated the psychometric properties of this measure in middle level schools. In the present study, 103 middle grades students at-risk for emotional or behavioral disorders were rated by their classroom teachers using the CPS. We conducted a psychometric evaluation using a multi-step analytic strategy, including confirmatory factor analysis and reliability analysis. Results showed that the CPS displayed evidence of a reliable two-factor structure (Academic Competence and Interpersonal Competence), predictive validity with academic outcomes (percent of assignments completed and student grades), convergent validity with established behavioral and observational measures, and sensitivity to change over time. Study results suggest positive implications for using the CPS as part of behavior intervention and monitoring efforts in middle level schools.

Keywords:

• Middle level schools
• behavior monitoring
• academic skills
• classroom performance survey
• social skills

Introduction

The transition from an elementary to middle level school is a stressful time that requires significant changes for students. The adjustments required are academic, procedural, and social (Akos & Galassi, 2004). Academic adjustments include greater difficulty of classes and increased workload or homework. Procedural concerns refer to learning and adjusting to the complexities of a larger school environment, such as many classrooms, multiple teachers, or getting lost. Social concerns include changes in peer relationships, fitting in, and getting along with new students. Middle level schools are often larger in size than elementary schools, resulting in new dynamics in student-teacher relationships (Eccles & Roeser, 2011; Young et al., 2012).

Differences between the educational environments of typical elementary and middle grades classrooms can also contribute to adolescent behavioral challenges. Secondary school involves shifts in classroom instruction, task complexity, locus of responsibility for learning, and quality of teacher-student relationships (Eccles et al., 1996). Evans et al. (2018) reported that a student’s experience of the transition to a middle level school can be affected by individual factors such as social schema (e.g., peer or teacher relationships and social support), learning schema (e.g., engagement, interest, and perceived control), and academic self-concept (e.g., motivation, academic self-efficacy, and perceived academic competence). Environmental factors also play a role, which include school (e.g., school or class size, teacher expectations, and student autonomy) and home environment (e.g., academic and social support, parenting style, and sibling relationships; Evans et al., 2018).

The change to a middle level school can have negative impacts on adolescent outcomes. For example, Ryan et al. (2013) found that as students transition into middle school, there is a decline in both grade point average (GPA) and intrinsic value for schoolwork. Chung et al. (1998) found that students experienced a significant decrease in academic achievement and a significant increase in psychological distress following the transition to the middle grades. Goldstein et al. (2015) found that increased stress about middle level school transition among adolescents is associated with increased test and performance anxiety, lower school bonding, and decreased academic performance. Evans et al. (2018) noted that the transition to a middle level school can negatively impact several outcomes for adolescents, including academic attainment, social integration, and emotional wellbeing.

Students with Emotional or Behavioral Disorder

Middle grades students with or at-risk for an emotional or behavioral disorder (EBD) can encounter further difficulties. Emotional disturbances adversely affect students’ ability to learn and sustain satisfactory interpersonal relationships (Individuals with Disabilities Education Act, 2004). Kauffman and Landrum (2017) noted that the National Mental Health and Special Education Coalition proposed a definition to replace the term emotional disturbance with emotional or behavioral disorder (in this paper emotional disturbance will be referred to as EBD). This definition describes EBD as a disability that includes behavioral or emotional responses adversely affecting educational performance. This may include externalizing (e.g., aggressive, hyperactive, oppositional), internalizing (e.g., anxiety, depression), or comorbid disturbances.

Students with EBD are more likely to experience academic achievement deficits in content areas of reading, writing, and math than students without EBD (Kauffman & Landrum, 2017). These deficits are likely to remain stable or increase as students move into adolescence (Nelson et al., 2004). Wagner et al. (2005) found that 61.2% of those with EBD have percentile scores in the bottom quartile for reading, while 43% score in the bottom quartile for math. Students with EBD receive special education services an average of one year later than students with other disabilities often due to lack of early identification and screening by schools (Kauffman & Landrum, 2017).

These students also tend to have poor school attendance rates. Students aged 6–17 years with emotional or behavioral difficulties are almost four times as likely to miss more than 10 days of school as compared to their peers without EBD (Centers for Disease Control and Prevention, 2017). Gonzalez (2006) showed evidence indicating that more than 60% of adolescents with EBD are subject to disciplinary actions in one school year, including suspensions and expulsions. They are also more likely to have been suspended or expelled than students in all other disability categories, with an average of seven disciplinary incidents per school year.

Students with EBD also face social problems, including withdrawal, peer rejection, or aggression (Kauffman & Landrum, 2017). Such students report significantly higher rates of peer victimization, as well as lower perceptions of school climate as compared to students without disabilities (Salle et al., 2018). In a study by Wagner et al. (2005), parents of elementary and middle grades students with EBD reported that they had significantly lower social skills than their peers with other disabilities. Discrepancies placing students with EBD at a disadvantage were also found in areas of self-control, assertion, cooperation, cognitive skills, and conversational skills.

Unfortunately, students with EBD are often identified in their teenage years with severe problems, after years of academic and other difficulties (Kauffman & Landrum, 2017). Given the difficulties students with EBD face, it is important to identify, intervene, and monitor their academic and social behavior in schools. Behavior monitoring is also important in the context of school-wide prevention, multi-tiered systems of supports, and response to intervention efforts.

Behavior Monitoring

Behavior monitoring refers to an evaluation process that is used to measure student progress on academic or behavioral goals. Such data helps to inform the effectiveness of an intervention and determine whether growth is occurring at an adequate rate (McDaniel et al., 2015; National Center on Intensive Intervention, 2013). Monitoring measures should be brief so that data on student progress can be collected frequently (Cook et al., 2014; McDaniel et al., 2015). They should also be sensitive to short-term student progress (Scott et al., 2010) and meet reliability and validity standards (Cook et al., 2014).

Student progress toward academic goals is frequently measured using tools such as curriculum-based measurement (CBM). However, monitoring of student behavior involves several challenges as emotional and behavioral symptoms cannot necessarily be measured by a single test score or the frequency of a specific behavior (Kauffman & Landrum, 2017). Unlike academic goals, there is also no specific standard of progress for behavior as there is for academics (Bruhn et al., 2018; McDaniel et al., 2015).

Monitoring of behavior is important to understand the efficacy of an intervention and identify whether a student is adequately responding to it. This data helps to determine whether to continue, modify, or terminate an intervention (Gresham et al., 2010). It is important that behavior monitoring involves documentation and evaluation of skills over time during natural classroom routines (Classen & Cheatham, 2015). Behavior monitoring tools can include the use of brief behavior rating scales (Cook et al., 2014).

Brief Behavior Rating Scales

Brief behavior rating scales (BBRSs) can be used to measure a student’s behavior in natural settings (Cook et al., 2014). They are shorter versions of rating scales that include a smaller number of items, with examples ranging from five (Briesch et al., 2021) to 12 items (Gresham et al., 2010). They are generally completed by a respondent who knows the student well and is considered an expert informant (e.g., a parent or a teacher). BBRSs can be used to assess a variety of behaviors and may be completed by multiple raters with minimal effort. Since they are brief in nature and do not require extensive training, they are appropriate to be used by teachers in school settings. These scales allow a student’s behaviors to be quantified so that they may be compared with a sample of other students or with their previous scores. Due to the advantages offered by BBRSs, they can be effective tools for behavior progress monitoring (Cook et al., 2014; Moulton et al., 2019; Whitcomb & Merrell, 2013).

BBRSs are frequently used to measure student behavior in schools as they are feasible to implement (Cook et al., 2014; Merrell, 1993; Whitcomb & Merrell, 2013). However, the research supporting BBRSs is limited (Moulton et al., 2019). Lewis et al. (2014) noted that additional research is needed regarding the sensitivity and utility of teacher rating scales to monitor students’ behavioral progress in schools. Research in the field of behavior assessment within tiered problem-solving systems is focused on the development of rating scales that are both psychometrically sound and feasible to use in a school context (Chafouleas et al., 2010). There is also a need for monitoring tools that can be used across a range of academic and behavior interventions due to the poor academic outcomes associated with behavior difficulties such as EBD (Cook et al., 2014; Wehby & Kern, 2014). Behavior monitoring tools should demonstrate reliability, validity, usability, and sensitivity to change. The Classroom Performance Survey (CPS) is one such tool that shows promise to track both academic and behavior outcomes for students in school settings.

The Classroom Performance Survey

The CPS was originally developed to address a need in secondary schools for a reliable measure that was effective in identifying academic and social behaviors of students with ADHD (Children and Adults with Attention Deficit Disorder [CAADD], 1996; Robin, 1998). The original version consisted of 20 Likert questions on a 5-point scale (from 1 = always to 5 = never) regarding academic and social strengths and weaknesses. The CPS has been used to examine school impairment in adolescents with ADHD (Kent et al., 2011) and in treatment outcome studies (e.g., Evans et al., 2011; Meyer & Kelley, 2007). Kent et al. (2011) used the CPS to measure work completion and academic potential of adolescents with ADHD. These students were found to complete less work and not work up to their potential as compared to students without ADHD. Evans et al. (2011) used the CPS to study the effects of a school-based psychosocial treatment program for middle grades students with ADHD and found the measure to be sensitive to treatment effects, though only a total score was calculated with no reports of reliability or validity evidence.

Brady et al. (2012) conducted the first psychometric analysis of the CPS across 23 high schools, 875 students, and 146 teachers. They found that a briefer 15-item version yielded the best factor loadings. They validated the measure with high school students and found a two-factor structure (Academic Competence and Interpersonal Competence) could be interpreted similarly across genders. These authors also noted that the CPS could serve as a general screener for school impairment, as a guide to school-based interventions, and as a behavior progress monitoring tool for at-risk students.

Caldarella et al. (2017) conducted the second psychometric evaluation of the CPS with an elementary-age population using a modified version of the measure. These researchers examined data on 356 students identified as at-risk for EBD collected from 160 elementary school teachers in 19 schools across three states. The modified CPS was also found to be composed of two factors (Academic Competence and Interpersonal Competence) and showed evidence of reliability and validity. In a follow-up study Caldarella et al. (2018) showed that the CPS was also sensitive to change following a classroom intervention designed to improve the social skills of elementary school students identified as at risk for EBD. Although there are no psychometric data published on the use of the CPS in middle level schools, it has been recommended by others as a measure targeting key areas of performance that middle grades students with functional impairments frequently struggle with (Evans et al., 2011).

Study Purpose

There is a need for valid and reliable behavior monitoring tools for use in middle level schools, particularly for students with or at-risk for EBD. This study conducted an examination of the psychometric properties of the CPS in middle level schools given that the measure has been shown to be reliable and valid for use in elementary school settings. Specific research questions were as follows: (1) What is the factor structure and model fit (reliability) of the CPS when used at the middle school level? (2) What is the evidence of predictive validity of the CPS scores with academic outcomes (e.g., assignments turned in and class grade)? (3) What is the evidence of convergent validity of the CPS scores with established behavioral and observational measures? (4) Are scores on the CPS sensitive to change over time?

Method

Context

Data for the present analysis were collected during a two-year study of Class-wide Function-related Intervention Teams for Middle School (CW-FIT MS), a classroom management program based on positive behavior support (see Caldarella et al., 2019; Wills et al., 2019 for details). CW-FIT MS uses a teaching approach to classroom management with emphasis on helping students learn necessary classroom skills. Intervention features include teachers (a) directly teaching classroom expectations and prosocial skills, (b) using an interdependent group contingency with differential reinforcement of desired behavior, and (c) minimizing attention for inappropriate behavior by using planned ignoring. These components are implemented with the entire class as part of a Tier 1 intervention or primary prevention.

Settings and Participants

Five Title I middle level schools in the urban Midwest (n = 3) and the interurban Mountain West (n = 2) participated (see Table 1). Data were collected across various subjects in general education classes: math (31.58%), social studies/history (28.95%), English language arts/reading (26.32%), and science (13.16%). Teachers selected one class period that was the most behaviorally challenging. Each participant was involved for a single year.

Table 1 Descriptive data for participating schools

		Student Demographics (Percentage)
School	Student Total	FRL^a	American Indian^b	Asian	Black	Hispanic	Pacific Islander^c	White	Multi-Race
1	648	67.0	0.9	5.5	15.0	18.0	0.8	49.0	10.0
2	558	83.3	0.2	5.5	56.0	19.0	0.4	14.6	4.1
3	812	60.1	0.3	5.3	16.6	15.0	1.6	55.0	5.8
4	847	56.9	1.9	0.8	1.1	35.7	3.8	54.9	1.9
5	875	55.0	1.3	0.3	0.7	34.8	1.6	57.9	3.3

^aFree and Reduced Priced Lunch.

^bAmerican Indian/Alaskan Native.

^cPacific Islander/Native Hawaiian.

Participants included 38 middle level school teachers who were predominantly female (73.68%) and White (92.11%). On average, teachers were 34.33 years old (SD = 9.74) and had taught for 6.74 years (SD = 6.65). Each teacher had approximately 26 students per class, with two to three students in each class identified as student participants. In Year 1, all teachers (n = 10) and their classes received the CW-FIT MS intervention. In Year 2, teachers and their classes were randomly assigned to treatment (n = 14) or control (n = 14) conditions as part of a randomized control trial.

Student participants consisted of 103 middle grades students who had been identified as at-risk for EBD. Students ranged in age from 11 to 14 years (M = 12.27, SD = 0.95) and were predominantly male (81.33%). Students were identified as Black/African American (38.83%), White (33.98%), Hispanic (22.33%), and Asian/Pacific Islander (2.91%), with 1.94% not reported. Students were in sixth (28.16%), seventh (33.98%), and eighth (37.86%) grades. The primary home language was English (72.82%), with no response from 12.62% of participants. Due to missing data, out of 91 students, 14 (13.59%) reported having an individualized educational program, and out of 89 students, 5 (4.85%) reported a 504-accommodation plan.

Identification of Students at Risk for EBD

Student participants were nominated by their teachers as being at-risk for EBD. Teachers completed Stage 1 of the Systematic Screening for Behavior Disorders–Second Edition (SSBD-2; Walker et al., 2014). The SSBD-2 is a standardized norm-based screening tool used to identify students at risk for EBD in elementary and middle level schools (Caldarella et al., 2008; Walker et al., 2014). Teachers (a) read definitions and examples of externalizing and internalizing behaviors, (b) considered all students in their identified class, (c) nominated students whose behavior patterns closely matched externalizing and internalizing behaviors, and then (d) rank ordered the nominated students in order of severity. Informed consent and student assent forms were then solicited for the top five ranked students who exhibited externalizing behaviors. For the CW-FIT MS study, we only solicited consent forms from students nominated for externalizing behaviors, as this was the focus of the intervention. On average, two to three at-risk students per class chose to participate. Each at-risk student was consented and observed in only one class. Direct observations were conducted by researchers to confirm teachers’ nominations of at-risk students. Nominated students were on-task less than 75% of the time and/or displayed greater than four disruptive behaviors during at least one 20-minute baseline observation.

Measures

Classroom Performance Survey

The Classroom Performance Survey (CPS; CAADD, 1996; modified by Robin, 1998) was designed as a behavior rating scale to identify academic and social functioning levels in adolescents, specifically those with ADHD. Lower scores on the CPS suggest higher levels of functioning, while higher scores suggest more impairment. Similar to Brady, we used the briefer 15-item version of the CPS in our study. Like Caldarella et al. (2017), we added five open-ended questions asking about the percentage of completed assignments and current grade of the student at-risk, as well as for the entire class, and if the student was working up to potential (see Appendix).

School Social Behavior Scales--2nd Edition

The School Social Behavior Scales–2nd Edition (SSBS-2; Merrell, 2008) is a nationally normed scale that measures social competence (32 items) and antisocial behavior (32 items) for students in kindergarten through twelfth grade. The SSBS-2 items are rated on a 5-point Likert scale (1 = never to 5 = frequently). Sample items include “follows school and classroom rules,” ‘is “looked up to” or respected by peers,’ “bothers and annoys other students,” and “acts impulsively without thinking.” Scale raw scores are converted to T-scores (x- = 50, SD = 10). According to Merrell (2008), internal consistency alphas range from .96 to .98.

Student Grades

Student grades were reported by the participating teachers in answer to the CPS question, “What is the student’s current grade?”. Due to teacher variability in reporting, we converted any percentage grades to a letter grade for consistency (i.e., A = 90 –100, B = 80–89, etc.) and then assigned the letter grades a number for analysis (i.e., A = 5 , B = 4, C = 3, D = 2, F = 1).

Direct Observations of Student Behavior

Direct observations of student behavior occurred during 20-minute observation periods by trained research observers. Paper/pencil momentary time sampling (30-second intervals) was used. Data were collected on at-risk students’ on-task and disruptive behaviors, as well as on class-wide on-task behaviors and teacher praise and reprimands. For this manuscript, we only report data on target students’ on-task and disruptive behavior. Data were collected three to five times during baseline, which lasted one to two weeks, then aggregated into a baseline average. Observations were conducted during the same class period per teacher, which they reported to be their most behaviorally challenging class period.

On-task behavior was defined as following teacher instruction, complying with classroom rules, looking at teacher/speaker or materials, cooperating with peers during group work, and reading and writing as directed by the teacher. Disruptive behavior was defined as verbal or motor behaviors interfering with the learning or participation of the student at risk and/or their peers. Examples included talking to a peer when not allowed, calling out, making excessive noise, throwing objects, or using them inappropriately, making inappropriate gestures or physical contact with another, or engaging in other distracting behaviors not related to teacher instruction.

To observe the students, the observer would systematically scan the room every 30 seconds and record on/off-task behavior (±) for each student at-risk (scanning student at-risk one, recording student at-risk one, scanning student at-risk two, recording student at-risk two). Observers recorded the frequency of target students’ disruptive behavior using momentary time sampling if that behavior occurred at the 30-second mark.

Interobserver Agreement (IOA)

Observations were conducted by eight research team members (doctoral, master, or bachelor level personnel in education, special education, or school psychology) who were trained by (a) studying behavior definitions and observation techniques and passing a quiz; (b) coding videos of classroom and individual student behaviors, when compared to a master template, until 90% accuracy was achieved three times; and (c) practicing with a research coordinator in a non-study classroom until 90% accuracy was achieved three times. Once trained, data collection began. To maintain quality data, 30.61% of the observations were conducted with two observers to calculate interobserver agreement (IOA) and ensure accuracy of data collection. IOA averaged 93.23% (SD = 6.00) for on-task behavior and 92.59% (SD = 12.57) for disruptive behavior.

Procedures

Researchers worked with local school districts to identify middle level schools to participate in the CW-FIT MS study. After schools were identified and principals agreed, teacher participants were solicited after a recruitment meeting in which an overview of the study and requirements were explained. Teachers voluntarily signed informed consent documents required by the school district and university institutional review boards. Later, teachers reported their most behaviorally challenging class period. A letter of notice of study participation was mailed home to all students in the identified class. After notice was sent home, teachers completed the SSBD-2 Stage 1. Informed consent/student assent was sent home for nominated students and their parents. The SSBS-2 and the CPS (including grades) were then completed by teachers and baseline observational data were collected by researchers. After these data were collected, a teacher training occurred, after which the CW-FIT MS intervention commenced (see Caldarella et al., 2019; Wills et al., 2019 for teacher training and intervention implementation details). Posttest data were collected at the conclusion of the study approximately 10 to 12 weeks later.

Analytic Strategy

A multistep analysis was performed (like Brady et al., 2012; Caldarella et al., 2017) in Mplus 8.4 and SPSS 26. Due to the Likert-type scale nature of the items, they could be treated as categorical. In previous work, Caldarella et al. (2017) intervention found that treating CPS data as continuous led to very similar results as the categorical analysis so that is what we did. Also, because of the nested nature of the data (i.e., multiple students in classrooms), we used CLUSTER=teacherID and the TYPE=COMPLEX in the ANALYSIS section of Mplus. For the present study, we used pre-intervention (baseline) data to address the first three research questions, so there would be no intervention effect on the data. To answer the fourth research question, we used CPS pre- and post-intervention data.

In step one we performed a confirmatory factor analysis (CFA) in Mplus on the CPS. Factor analysis can be used to investigate the validity of behavior monitoring measures (see e.g., Briesch et al., 2021). CFA with continuous items has several assumptions, which were checked and are reported in the “Results” section: linearity between the items, no extreme collinearity, independence of observations, no multivariate outliers, missing data handled appropriately, and the correct model is specified. A good model fit was established by the fit indices produced by Mplus, and Cronbach’s alpha was calculated in SPSS.

Step two investigated predictive validity using the students’ CPS latent variables to predict teacher-reported grades and percent of academic assignments completed. During this process, some model selection decisions were made to avoid overfitting. We found grade level and none of the ethnicities (except for Black) to have statistically significant relations with the outcomes. Thus, all the non-significant controls were excluded for parsimony.

Step three investigated convergent validity of the CPS by predicting observed on-task student behavior, observed student disruptive behavior, and the SSBS-2 teacher reported dimensions by the latent variables of CPS.

Step four investigated the ability of the measure to detect change over time by tracking students (n = 71) involved in the randomized control trial in Year 2 and conducting a structural equation model of their post-trial CPS latent scores on treatment status. Given the hypothesis that treatment would improve Academic Competence and Interpersonal Competence, and not cause a decrease in these measures, a one-tailed p-value was used.

Results

The results section mirrors the four research questions and analytic strategy. Table 2. contains descriptive statistics and correlations of the demographics and test results of the sample. Mplus rescales the factor scores with a mean of 0 by default which explains why the CPS factors have means of 0. The first research question examined the factor structure and model fit (reliability) of the CPS when used at the middle school level. The assumptions for CFA were checked by examining histograms and scatterplots in SPSS and were found to be met insomuch that there were no strong curvilinear relations among the items. The items were handled categorically, thus normality was not a large concern, and the lack of independence was handled by using CLUSTER=student and TYPE=COMPLEX. The data were assumed to be Missing at Random at worst and thus were handled by the Full Information Maximum Likelihood method in Mplus. The CFA was conducted on the data. One item (8: Arrives to class on time) was deleted as it had a very low factor loading (p > .1) and we felt this was theoretically justifiable as the remaining items assessed in-class behavior as opposed to punctuality. The model fit indices showed reasonable model fit for two of the three fit statistics (RMSEA = 0.06, CFI = 0.96, TLI = 0.95) with cutoffs for RMSEA <.08 (Browne & Cudeck, 1993; Byrne, 2013; MacCallum et al., 1996), CFI >.95 (Hu & Bentler, 1998, 1999), and TLI >.90 (Wang & Wang, 2012). The correlation between the two derived factors was significant with a value of .67 (p < .001). Item loadings are shown in Table 3. Cronbach’s alpha was .93 for Academic Competence and .73 for Interpersonal Competence.

Table 2 Correlations of variables and descriptive statistics in model

Variables	1	2	3	4	5	6	7	8	9
1. Black	1.00
2. Assignments complete %	.07	1.00
3. Student grades	.03	.60**	1.00
4. SSBS-2 Social Competence	−.00	.55**	−.32**	1.00
5. SSBS-2 Antisocial Behavior	.18	−.27*	−.23*	−.57**	1.00
6. Student on-task behavior	−.19	.19	.12	.31**	−.30**	1.00
7. Student disruptive behavior	.24*	.03	.10	−.04	.26*	−.58**	1.00
8. CPS Academic Competence	.01	−.84**	−.67**	−.69**	.32**	−.35**	.03	1.00
9. CPS Interpersonal Competence	−.07	−.32**	−.27**	−.88**	.66**	−.30**	.02	.50**	1.00
Descriptive Statistics	1	2	3	4	5	6	7	8	9
Mean	.39	59.73	3.18	90.88	80.67	43.80	7.74	.00	.00
Standard Deviation	.49	33.17	1.61	24.00	25.64	15.51	5.25	.55	.62
Minimum	.00	.00	1.00	39.00	32.00	10.83	0.00	−1.17	−1.68
Maximum	1.00	100.00	5.00	159.00	142.00	82.87	27.00	1.12	1.58
n	103	95	93	96	96	100	100	103	103

CPS = Classroom Performance Survey; Higher CPS scores suggest more impairment; SSBS-2 = School Social Behavior Scales 2nd Edition; Higher SSBS-2 Social Competence scores suggest better competence; Higher SSBS-2 Antisocial Behavior scores suggest more impairment; Mplus rescales the CPS latent variables to have a mean of 0 by default; Higher grade scores suggest higher classroom grades (5 = A[90-100], 4 = B[80-89], 3 = C[70-79], 2 = D[60-69], 1 = F[0-59]). *p < .05, **p < .01

Table 3 Standardized factor loadings for (CPS) academic competence and interpersonal competence factors (N = 103) with items treated as categorical

CPS Items	Factor Loadings
Academic Competence Factor (alpha = .93)
1. Brings necessary items to class	0.52**
2. Completes class assignments	0.88**
3. Completes homework on time	0.85**
4. Records assignments consistently	0.77**
5. Turns in completed work	0.88**
6. Completes long-term assignments	0.77**
7. Attends to instructions in class	0.65**
9. Performs satisfactorily on tests	0.79**
10. Completes assigned work with accurate computation/detail	0.75**
Interpersonal Competence Factor (alpha = .73)
11. Relates positively to peers	0.70**
12. Relates positively to teacher(s)	0.80**
13. Demonstrates respect for property	0.41**
14. Communicates own needs or asks questions	0.45**
15. Accepts assistance when needed or offered	0.60**
Deleted item
8. Arrives to class on time

Standardized factor loadings as produced by Mplus are shown. ESTIMATOR=WLSMV

**p < .01.

The second research question examined if scores on the CPS latent variables (Academic Competence and Interpersonal Competence) predicted academic outcomes (see Table 4). It is important to note that the CPS factors (Academic Competence and Interpersonal Competence) are inversely scored such that a higher score means less of the underlying construct. The model showed that Academic Competence predicted Student On-Task Behavior (B = > −7.11, p = .04, β = > −0.72). Standardized betas (β) are defined as for every standard deviation increase in Academic Competence, there is a −0.72 decrease (improvement) in Student On-Task Behavior indicating a strong effect. Academic Competence also predicted the SSBS-2 Social Competence score (B = > −15.76, p < .01, β = > −0.38), percent assignments completed (B = > −49.42, p < .01, β =-0.87), and student grades (B = > −1.88, p < .01, β = > −0.68). Interpersonal Competence predicted SSBS-2 Social Competence (B = > −22.99, p < .01, β = > −0.65), and SSBS-2 Antisocial Behavior score (B = 23.28, p < .01, β = 0.62). All other predictions were nonsignificant (p > .1).

Table 4 SEM of the Classroom Performance Survey (CPS) latent constructs predicting outcomes in the presence of covariates (N = 103)

Outcomes	CPS Academic Competence			CPS Interpersonal Competence
	B	(SE)	β	B	(SE)	β
Student On-Task Behavior	−7.11*	(3.50)	−0.27	−3.89	(3.31)	−0.17
Student Disruptive Behavior	0.12	(1.34)	0.01	−0.24	(1.12)	0.03
SSBS-2 Social Competence	−15.76**	(5.33)	−0.38	−22.99**	(3.60)	−0.65
SSBS-2 Antisocial Behavior	1.51	(4.52)	0.03	23.28**	(4.76)	0.62
Assignments complete %	−49.42**	(11.94)	−0.87	5.07	(3.74)	0.10
Student Grade	−1.88**	(0.48)	0.68	0.15	(0.22)	−0.06

Note. CPS = Classroom Performance Survey; CPS items are inversely scored which explains the negative betas. *p < .05, **p < .01.

The third research question examined evidence of convergent validity of the CPS factor scores with established behavioral and observational measures. Step three of the analytic strategy correlated the SSBS-2, Student On-Task Behavior, and Student Disruptive Behavior scores with the two CPS factors. The results are shown in Table 2. There were significant correlations between the Academic Competence and Interpersonal Competence and the SSBS-2 scales (Social Competence and Antisocial Behavior) and Student On-Task Behavior in the expected directions, given that CPS Academic Competence and Interpersonal Competence are reverse coded. This shows convergent validity. Student Disruptive Behavior was not correlated with Academic Competence and Interpersonal Competence, like the results of Caldarella et al. (2017).

The fourth research question examined if scores on the CPS were sensitive to change. As mentioned in the analytical strategy section, a structural equation model was run where the students CPS latent scores in Year 2 were regressed on treatment status. The results showed a treatment effect on Academic Competence (B = > −0.38, β = > −0.26, p = .04), and a marginal treatment effect on Interpersonal Competence (B = > −0.31, β = > −0.22, p = .07). Students in the treatment group had an estimated 0.26 standard deviation improvement in Academic Competence and a 0.22 standard deviation improvement in Interpersonal Competence compared to the control group.

Discussion

In this study, we examined the psychometric properties of the CPS when used in middle level school settings. We conducted a CFA of the instrument, examined internal consistency, and studied whether CPS scores predicted student academic outcomes and if it was sensitive to change. The convergent validity of the CPS with other behavioral measures was examined, as recommended by Brady et al. (2012). Results indicate that the CPS shows promise as a BBRS to measure and monitor academic and social skills in middle grades classrooms for students at-risk for EBD. We confirmed that the CPS has two factors (Academic Competence and Interpersonal Competence), like the work of Brady et al. (2012) in high schools, as well as the work of Caldarella et al. (2017) in elementary schools. The CPS appears to be a reliable measure, with high internal consistency scores from both factors. Predictive validity was supported as well, with scores on the CPS Academic Competence factor predicting academic outcomes (i.e., percent of assignments completed and student grades) in the expected directions. Like Caldarella et al. (2017), scores on the Interpersonal Competence factor were not a significant predictor of academic outcomes, as items on this factor are focused on students’ social skills which are more distally related to academic outcomes than the Academic Competence factor items.

Similar to past studies (Brady et al., 2012; Caldarella et al., 2017), the two CPS factors were found to correlate with other measures of academic and social behavior, including direct observations of behavior, helping to further establish validity. The CPS appears to measure constructs that directly relate to academic and interpersonal competence in the middle grades, beyond simply teacher likability of students. In addition, this study showed that the CPS scores of middle level students receiving the CW-FIT MS intervention showed improvement over time compared with the students not receiving the treatment, similar to results found in elementary school (Caldarella et al., 2018). These results suggest that the CPS is sensitive to change in response to an intervention designed to improve student behavior. This is important given the need for additional BBRS and behavior progress monitoring measures (Wehby & Kern, 2014), especially in the middle grades. The results of this study can serve as a reference for additional behavioral monitoring research and intervention efforts.

Limitations and Directions for Future Research

There were some limitations to the current study. While the sample used was diverse, it only included Title I schools and was not extremely large or nationally representative. The generalizability of the findings is also somewhat limited since the study only included students identified as at-risk for EBD nominated as externalizers. In future research, it would be helpful to evaluate the CPS with larger nationally representative samples to establish norms and determine whether the psychometric properties shown in the current study are similar. While we used the shortened version of the CPS, it consisted of 15-items, which is somewhat longer than typical BBRSs. We found that one CPS item (#8) could be dropped because of low factor loadings, which may help to shorten it, though this is an area worthy of further investigation. The CPS scales are scored such that a 1 indicates the behavior always occurs and a 5 indicates the behavior never occurs. This could be confusing to raters when measuring skills rather than deficits. Since previous studies of the CPS have used this scoring procedure (see e.g., Brady et al., 2012; Caldarella et al., 2017; Evans et al., 2011; Meyer & Kelley, 2007), we did not think it wise to change it. We only examined internal consistency reliability, though future studies could investigate other forms of reliability, such as test-retest or inter-rater. This study did not examine whether teachers would find it acceptable and feasible to administer the CPS to monitor student behavior, though this is an area worthy of further exploration. Rather than relying solely on teacher reports, another area for future research is to create a self-report version to gain insight into student perceptions of their own academic and interpersonal skills.

Implications

The study results suggest positive implications for using the CPS as part of behavior intervention efforts in middle level schools. The measure showed evidence of a reliable two factor structure that was predictive of academic outcomes, convergent with other behavioral and observational measures, and sensitive to change over time. We agree with Brady et al. (2012) that the CPS could serve as a general screener for behavioral impairment, as a guide to school-based interventions, and as a behavior monitoring tool for at-risk students. This is important given the need for additional progress monitoring tools in middle level schools, especially for students at risk for EBD. The CPS is brief and should be feasible for teachers to complete to help develop individualized interventions. This could be essential for students struggling with critical academic skills such as organizing their work, participating appropriately in classroom activities, or completing their classroom assignments (Farrington et al., 2012; Minskoff & Allsopp, 2003). Such students could be provided with secondary or tertiary level interventions to improve their academic outcomes, such as instruction in essential academic behaviors (Anderson et al., 2008) or the teaching of self-management strategies (Briesch & Chafouleas, 2009). To determine whether students are improving, or to adjust interventions as needed, the CPS could be used to monitor students’ behavior over time, like how academic skills are monitored. We encourage additional use and study of the CPS in these ways to further investigate its validity and utility.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R324A160279, to the University of Kansas in collaboration with Brigham Young University. The opinions expressed are those of the authors and do not represent the views of the Institute or the U.S. Department of Education.

Appendix

Classroom Performance Survey

Student’s Name: ______________________ Teacher’s Name: _____________________Date Completed: ______________________ Academic Subject: ____________________School: ______________________________

Please complete the following ratings to help us identify the student’s strengths and areas of concern in the classroom. Circle the number that best represents this student’s behaviors in the past 3 week.

Table

	Always		Sometimes		Never
1. Brings necessary items to class	1	2	3	4	5
2. Completes class assignments	1	2	3	4	5
3. Completes homework on time	1	2	3	4	5
4. Records assignments consistently	1	2	3	4	5
5. Turns in completed work	1	2	3	4	5
6. Completes long-term assignments	1	2	3	4	5
7. Attends to instructions in class	1	2	3	4	5
8. Arrives to class on time	1	2	3	4	5
9. Performs satisfactorily on tests	1	2	3	4	5
10. Completes assigned work with accurate computation/detail	1	2	3	4	5
11. Relates positively to peers	1	2	3	4	5
12. Relates positively to teacher(s)	1	2	3	4	5
13. Demonstrates respect for property	1	2	3	4	5
14. Communicates own needs or asks questions	1	2	3	4	5
15. Accepts assistance when needed or offered	1	2	3	4	5

• 16. In the past 3 weeks, what percentage of this student’s assignments were turned in and completed on time? ___%

• 17. What percentage of assignments were handed in completed and on time by the average student in your class? ____%

• 18. What is the student’s current grade? ____

• 19. What is the average grade for students in this class? _____

• 20. Is the student working up to potential? YES NO

Note. Items 16–20 are not part of the Classroom Performance Survey but were added for the purposes of this study.