The cognition of programming: logical reasoning, algebra and vocabulary skills predict programming performance following an introductory computing course

Figures & data

Figure 1. Example of SCS1-Short question. Note. This figure presents question 5 of Version 1 of the SCS1-Short.

Figure 2. Example of course exam question. Note: This figure presents a question on the topic of “variables and conditions” from the course exam.

Figure 3. Example of a learning item on the vocabulary learning test. [To view this figure in color, please see the online version of this journal.] Note: This figure shows one of 20 learning items on the vocabulary learning test.

Figure 4. Example of a learning item on the grammar learning test. [To view this figure in color, please see the online version of this journal.] Note: This example from the grammar learning test shows the training sentence on the left that described the image on the right.

Figure 5. Partial slopes for each standardised cognitive skill predicting scores on the SCS1-short and course grades. [To view this figure in color, please see the online version of this journal.] Note: Top row: Partial slopes for each standardised cognitive skill predicting raw scores on the SCS1-Short. Bottom row: Partial slopes for each standardised cognitive skill predicting raw course grades. For the models relating to each plot the other predictors and control variables are held constant. The shaded area represents a pointwise confidence band based on standard errors. The lines on the horizontal axes show the exact scores on the cognitive tests for each individual participant.

Table 1. Predictors of score on the SCS1-short programming test and course grade at the end of the semester.

		SCS1-Short					Course grade
	Type of variable	β Estimate	Std Error	t-value	p-value	BF10	β Estimate	Std Error	t-value	p-value	BF10
First experience	Control	1.072	.246	4.351	<.001***	1249.965^+	10.997	2.138	5.144	<.001***	30335.880^+
Version SCS1-Short	Control	0.598	.247	2.422	.016*	3.802^+	-	-	-	-	-
Pattern recognition	Predictor	0.042	.134	.327	.744	0.262^-	1.532	1.128	1.358	.176	0.564
Algebra	Predictor	0.300	.135	2.228	.027*	2.513	1.762	1.175	1.500	.135	0.682
Logical reasoning	Predictor	0.408	.134	3.040	.003**	17.628^+	3.106	1.181	2.630	.009**	5.937^+
Grammar learning	Predictor	0.122	.149	0.816	.415	0.341	1.750	1.289	1.358	.176	0.564
Vocabulary delayed recall	Predictor	0.370	.135	2.737	.007**	7.981^+	2.236	1.173	1.907	.058	1.304

Note: Results for the multiple regression models with standardised scores on the cognitive tests at the start of the semester as predictors, and raw scores on the SCS1-Short and the course grades at the end of the semester as outcome measures. Whether the course was their first programming experience was added as a control variable for both regression models. For the regression model with SCS1-Short as the dependent variable the version of the SCS1-Short was added at a control variable as well. *indicates p-value below .05, ** indicates p-value below .01, *** indicates p-value below .001. ⁺indicates BF₁₀ > 3; – indicates BF₁₀ < .333.

Table 2. Correlations between cognitive tests.

	Pattern recognition	Algebra	Logical reasoning	Grammar learning	Vocabulary learning
Pattern recognition	1	0.270*	0.185*	0.197*	0.082
Algebra	0.270*	1	0.254*	0.287*	0.173*
Logical reasoning	0.185*	0.254*	1	0.267*	0.219*
Grammar learning	0.197*	0.287*	0.267*	1	0.369**
Vocabulary learning	0.082	0.173*	0.219*	0.369**	1

Note: Correlations between cognitive tests only including complete observations. *indicates a small correlation, and ** indicates a moderate correlation.

Figure 6. Fitted structural model representing the relationship between language and algorithmic/mathematical thinking and generalised programming skill as measured by the SCS1-short. [To view this figure in color, please see the online version of this journal.] Note: Blue arrows and * indicate significant estimates at p < .05. Red arrows indicate p > .05. Estimates are written along their corresponding model lines.

Figure 7. Fitted structural model representing the relationship between language and algorithmic/mathematical thinking and course-related programming skill as measured by the course grades. [To view this figure in color, please see the online version of this journal.]

Note: Blue arrows and * indicate significant estimates at p < .05. Red arrows indicate p > .05. Estimates are written along their corresponding model lines.

Table

SCS1-S version 1				SCS1-S version 2
Question number SCS1-S1	Original question from SCS1	Topic	Type of question	Question number SCS1-S2	Original question from SCS1	Topic	Type of question
1	1	for	definitional	1	3	while	definitional
2	2	logical operator	tracing	2	4	arrays	definitional
3	5	function return values	tracing	3	7	while	code completion
4	6	if	definitional	4	8	for	Tracing
5	9	while	tracing	5	10	logical operator	definitional
6	12	basics	definitional	6	11	function return values	definitional
7	13	for	code completion	7	16	function parameters	code completion
8	14	recursion	definitional	8	19	if	tracing
9	17	arrays	code completion	9	20	function parameters	Definitional
10	18	recursion	code completion	10	21	if	code completion
11	22	arrays	tracing	11	23	basics	tracing
12	25	basics	code completion	12	24	recursion	Tracing
13	27	function parameters	tracing	13	26	logical operator	code completion

Data availability statement

The data that support the findings of this study are openly available in the Cognition of Coding project in OSF storage at https://osf.io/8nax4/?view_only=eb0341df544b435792e436451929e2cd.