Review of measurements used in computing education research and suggestions for increasing standardization

ABSTRACT

Background and context: The variables that researchers measure and how they measure them are central in any area of research, including computing education. Which research questions can be asked and how they are answered depends on measurement.

Objective: To summarize the commonly used variables and measurements in computing education and to compare them to best practices in measurement for human-subjects research.

Method: Systematic literature review analyzing 197 papers published during 2013–2017 in computing education research venues.

Findings: The review illuminates common practices related to: variables measured (including learner characteristics), measurements used, and type of data analysis. The paper lists standardized measurements that were used and highlights commonly used variables for which no standardized measures exist.

Implications: The paper concludes with information about best practices currently being used in the community that should be continued, as well as pointing out practices that could be improved along with recommendations for how to begin to adopt those best practices.

KEYWORDS:

• Literature review
• measurement
• learner characteristics
• standardized instruments
• methods

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

Lauren Margulieux

Lauren Margulieuxis an Assistant Professor in the Department of Learning Sciences at Georgia State University.

Tuba Ayer Ketenci

Tuba Ayer Ketenci is an Academic Professional int he Stewart School of Industrial and Systems Engineering at the Georgia Institute of Technology.

Adrienne Decker

Adrienne Deckeris an Assistant Professor in the Department of Engineering Education at the University at Buffalo.