Using Learning Analytics to Predict At-Risk Students in Online Graduate Public Affairs and Administration Education

Abstract

In this global information age, schools that teach public affairs and administration must meet the needs of students. Increasingly, this means providing students information in online classrooms to help them reach their highest potential. The acts of teaching and learning online generate data, but to date, that information has remained largely untapped for assessing student performance.

Using data generated by students in an online Master of Public Administration program, drawn from the Marist College Open Academic Analytics Initiative,1 we identify and analyze characteristics and behaviors that best provide early indication of a student being academically at risk, paying particular attention to the usage of online tools. We find that fairly simple learning analytics models achieve high levels of sensitivity (over 80% of at-risk students identified) with relatively low false positive rates (13.5%). Results will be used to test interventions for improving student performance in real time.

Keywords:

• Learning analytics
• master of public administration
• graduate education
• online learning
• early alerts

Additional information

Notes on contributors

Jay Bainbridge

Jay Bainbridge is an assistant professor of public administration in the School of Management at Marist College. His research applies statistical and research methods to public sector problems. His current research focuses on trends, causes, and solutions to homelessness, especially with respect to the street homeless, and use of performance measurement in the public sector. Prior to joining Marist College, Bainbridge was assistant commissioner of policy and research at the NYC Department of Homeless Services.

James Melitski

James Melitski is an associate professor of public administration in the School of Management at Marist College. His teaching and research interests include organizational theory and behavior, performance measurement, public strategic management, and digital government. His research assesses the use of technology to engage citizens, implement successful programs, and improve the performance of public organizations. His work has appeared in journals such as Public Performance and Management Review, American Review of Public Administration, Public Administration Quarterly, International Journal of Electronic Government Research, International Journal of Public Administration, and International Journal of Organization Theory and Behavior. He is a leading authority on e-government and continues to write and consult throughout the United States and internationally on the subject.

Anne Zahradnik

Anne Zahradnik is an assistant professor of health care administration in the School of Management at Marist College, and a Teagle Scholar. Her research focuses on health care communication and learning outcomes assessment.

Eitel J. M. Lauría

Eitel J. M. Lauría is a professor and graduate director of information systems at the School of Computer Science & Mathematics at Marist College. His broad research interests cover the fields of data and decision science, business intelligence, data mining and predictive analytics, and statistical machine learning, focusing on the application of these disciplines in a variety of domains. His research has been published in a number of prestigious journals, including Decision Support Systems, European Journal of Operational Research, ACM Journal of Data and Information Quality, and Expert Systems with Applications. He is coauthor of a textbook on data and information quality published by MIT/IQ. He has served as an information systems and technology consultant to IBM, Microsoft, Exxon Mobil, Reuters, Hewlett-Packard, Stet-France Telecom, GE Global Research, Ryder, and the World Bank, among other global corporations. Lauría served as the analytics lead of the Open Academic Analytics Initiative (OAAI).

Sandeep Jayaprakash

Sandeep Jayaprakash is a learning analytics specialist in the Academic Technology group at Marist College. He works on applying big data and data-mining techniques in education to identify trends and patterns among learners in a variety of academic settings and their learning methodologies. The insights gained are used to further enrich the learning experience of faculty, students, and others, through building practical and innovative applications to supplement their learning efforts. He has a master’s degree in software development, and his current research focuses on predictive modeling and building large-scale academic early alert and retention systems to identify at-risk student populations.

Josh Baron

Josh Baron is senior academic technology officer at Marist College, responsible for supporting instructional technology initiatives, including distance learning, faculty training, and student support. He plays a leadership role on campus in the area of strategic planning for instructional technology, is currently serving on the Apereo Foundation Board of Directors and was principal investigator on the Open Academic Analytics Initiative (OAAI). He graduated from the University of Michigan with a BS in aerospace engineering, holds an MS in educational technology leadership from the George Washington University, and has successfully completed the Institute for Management and Leadership in Education at Harvard University’s Graduate School of Education.