Assessing student-perceived impact of using artificial intelligence tools: Construction of a synthetic index of application in higher education

Abstract

This study aims to assess the adoption and impact of Artificial Intelligence (A.I.) tools in higher education, focusing on a private university in Latin America. Guided by the question, “What is the impact, as perceived by university students, of using Artificial Intelligence tools on various dimensions of learning and teaching within the context of higher education?” the study employs a rigorously validated 30-item instrument to examine five key dimensions: 1) Effectiveness use of A.I. tools, 2) Effectiveness use of ChatGPT, 3) Student’s proficiency using A.I. tools, 4) Teacher’s proficiency in A.I. and 5) Advanced student skills in A.I. These dimensions form a synthetic index used for comprehensive evaluation. Targeting 4,127 students from the university’s schools of Engineering, Business, and Arts, the study garnered 21,449 responses, analyzed using Confirmatory Factor Analysis for validity. Findings indicate a significantly positive impact of A.I. tools on student academic experiences, including enhanced comprehension, creativity, and productivity. Importantly, the study identifies areas with low and high A.I. integration, serving as an institutional diagnostic tool. The data underscores the importance of A.I. proficiency among both educators and students, advocating for its integration as a pedagogical evolution rather than just a technological shift. This research has critical implications for data-driven decision-making in higher education, offering a robust framework for institutions aiming to navigate the complexities of A.I. implementation.

Keywords:

• Artificial intelligence in higher education
• ChatGpt
• synthetic index
• learning
• teaching
• Artificial Intelligence tools

PUBLIC INTEREST STATEMENT

This study introduces a pioneering tool, a survey instrument, crafted to gauge the impact of Artificial Intelligence (AI) tools on the university educational experience. Through this instrument, which has been rigorously validated at a private university in Latin America, other institutions can now assess how their students and faculty are utilizing AI. By administering the survey to over 4,000 students, we have found that AI significantly benefits students’ understanding, creativity, and productivity. The developed synthetic index provides universities with a tangible way to measure AI integration and promote its effective use. The findings of this study not only highlight AI as a crucial pedagogical advancement but also offer a robust framework for data-driven decision-making in higher education.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Availability of data and materials

The datasets generated and analyzed during the current study are not publicly available due to privacy restrictions but are available from the corresponding author on reasonable request.

Additional information

Notes on contributors

Alberto Grájeda

Alberto Grájeda, Ph.D. in Business Innovation from the Universitat Politècnica de València, Spain, leads the Center for Innovation in Information Technology for Education at UPB. His expertise lies in the effective implementation of educational tools utilizing ICT.

Johnny Burgos

Johnny Burgos, professor and researcher in Artificial Intelligence, specializing in international marketing and pricing. He is a doctoral candidate in Business Administration at UPB, Bolivia. Currently, he serves as an academic advisor to the president at UPB.

Pamela Córdova

Pamela Córdova, an expert in Health Economics and Impact Evaluation, holds a Ph.D. in Economics from UPB. She is a professor, researcher at the Economics Center, and currently serves as the Dean of the Faculty of Business Sciences and Law.

Alberto Sanjinés

Alberto Sanjinés, an expert in Business Ethics, Compliance, and Social Responsibility, holds a Ph.D. in Business Administration from UPB, Bolivia. He currently serves as the Academic Vice President at UPB, leading the process of academic and tech-ed innovation.