Assessing student-perceived impact of using artificial intelligence tools: Construction of a synthetic index of application in higher education

Figures & data

Figure 1. The five dimensions of the use of A.I. tools in higher education.

Table 1. Developed survey instrument

Dimension	Items	Questions
Effectiveness use of A.I. tools	I1	A.I. tools helped me solve my doubts effectively in this subject
	I2	By using A.I. tools in this subject, I was able to improve my understanding of the course material
	I3	The A.I. tools used in this subject allowed me to explore and delve into topics effectively
	I4	The A.I. tools used in this subject encouraged my creativity and innovation
	I5	The A.I. tools used in this subject increased my productivity
	I6	The A.I. tools used in this subject helped me find answers faster than other available resources (such as Google search)
	I7	The A.I. tools used in this subject were useful for the evaluation of my knowledge
	I8	By using A.I. tools in this field, I was able to improve my university education
	I9	I recommend the use of A.I. tools to other students for this subject
Effectiveness use of ChatGPT	I10	The ChatGPT used in this subject was a useful tool to improve my interaction with my classmates
	I11	The ChatGPT used in this subject was a useful tool to improve my interaction with the teacher
	I12	I got proper answers to my questions in ChatGPT used in this subject
	I13	ChatGPT was a reliable tool for learning in this subject
	I14	The ChatGPT used in this subject improved my ability to apply the knowledge acquired to practical situations
	I15	The use of ChatGPT used in this subject, improved my ability to conduct academic research
	I16	When I used A.I. tools such as ChatGPT in practices, tasks or projects in this subject, I included the bibliographical references
Student’s proficiency using A.I. tools	I17	I had no difficulties to formulate the questions in the ChatGPT in this subject
	I18	I found it easy to use A.I. tools to formulate my questions in this subject
	I19	The A.I. tools used in this subject were easy to use to solve problems
	I20	I frequently used A.I. tools in this subject
	I21	I master A.I. tools applied to my career promoted by this subject
	I22	I master the use of the ChatGPT tool in this subject
Teacher’s proficiency in A.I.	I23	The teacher of this subject demonstrated that he masters A.I. tools
	I24	The teacher of this subject promoted the use of GPT Chat in this subject
	I25	ChatGPT is the A.I. tool that I have used the most in this subject
	I26	A.I. tools were applied to this subject
Advanced student skills in A.I.	I27	I used A.I. tools in this subject, to improve a text that I wrote myself
	I28	With the help of the A.I. used in this subject, I was able to expand or synthesize a text that I wrote
	I29	I used A.I. tools in this subject, to evaluate the works that I send to the teacher
	I30	I used A.I. tools in this subject, to verify the relevance of the teacher evaluation

Figure 2. Measurement model.

Table 2. Summary of item level descriptive statistics

Dimension	Items	Range	Mean	SD	Skewness	Kurtosis
Effectiveness use of A.I. tools	I1	1 to 4	3.08	1.02	−0.85	2.54
	I2	1 to 4	3.05	1.01	−0.80	2.50
	I3	1 to 4	3.05	1.02	−0.80	2.46
	I4	1 to 4	3.02	1.03	−0.75	2.39
	I5	1 to 4	3.05	1.02	−0.80	2.46
	I6	1 to 4	3.07	1.01	−0.84	2.55
	I7	1 to 4	3.01	1.03	−0.73	2.35
	I8	1 to 4	3.04	1.02	−0.79	2.47
	I9	1 to 4	3.07	1.01	−0.84	2.55
Effectiveness use of ChatGPT	I10	1 to 4	2.96	1.06	−0.67	2.19
	I11	1 to 4	2.95	1.06	−0.65	2.16
	I12	1 to 4	3.00	1.03	−0.73	2.34
	I13	1 to 4	2.98	1.04	−0.70	2.29
	I14	1 to 4	2.99	1.04	−0.71	2.30
	I15	1 to 4	3.01	1.04	−0.74	2.34
	I16	1 to 4	2.97	1.06	−0.68	2.21
Student’s proficiency using A.I. tools	I17	1 to 4	3.02	1.04	−0.75	2.36
	I18	1 to 4	2.98	1.06	−0.70	2.22
	I19	1 to 4	2.96	1.08	−0.67	2.14
	I20	1 to 4	2.99	1.05	−0.71	2.27
	I21	1 to 4	2.99	1.04	−0.71	2.31
	I22	1 to 4	2.99	1.04	−0.71	2.29
Teacher’s proficiency in A.I.	I23	1 to 4	3.02	1.04	−0.77	2.38
	I24	1 to 4	3.03	1.03	−0.78	2.43
	I25	1 to 4	3.02	1.03	−0.76	2.39
	I26	1 to 4	2.94	1.06	−0.63	2.14
Advanced student skills in A.I.	I27	1 to 4	2.94	1.08	−0.64	2.10
	I28	1 to 4	2.95	1.07	−0.66	2.14
	I29	1 to 4	2.89	1.10	−0.58	−0.56
	I30	1 to 4	2.88	1.11	1.99	1.94

Table 3. KMO statistic by dimension

Dimension	Effectiveness use of A.I. tools	Effectiveness use of ChatGPT	Student’s proficiency using A.I. tools	Teacher’s proficiency in A.I.	Advanced student skills in A.I.
KMO	0.9718	0.9531	0.9357	0.8766	0.8531

Table 4. Factor loadings based on CFA and promax rotation with Kaiser normalization

Dimension	Items	Factor1	Factor 2	Factor 3	Factor 4	Factor 5
Effectiveness use of A.I. tools	I1	0.878	0.100	−0.079	0.059	−0.082
	I2	0.624	0.034	0.291	0.157	−0.010
	I3	0.608	0.020	0.038	0.301	0.188
	I4	0.597	−0.024	0.304	−0.010	0.267
	I5	0.538	0.062	0.286	0.025	0.249
	I6	0.536	0.132	0.124	0.265	0.163
	I7	0.513	0.173	0.285	0.190	0.048
	I8	0.502	0.095	0.429	0.067	0.191
	I9	0.500	0.226	0.263	−0.183	0.309
Effectiveness use of ChatGPT	I10	−0.073	0.866	0.063	−0.098	0.241
	I11	−0.023	0.800	0.158	0.083	0.076
	I12	0.305	0.734	−0.016	0.035	−0.071
	I13	0.117	0.691	0.091	0.195	−0.097
	I14	0.151	0.651	0.007	0.338	0.019
	I15	0.308	0.617	0.107	0.140	−0.077
	I16	0.025	0.534	0.148	0.411	0.269
Student’s proficiency using A.I. tools	I17	0.084	0.029	0.671	0.140	0.205
	I18	0.027	0.420	0.605	0.154	0.010
	I19	0.298	0.067	0.569	0.250	0.022
	I20	0.131	0.414	0.562	0.148	0.022
	I21	0.306	0.079	0.524	0.214	0.131
	I22	0.307	0.218	0.542	0.092	0.278
Teacher’s proficiency in A.I.	I23	0.022	−0.065	0.257	0.698	−0.042
	I24	−0.002	0.316	0.229	0.697	−0.157
	I25	0.404	0.014	0.147	0.505	0.161
	I26	0.026	0.353	0.384	0.545	−0.074
Advanced student skills in A.I.	I27	−0.045	0.030	−0.127	0.146	0.881
	I28	0.229	−0.008	0.125	0.100	0.646
	I29	0.204	0.101	0.103	0.269	0.500
	I30	0.303	0.082	0.204	0.265	0.610
Eigenvalue		5.69	3.74	2.52	1.32	1.11
Variance contribution		28.45%	23.74%	15.25%	13.86%	9.24%
Cumulative variance		28.45%	52.19%	67.44%	81.30%	90.54%
Weighing		31.42%	26.22%	16.84%	15.31%	10.21%

Figure 3. Scree plot for the entire set of indicators simultaneously.

Figure 4. Factor scores and SIUAIT at the university level and for each of the three distinct schools.