The development and validation of a self-audit survey instrument that evaluates preservice teachers’ confidence to use technologies to support student learning

Figures & data

Table 1. TPACK Component definitions and survey component and item examples. Adapted from Chai et al. (2013).

TPACK Components	Definition	Survey component Item
TK Technological Knowledge	Knowledge about how to use ICT hardware and software and associated peripherals.	General computing skills I can set up a data projector and external speakers to the computer for a presentation.
PK Pedagogical Knowledge	Knowledge about the students’ learning, instructional methods, different educational theories, and learning assessment to teach a subject matter without references towards content.	ICT and technologies pedagogies skills I can use ICT in lesson planning and preparation.
CK Content Knowledge	Knowledge of the subject matter without consideration about teaching the subject matter.	World wide web skills I can explain how computers communicate via a network
PCK Pedagogical Content Knowledge	Knowledge of representing content knowledge and adopting pedagogical strategies to make the specific content/topic more understandable for the learners.	ICT and Technologies pedagogy skills I can use the SAMR model to select how ICT and Technologies will be used in teaching strategies to engage students in their learning
TPK Technological Pedagogical Knowledge	Knowledge of the existence and specifications of various technologies to enable teaching approaches without reference towards subject matter.	Common Technologies skills I can use a Makey Makey and Micro:bit for projects teaching circuitry, programing and electricity
TCK Technological Content Knowledge	Knowledge about how to use technology to represent/research and create the content in different ways without consideration about teaching.	Coding and Robotics I can programme a robot to perform a complex task or solve a complex problem using sensors

Table 2. Key national and international literature sources used to develop the survey.

Source	Author
TPACK framework	Koehler et al. (2013)
Australian Professional Standards for Teachers	AITSL (2018)
Australian Curriculum	ACARA (2022)
UNESCO ICT Competency Framework for Teachers	UNESCO (2018)
Teachers for the Future (TTF) project 2011–12	Romeo et al. (2012)
ICT Elaborations for Graduate Teachers	AITSL (2011)
Teacher ICT Skills: Evaluation of the ICT knowledge and skill levels of Western Australian government schoolteachers	Western Australian Department of Education &Training (2006)

Table 3. Survey components.

1	Components
1	General computing skills
2	Email skills
3	World-wide web skills
4	Word Processing skills
5	Presentation skills
6	Spreadsheet skills
7	Multimedia skills
8	Coding and robotics skills
9	ICT and technologies pedagogy
10	Common technologies used in schools

Table 4. Item fit statistics for n = 10 items in Component 1, General computing skills (n = 296 respondents). Items marked * were identified as ill-fitting.

	Infit		Outfit
Item	MNSQ	ZSTD	MNSQ	ZSTD
1	1.06	.65	1.45	2.26
2	1.04	.49	1.13	.95
3*	.92	−.41	.55	−1.13
4*	.74	−1.52	.28	−2.27
5	1.12	1.22	1.04	.29
6	1.17	2.05	1.13	1.38
7	.79	−2.71	.86	−1.69
8	.83	−2.04	.83	−1.58
9*	.98	−.04	.90	−.10
10	1.07	.87	1.14	1.61

Table 5. Component 1, Item summary fit statistics, Separation and Reliability indices for n = 10 items and n = 296 respondents.

	Infit		Outfit
	MNSQ	ZSTD	MNSQ	ZSTD
Mean	.97	−.1	.93	.0
S.D.	.14	1.4	.31	1.5
Separation = 14.04 Item Reliability = .99

Table 6. Component 1, Item summary fit statistics, Separation and Reliability indices for n = 7 items and n = 296 respondents.

	Infit		Outfit
	MNSQ	ZSTD	MNSQ	ZSTD
Mean	.98	− .2	1.04	.06
S.D.	.12	1.5	.18	1.5
Separation = 12.86 Item Reliability = .99

Figure 1. All n = 10 Component 1 items displayed in a bubble chart.

Figure 2 Component 1 items (n = 7) displayed in a bubble chart after the three non-fitting items were removed.

Figure 3. Wright map of Component 1, General computing skills, showing the initial n = 10 items and n = 7 items that fitted the model.

Table 7. Participants’ ability to affirm Component 1, general computing skills (n = 296).

	Infit		Outfit
	MNSQ	ZSTD	MNSQ	ZSTD
Mean	.99	−.1	1.04	.0
S.D.	.66	1.1	.93	1.1
Separation = 1.9 Participant Reliability = .78

Table 8. Participant and item separation, reliability and fit for survey items within each component after removing n = 23 non-fitting items.

Component		Participant				Item
	n	Sep.	Reliability	MNSQ		Sep.	Reliability	MNSQ
				Infit	Outfit			Infit	Outfit
1. General computing	7	1.90	.78	0.99	1.04	12.86	.99	0.98	1.04
2. Email	8	1.71	.74	0.98	0.93	11.94	.99	1.01	0.93
3. WWW	9	2.18	.83	1.01	0.94	13.26	.99	1.01	0.94
4. Word processing	6	0.79	.38	0.95	0.91	8.34	.99	1.01	0.91
5. Presentation	7	1.90	.78	1.01	0.96	11.76	.99	1.04	0.96
6. Spreadsheet	8	2.62	.87	0.97	0.96	14.80	1.00	1.01	0.96
7. Multimedia	7	2.07	.81	0.97	1.00	10.04	.99	1.00	1.00
8. Coding and robotics	6	0.66	.30	0.99	0.92	5.35	.97	1.03	0.92
9. ICT and technologies	9	2.70	.88	0.99	0.96	10.05	.99	1.01	0.96
10. Common technologies	10	2.11	.82	1.00	0.98	13.64	.99	1.03	0.98

Figure 4. Wright map of all 77 items and 10 components in the instrument.

Table 9. Participant and item fit for the 77 items across all components in the instrument.

Participant	Infit		Outfit
	MNSQ	ZSTD	MNSQ	ZSTD
Mean	1.02	.0	1.00	−.1
S.D.	.33	1.9	.44	1.6
Participant Separation = 4.98 Participant Reliability = .96
Item	Infit		Outfit
	MNSQ	ZSTD	MNSQ	ZSTD
Mean	1.04	.2	1.00	.0
S.D.	.18	1.9	.20	1.6
Item Separation = 14.60 Item Reliability = 1.00

Table 10. PCA: standardized residual variance for all 77 items.

	Eigenvalue	Observed %	Expected%
Total raw variance in observations	199.2608	100.0	100.0
Raw variance explained by measures	122.2608	61.4	62.1
Raw variance explained by participants	39.8481	20.0	20.2
Raw Variance explained by items	82.4127	41.4	41.9
Raw unexplained variance (total)	77.0000	38.6	37.9
Unexplained variance in 1st contrast	5.4615	2.7	7.1
Unexplained variance in 2nd contrast	4.4771	2.2	5.8
Unexplained variance in 3rd contrast	3.8962	2.0	5.1
Unexplained variance in 4th contrast	3.1691	1.6	4.1
Unexplained variance in 5th contrast	3.0406	1.5	3.9

Table 11. PCA: standardized residual variance for each individual component.

Component	Raw measure (explained variance %)	Contrasts (unexplained variance %)
		1st	2nd	3rd	4th	5th
1. General computing	64.2	7.9	7.1	6.3	5	4.8
2. Email	61.4	10.8	6.7	5.6	5	4.5
3. WWW	65.2	5.7	5.4	4.9	4.5	4.4
4. Word processing	57.7	11.1	8.8	7.7	7.3	7.1
5. Presentation	63.6	8.4	6.8	6.4	5.6	4.8
6. Spreadsheet	71.6	6.2	5.7	4.4	3.4	3
7. Multimedia	63.3	11.7	6.9	6.3	5.1	3.6
8. Coding and robotics	50.5	14.7	11.5	9.6	8.5	5.1
9. ICT and technologies	61.7	9.7	6.1	4.7	4.6	4.1
10. Common technologies	64.1	7	4.9	4.5	3.7	< 1

Figure 5. Category probability curves.

Table 12. Summary of category structure.

Category Label	Observed	Obsvd	Sample	Infit	Outfit	Andrich	Category measure
	Count %	Avrge	Expect	MNSQ	MNSQ	Threshold	Measure
1	4908 22	−1.60	−1.59	1.01	1.03	None	(−2.10)
2	4250 19	−.21	−.27	1.02	.97	−.74	−.57
3	5010 22	.61	.69	1.03	.95	.05	.58
4	8624 38	1.85	1.84	1.02	1.07	.69	(2.08)

Table

1. I can identify and name computer components and their functions
2. I can use keyboard shortcuts
3. I can create folders in My Documents and save documents to the correct folders *
4. I can copy, delete and rename files and folders on the computer *
5. I can save a document or file in a different format
6. I can zip and unzip files on the computer
7. I can explain how to do advanced searches for files on the computer
8. I can maintain data on different storage mediums
9. I can print out documents from the computer *
10. I can set up a data projector and external speakers to the computer for a presentation

Table

1. I can access, open, create and send emails *

2. I can identify SPAM emails

3. I can access and use address book entries

4. I can create new entries to an address book for individuals and groups

5. I can locate sent and deleted emails

6. I can add attachments to emails *

7. I can set up emails to include a signature

8. I can set up a mailing list and send an email to multiple users

9. I can set up folders in email

10. I can set up parameters to automatically move messages into folders

Table

1. I can explain how computers communicate via a network

2. I can locate and retrieve information using a variety of search engines and web browsers

3. I can explain how to analyse search results for quality

4. I can explain how search results are selected and ranked

5. I can save images, text and download whole documents from a website to my computer

6. I can explain copywrite laws and use referencing conventions to acknowledge sources *

7. I can create bookmarks and save them to the bookmark bar in a web browser

8. I can explain internet safety

9. I can explain the differences between blogs, wikis and discussion forums

10. I can set up notifications

Table

1. I can open, edit, save and print documents *

2. I can use formatting features

3. I can insert pictures (from file and online) *

4. I can insert and format tables in a document *

5. I can create, edit and apply custom styles in a document

6. I can use spell check and word count in documents *

7. I can insert a header and footer in a document and insert page numbers

8. I can set up an automated table of contents for a document

9. I can change the page orientation from portrait to landscape

10. I can insert, create and edit hyperlinks to text in a document

Table

1. I can open, edit and save a new slide show

2. I can copy/paste or insert images and shapes into a slide show *

3. I can use slide sort organization and make notes in a slide show

4. I can use design templates, transitions, animations and timings to run a presentation *

5. I can link and embed video files into a presentation and can explain the difference between the two methods

6. I can record audio in a slide show presentation

7. I can insert, create and edit hyperlinks in a presentation to navigate around the presentation

8. I can export a slide show as a video file

9. I can print handouts and notes from a slide show *

10. I can select and use appropriate tools in a presentation to share and exchange information to a variety of audiences

Table

1. I can open, edit, add data and save in an existing Excel document

2. I can insert and delete rows and columns in a spreadsheet

3. I can identify a cell reference in a spreadsheet *

4. I can create a new spreadsheet with simple formulas and basic formatting of text and cell styles *

5. I can set up a spreadsheet to automatically alter the number of decimal places

6. I can use the Formula feature to calculate a sum of numbers

7. I can sort data alphabetically or numerically

8. I can create graphs from tables of information in a spreadsheet

9. I can print preview a spreadsheet, set page breaks and print area

10. I can create hyperlinks to the web and between spreadsheets

Table

1. I can open, edit and save multimedia projects

2. I can combine text, image, audio, video and animation in a multimedia presentation

3. I can open, edit and save a mind map, creating sections and subsections to organize ideas

4. I can use a digital video camera for filming and recording and transfer the files to a computer or cloud-based storage

5. I can use a digital camera to take photos and transfer the files to a computer or cloud-based storage

6. I can present and participate in video conferencing, schedule times, manage participants and screen share

7. I can use an iPad and am able to download apps*

8. I can create and edit videos via computer software or apps

9. I can create and maintain a professional social media site for idea sharing *

10. I can record and edit a green screen recording and explain the Chroma Key *

Table

1. I can programme a simple robot to perform a task or solve a problem *

2. I can programme a robot to perform a complex task or solve a complex problem using sensors

3. I can explain the elements of a digital system

4. I can teach the binary number system

5. I can explain the way in which binary digits are used to encode individual pixels of an image

6. I can teach computational thinking strategies

7. I can describe and follow algorithms

8. I can design an algorithm, or modify an existing one, to fix an error or change functionality *

9. I can design a control structure in an algorithm using branching and iteration *

10. I can declare, assign value and use variables when programing *

Table

1. I can use ICT when planning scope and sequence, unit plans and lesson preparation

2. I can use ICT for monitoring, recording and assessing student achievement

3. I can use ICT in administrative tasks like creating classroom newsletters for parents

4. I can use the SAMR model to select how ICT and Technologies will be used in teaching strategies to engage students in their learning

5. I can evaluate the content of ICT and Technologies resources in relation to schooling level, context, teaching implications, cyber safety and cyber ethics *

6. I can explain laws/codes of ethics associated with using ICT and Technologies safely, respectfully and responsibly

7. I can explain the benefits and risks of the use of ICT and Technologies at home, school and in the workplace

8. I can explain common uses for ICT and Technologies beyond school and the future workforce

9. I can set up accessibility functions in computer software and iPad settings to cater for the learning needs of students

10. I can apply student-centred pedagogy to teaching with ICT and Technologies

Table

1. I can create a Microsoft OneNote and set up a OneNote Classroom

2. I can create content in Microsoft 3D Paint

3. I can record and edit on screen recordings to create instructional videos using software

4. I can save, retrieve and share files in OneDrive

5. I can use a webmail calendar to schedule and remind me of appointments and events

6. I can programme in Scratch Jnr and Scratch 3.0

7. I can programme and create games in Minecraft edu, Kodu Game Lab and Hopscotch app

8. I can use a Makey Makey and Micro:bit for projects teaching circuitry, programing and electricity

9. I can use software to create student portfolios

10. I can create and use QR codes for use in my lessons

Chai, C.S., Koh, J.H.L., and Tsai, C.C., 2013. A review of technological pedagogical content knowledge. Educational technology & society, 16 (2), 31–51. https://www.jstor.org/stable/jeductechsoci.16.2.31.

 Web of Science ®Google Scholar

Koehler, M.J., Mishra, P., and Cain, W., 2013. What is technological pedagogical content knowledge (TPACK)? Journal of education, 193 (3), 13–19. doi:10.1177/002205741319300303.

 Google Scholar

Australian Institute for Teaching and School Leadership (AITSL). 2018. Australian professional standards for teachers. Available from: https://www.aitsl.edu.au/docs/default-source/national-policy-framework/australian-professional-standards-for-teachers.pdf?sfvrsn=5800f33c_64.

 Google Scholar

Australian Curriculum, Assessment and Reporting Authority (ACARA). 2022. Australian curriculum version 9, understand this learning area, technologies. Available from: https://v9.australiancurriculum.edu.au/teacher-resources/understand-this-learning-area/technologies.

 Google Scholar

United Nations Educational, Scientific and Cultural Organization. 2018. UNESCO ICT competency framework for teachers. Available from: https://unesdoc.unesco.org/ark:/48223/pf0000265721.locale=en.

 Google Scholar

Romeo, G., Lloyd, M., and Downes, T., 2012. Teaching teachers for the future (TTF): building the ICT in education capacity of the next generation of teachers in Australia. Australasian journal of educational technology, 28 (6), 949–964. doi:10.14742/ajet.804.

 Web of Science ®Google Scholar

Australian Institute for Teaching and School Leadership (AITSL). 2011. ICT elaborations for graduate teachers. Available from: http://technologiesforteaching.weebly.com/uploads/1/6/3/3/16335480/ttf_-_graduate_teacher_standards_-_ict_elaborations_-_200411.pdf.

 Google Scholar

Western Australian, Department of Education and Training. 2006. Teacher ICT skills: evaluation of the Information and Communication Technology (ICT) knowledge and skill levels of Western Australian government schoolteachers. Available from: https://docplayer.net/10374438-Teacher-ict-skills-evaluation-and-accountability-department-of-education-and-training-western-australia.html.

 Google Scholar

Data availability statement

The data that support the findings of this study are openly available in OSF at: https://osf.io/v5wru/?view_only=335c5e541cef4345b029bc0f9e5ad936