Factors That Affect Student Outcomes in U.S. Geography Education

Figures & data

Table 1. 2018 National Assessment of Education Progress (NAEP) Geography descriptive statistics for all student-level (Level 1) predictors

Predictor variable	N	M	SD	Minimum	Maximum
Dichotomous
White	12,900	0.39	0.49	0	1
Black	12,900	0.13	0.34	0	1
Hispanic	12,900	0.36	0.48	0	1
Asian	12,900	0.05	0.21	0	1
American Indian, Alaskan Native	12,900	0.02	0.13	0	1
Native Hawaiian, Pacific Islander	12,900	0.01	0.08	0	1
Two or more races	12,900	0.05	0.22	0	1
Free/reduced-price lunch eligible	12,900	0.52	0.50	0	1
Free/reduced-price lunch not eligible	12,900	0.41	0.49	0	1
Free/reduced-price lunch N/A	12,900	0.07	0.26	0	1
ELL	12,900	0.08	0.27	0	1
IEP	12,900	0.12	0.33	0	1
Above modal age	12,900	0.37	0.48	0	1
Female	12,900	0.49	0.50	0	1
Took geography prior to grade 8	12,330	0.73	0.44	0	1
Ordinal
Books in home	12,420	1.56	1.03	0	3
Days absent in last month	12,350	0.91	0.97	0	4
Parental education	10,941	1.23	1.03	–1	2
Use computer tech in classes	12,360	2.40	1.30	0	4
Continuous/factor score
Subjective NAEP importance	12,690	0.00	0.86	–2.52	1.70
Geo activities outside of school	5,120	0.00	0.88	–1.39	2.58
Socioeconomic status composite	12,360	0.00	0.73	–0.57	4.04
Activity: Time spent studying geo topics	12,500	0.00	0.87	–1.66	1.94
Activity: Text-based instruction	12,480	0.00	0.90	–1.42	2.14
Activity: Process-oriented instruction	12,460	0.00	0.87	–0.83	3.32
Activity: Apply geo practices	12,050	0.00	0.94	–1.60	2.38
Activity: Assessment variety	11,900	0.00	0.87	–1.98	2.05
Activity: Computer-mediated process-oriented instruction	11,460	0.00	0.91	–1.19	2.77
Activity: Composite of geo activities	10,970	0.00	0.96	–2.03	3.42

Note: Sample size is unweighted and rounded to nearest 10. Mean and standard deviation are unweighted statistics. ELL = English language learner; IEP = individualized education plan. Source: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) Geography, 2018.

Table 2. 2018 National Assessment of Educational Progress (NAEP) Geography descriptive statistics for all dichotomous Level 2 predictors derived from the teacher questionnaire

Predictor variable	N	M	SD	Minimum	Maximum
Undergraduate geo major	1,330	0.03	0.21	0	1
Undergraduate geo minor	1,330	0.09	0.34	0	1
Graduate geo major	1,000	0.02	0.17	0	1
Graduate geo minor	1,000	0.04	0.27	0	1
Taken graduate courses	1,600	0.64	0.57	0	1
Professional development: Took college course	1,490	0.16	0.42	0	1
Professional development: Workshop/training	1,590	0.64	0.57	0	1
Professional development: Mentoring	1,540	0.41	0.57	0	1
Professional development: Co/team teaching	1,390	0.28	0.54	0	1
Digital device provided for teachers’ use	1,630	0.74	0.54	0	1
Digital device provided for students’ use	1,630	0.84	0.49	0	1
Discrete social studies curriculum	1,630	0.33	0.52	0	1
Combined social studies curriculum	1,630	0.48	0.56	0	1
Integrated social studies curriculum	1,630	0.07	0.28	0	1
Teacher teaches geography	1,620	0.55	0.57	0	1
Attended professional development program	1,020	0.17	0.53	0	1
Female	1,620	0.45	0.55	0	1
Hispanic	1,630	0.12	0.35	0	1
White	1,630	0.77	0.53	0	1
Black	1,630	0.08	0.29	0	1
Asian	1,630	0.02	0.15	0	1
American Indian, Alaskan Native	1,630	0.03	0.17	0	1
Native Hawaiian, Pacific Islander	1,630	0.01	0.08	0	1

Note: Sample size is unweighted and rounded to nearest 10. Mean and standard deviation are unweighted statistics. Source: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) Geography, 2018.

Table 3. 2018 National Assessment of Educational Progress (NAEP) Geography descriptive statistics for ordinal and continuous Level 2 predictors derived from the teacher questionnaire

Predictor variable	N	M	SD	Minimum	Maximum
Ordinal
Computer tech: already proficient	1,530	1.74	2.11	0	5
Computer tech: no training	1,530	0.82	1.45	0	5
Computer tech: had training	1,530	1.59	2.06	0	5
School computer technology: poor quality	1,550	1.38	1.20	1	4
Teachers discuss student progress	1,600	3.22	2.03	1	5
% time teaching civics/government	1,570	2.72	2.08	1	6
% time teaching geo	1,540	2.18	1.79	1	6
% time teaching U.S. history	1,600	4.06	2.68	1	6
% time teaching other social studies	960	1.06	2.28	1	6
Use multiple-choice or true–false test	1,610	2.69	1.64	1	5
Use written response assessment	1,590	2.12	1.66	1	5
Years worked as elementary or secondary teacher	1,630	2.97	2.08	0	5
Years social studies teaching experience (Grades 6–12)	1,630	2.57	1.99	0	5
How often team teach	1,620	1.57	1.61	1	5
How often observe other teachers	1,620	1.58	1.21	1	5
Students can be more intelligent	1,620	3.19	2.03	1	5
Students can improve effort and behavior	1,620	3.94	2.23	1	5
Continuous
Time spent teaching social studies	1,070	157.59	304.61	0.00	2,460
Continuous/factor score
School conditions	1,590	0.01	0.87	–0.98	3.06
Job satisfaction	1,610	0.01	0.92	–3.45	0.95
Practice text-based instruction in social studies class	1,500	0.01	0.78	–2.98	1.83
Practice authentic instruction and assessment in social studies class	1,600	0.01	0.88	–1.75	2.17
Practice computer-mediated instruction in social studies class	1,480	0.01	0.86	–1.94	2.40
Use written assessments in social studies class	1,580	–0.02	0.84	–2.88	2.28
Apply assessment results	1,580	0.04	0.97	–1.87	1.80
Teach geo topics	940	0.01	0.91	–2.20	1.91
Teach geo practices	910	0.01	0.95	–2.32	2.40
Students test well because they studied, applied effort, were taught well	1,560	–0.02	0.93	–3.71	1.64
Students test well because just guessed, always do well in social studies.	1,560	0.00	0.85	–2.09	3.30

Note: Sample size is unweighted and rounded to nearest 10. Mean and standard deviation are unweighted statistics. Source: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) Geography, 2018.

Table 4. 2018 National Assessment of Educational Progress (NAEP) Geography descriptive statistics for all Level 2 predictors derived from the school questionnaire and Education Demographic and Geographic Estimates (EDGE) school neighborhood income-to-poverty ratio (IPR) estimates data file

Predictor variable	N	M	SD	Minimum	Maximum
Dichotomous
Students take geo in Grade 8	1,710	0.13	0.36	0	1
Students take geo before Grade 8	1,710	0.49	0.54	0	1
Offer professional development: Improve general instruction	1,710	0.68	0.52	0	1
Offer professional development: Improve reading and writing instruction	1,710	0.88	0.42	0	1
Offer professional development: Improve social studies instruction	1,710	0.82	0.46	0	1
Offer professional development: Use tech in classroom	1,710	0.80	0.47	0	1
Help students: tutoring	1,670	0.64	0.55	0	1
Help students: instructional aides in class	1,690	0.42	0.53	0	1
Help students: instructional aides for pull-out instruction	1,680	0.28	0.48	0	1
Help students: extra work/homework	1,670	0.64	0.54	0	1
Help students: Before- and after-school programs	1,680	0.55	0.55	0	1
Help students: summer school	1,670	0.32	0.50	0	1
Help students: ELL	1,660	0.72	0.53	0	1
Northeast	1,840	0.18	0.39	0	1
Southeast	1,840	0.27	0.44	0	1
Central	1,840	0.15	0.36	0	1
West	1,840	0.4	0.49	0	1
Private school	1,840	0.09	0.28	0	1
City	1,840	0.34	0.47	0	1
Suburb	1,840	0.45	0.50	0	1
Town	1,840	0.08	0.28	0	1
Rural	1,840	0.12	0.33	0	1
Ordinal
% FRLE	1,840	5.04	2.20	0	8
% ELL	1,840	1.79	1.33	0	7
Percentage
% Black	1,840	14.81	20.39	0	100
% Hispanic	1,840	30.45	28.77	0	100
Continuous/factor score
Social studies curriculum based on standards/assessments	1,300	0.08	0.90	–2.20	1.66
Social studies curriculum based on local control	1,300	0.01	0.84	–3.42	1.86
IPR	1,650	301.74	222.46	81	956

Note: Sample size is unweighted and rounded to nearest 10. Mean and standard deviation are unweighted statistics. ELL = English language learner; FRLE = free and reduced-price lunch eligibility; IPR = income-to-poverty ratio. Source: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) Geography, 2018.

Figure 1 Baseline model (A) Level 1 and (B) Level 2 predictors of geography achievement in the United States, 2018, based on an analytic sample size of 650 schools, 1,560 teachers, and 10,830 students. Note: Effect sizes are conditional on all retained Level 1 and 2 predictors. ELL = English language learner; IEP = individualized education plan; FRLE = free and reduced-price lunch eligibility. *p < 0.05. **p < 0.01. ***p < 0.001. Source: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) Geography, 2018.

Table 5. Specifications for retained National Assessment of Educational Progress (NAEP) Geography predictor variables, 2018

Level	Variable category	Description
Student	Sex	1 dummy code: female (reference category: male)
	Race and ethnicity	6 dummy codes: Black, Hispanic, Asian, Native Hawaiian/Pacific Islander, American Indian/Alaskan Native, two or more races (reference category: White)
	Free/reduced-price lunch eligibility	2 dummy codes: eligible, information not available (reference category: free/reduced-price lunch not eligible)
	English language learner (ELL)	1 dummy code: ELL classification, referring to a student who is learning the English language in addition to their native or other language (reference category: not ELL student)
	Individualized education plan (IEP)	1 dummy code: IEP classification, referring to a student with an identified disability that qualifies for specialized instruction (reference category: student not on an IEP)
	Above modal age	1 dummy code (reference category: at modal age)
	Books in home	Ordinal: 0–10 books (0), 11–25 books (1), 26–100 books (2), > 100 books (3)
	Parental education	Ordinal: Did not finish high school (–1), graduated from high school (0), attended some college (1), graduated from college (2)
School	Urbanicity	3 dummy codes: school location in suburb, town, rural (reference category: city)
	Region	3 dummy codes: school location in U.S. Southeast, Central, West (reference category: Northeast)
	School composition	4 calculated percentages: 0–100% Black, Hispanic, ELL, free/reduced lunch eligible
	Private school	1 dummy code (reference category: public school)

Source: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) Geography, 2018.

Table 6. Specifications for National Assessment of Educational Progress (NAEP) Geography Level 1 opportunity to learn (OTL) predictor variables (statistically significant only), 2018

Level	Variable	Description
Student	Took geo class before Grade 8	1 dummy code: had geography class/course before Grade 8 (reference category: did not take geography before Grade 8)
	Activity: Geo practices	Factor score: During this school year, how often do you do each of the following activities when you study geography? Use a map (e.g., a road map, MapQuest, or Google Maps), atlas, or globe to locate different cities, countries, and continents. Examine how natural features of Earth (e.g., rivers or mountains) are formed. Examine why groups of people migrate to different parts of the world. Examine how humans affect the natural environment. Examine how humans adjust to the natural environment. Examine why different food and energy resources are traded globally. Use geographic information (e.g., maps, satellite images, or population data) to answer a question or solve a problem. Answer choices: never or hardly ever, once in a while, sometimes, often, always or almost always
	Activity: Study topics	Factor score: In your social studies class this year, how much have you studied the following topics? Use of physical or digital maps (e.g., a road map, MapQuest, or Google Maps) and globes. Natural resources (e.g., oil, forests, or water). Countries and cultures. Environmental issues (e.g., pollution, recycling, climate change, or genetically modified food). Answer choices: not at all, very little, some, quite a bit, a lot
	Activity: Geo process	Factor score: During this school year so far, how often have you done each of the following activities? Gone on a class field trip to learn about geography topics. Given class presentations on geography topics. Written about your opinion on geographic problems or issues (e.g., in a letter, e-mail, or blog post). Taken part in environmental debates or panel discussions. Answer choices: never, once, two or three times, four or five times, more than five times
	Activity: Computer-mediated geo process	Factor score: When you study social studies, how often do you use computers or other digital devices to do the following? For this question, include both schoolwork and homework assignments. Play games or use apps in which you play a role (e.g., an explorer, geographer, or historian). Organize information about geography topics by creating tables, charts, or graphs. Create reports or projects about geography using different forms of media (e.g., a slide presentation that combines text and video clips). Participate in online discussions about geography on a Web site (e.g., in forums or social media). Use the Internet to look for evidence or sources (e.g., text documents, photographic images, or films). Answer choices: never, about once or twice a year, about once or twice a month, about once or twice a week, every day or almost every day
	Days absent	How many days were you absent from school in the last month? Ordinal: none (0), 1 or 2 days (1), 3 or 4 days (2), 5 to 10 days (3), more than 10 days (4)
	Use computer tech in classes	In this school year, how often did you use a laptop, desktop computer, or tablet during your classes at school? Ordinal: never (0), in some classes (1), in about half of the classes (2), in more than half of the classes (3), in all or almost all classes (4)

Source: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) Geography, 2018.

Table 7. Specifications for National Assessment of Educational Progress (NAEP) Geography Level 2 opportunity to learn (OTL) predictor variables (statistically significant only), 2018

Level	Variable	Description
Teacher/school	School computers and devices_Poor quality	How well do the desktop computers, laptop computers, or tablets perform in your schoolwork? (Teacher-reported) Ordinal: All computers or tablets are functional and operate quickly (1); all computers or tablets are functional, but some run more slowly than others (2); all computers or tablets are functional, but all or almost all run slowly (3); some of the computers or tablets do not operate and cannot be used (4)
	Years of social studies teaching	Excluding student teaching, how many years have you taught civics, geography, history, or social studies in Grades 6 through 12, counting this year? (Teacher-reported) Ordinal: less than 1 year (0), 1–2 years (1), 3–5 years (2), 6–10 years (3), 11–20 years (4), 21 or more years (5)
	Why students test well in social studies	Factor score: Suppose your students did very well on their last social studies test. How likely do you think each of the following explanations is in this situation? (Teacher-reported) My students did well because they studied and were prepared. My students did well because they put in a lot of effort. My students did well because I taught the concepts well. Answer choices: not at all likely, not likely, somewhat likely, quite likely, extremely likely
	School professional development: Improve social studies instruction	This year, have the following professional development programs (e.g., training sessions and workshops, including online classes) been offered in your school for social studies teachers? (School-reported) Programs aimed at developing and improving practices for social studies instruction. Dummy code: Yes (1) (reference category: No (0))
	School professional development: Improve reading and writing instruction	This year, have the following professional development programs (e.g., training sessions and workshops, including online classes) been offered in your school for social studies teachers? (School-reported) Programs aimed at developing and improving general teaching of critical reading and writing skills. Dummy code: Yes (1) (reference category: No (0))

Source: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) Geography, 2018.

Figure 2 Expanded model (A) Level 1 and (B) Level 2 predictors of geography achievement in the United States, 2018, based on analytic sample size of 340 schools, 980 teachers, and 6,620 students. Note: Effect sizes are conditional on all retained and newly added Level 1 and 2 predictors. Newly added predictor variables appear above the dashed line. FRLE = free and reduced-price lunch eligibility; ELL = English language learner; IEP = individualized education plan; IPR = income-to-poverty ratio. *p < 0.05. **p < 0.01. ***p < 0.001. Source: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) Geography, 2018.

Figure 3 Net change in student-level effects after controlling for OTL and school neighborhood poverty. Note: ELL = English language learner; IEP = individualized education plan. *p < 0.05. **p < 0.01. ***p < 0.001. Source: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP) Geography, 2018.