Critical views of 8th grade students toward statistical data in newspaper articles: Analysis in light of statistical literacy

Figures & data

Figure 1. A model for statistical literacy (Gal, 2004a, p. 67).

Table 1. Critical questions

1. Where did the data (on which this statement is based) come from? What kind of study was it? Is this kind of study reasonable in this context?

2. Was a sample used? How was it sampled? How many people actually participated? Is the sample large enough? Did the sample include individual units that are representative of the population? Is the sample biased in some way? Overall, could this sample reasonably lead to valid inferences about the target population?

3. How reliable or accurate were the instruments or measures (tests, questionnaires, interviews) used to generate the reported data?

4. What is the shape of the underlying distribution of raw data (on which this summary statistic is based)? Does it matter how it is shaped?

5. Are the reported statistics appropriate for this kind of data? For example, was an average used to summarize ordinal data? Is a mode a reasonable summary? Could outliers cause a summary statistic to misrepresent the true picture?

6. Is a given graph drawn appropriately, or does it distort trends in the data?

7. How was this probabilistic statement derived? Are there enough credible data to justify the estimate of likelihood given?

8. Overall, are the claims made here sensible and supported by the data? For example, is correlation confused with causation, or a small difference made to loom large?

9. Should additional information or procedures be made available to enable me to evaluate the sensibility of these arguments? Is something missing? For instance, did the writer “conveniently forget” to specify the base of a reported percent-of-change or the actual sample size?

10. Are there alternative interpretations for the meaning of the findings or different explanations for what caused them; e.g. an intervening or a moderating variable affecting the results? Are there additional or different implications that are not mentioned?

Table 2. Distribution of objectives for statistics according to grade level in the middle school teaching program

Grade	Objective
6th	(1) Create research questions for a problem, identify an appropriate sample, and collect data (2) Interpret and display the data in their statistical representation forms (3) Explain the situations resulting from incorrect interpretations of column charts (4) Interpret data through calculating arithmetic average and range (5) Make assumptions based on the data
7th	(1) Make and interpret column and line charts according to more than one criterion (2) Make and interpret a pie chart (3) Form opinions about real-life situations by interpreting and making statistical presentations (4) Make assumptions based on data (5) Explain the situations resulting from incorrect interpretations of line, picture, and figure charts (6) Calculate mean, median and quartiles (7) Interpret the measures of central tendency and quartiles
8th	(1) Form and interpret a histogram (2) Calculate standard deviation (3) Develop opinions about real-life situations by using appropriate statistical presentation forms, measures of central tendency, and standard deviation

Table 3. Selected newspaper articles

Article 1	One in three smart ticket holders has an automobile
Article 2	The most comfortable shopping city of Europe is Istanbul
Article 3	The number of ATMs has increased twice in 6 years
Article 4	Twenty thousand new mobile customers per month

Table 4. Distribution of the critical questions with respect to dimensions

Categories	Questions
Consistency	Questions 1 and 8
Sampling	Question 2
Data collection	Question 3
Data analysis	Questions 4 and 5
Data presentation	Questions 6 and 7
Data support	Questions 9 and 10

Table 5. Critical questions used by students pertain to newspaper articles

		S1				S2				S3				S4				S5				S6				S7				S8				S9
	Critical questions	1	2	3	4	1	2	3	4	1	2	3	4	1	2	3	4	1	2	3	4	1	2	3	4	1	2	3	4	1	2	3	4	1	2	3	4
Consistency	1a. What kind of study is it?	✓	✓			✓	✓	✓		✓	✓	✓			✓	✓			✓				✓	✓	✓		✓		✓	✓	✓	✓	✓		✓		✓
	1b. Is this kind of study reasonable in this context?					✓			✓				✓						✓			✓					✓		✓	✓	✓		✓	✓			✓
	8. Are the claims made here meaningful and supported by the data?							✓	✓			✓	✓						✓	✓			✓	✓	✓					✓	✓	✓		✓	✓		✓
Sampling	2a. How was it sampled?									✓		✓	✓			✓	✓	✓		✓	✓			✓	✓	✓	✓				✓	✓	✓	✓	✓	✓
	2b. How many people actually participated?	✓	✓							✓	✓✓							✓	✓			✓		✓	✓	✓	✓		✓	✓				✓		✓	✓
	2c. Is the sample large enough?							✓				✓		✓							✓					✓			✓	✓
	2d. Could this sample reasonably lead to valid inferences about the target population?																									✓										✓
Data collection	3. How reliable were the instruments or measures (tests, questionnaires, and interviews) used to generate the reported data?													✓	✓					✓	✓		✓		✓	✓		✓			✓	✓	✓	✓	✓
Data analysis	4. How was the data distribution accomplished?									✓	✓	✓										✓	✓										✓				✓
	5a. Are the reported statistics appropriate for this kind of data?									✓	✓	✓										✓		✓								✓	✓				✓
	5b. Could outliers cause a summary statistic to misrepresent the true picture?						✓								✓					✓	✓
Data presentation	6. Is a given graph (or table) drawn appropriately, or does it distort trends in the data?		✓		✓																✓														✓		✓
	7. How was this probabilistic statement derived? Are there enough credible data to justify the estimate of likelihood given?
Data support	9. Should additional information or procedures be made available to enable me to evaluate the sensibility of these arguments? Is something missing?	✓	✓			✓			✓		✓	✓		✓		✓		✓	✓					✓	✓	✓	✓	✓	✓		✓	✓	✓			✓	✓
	10. Are there alternative interpretations for the meaning of the findings or different explanations for what caused them?					✓	✓	✓	✓		✓		✓						✓				✓					✓	✓	✓			✓			✓	✓

Notes: 1: Article 1, 2: Article 2, 3: Article 3, 4: Article 4.

Table 6. Frequency of critical questions usage related to the consistency dimension

	Article 1		Article 2		Article 3		Article 4
	f	%	f	%	f	%	f	%
1a. What kind of study is it?	4	44	9	100	5	55	4	44
1b. Is this kind of study reasonable in this context?	4	44	3	33	0	0	5	55
8. Are the claims made here meaningful and supported by the data?	2	22	4	44	5	55	4	44

Table 7. Frequency of critical questions usage related to the sampling dimension

	Article 1		Article 2		Article 3		Article 4
	f	%	f	%	f	%	f	%
2a. How was it sampled?	4	44	3	33	6	66	5	55
2b. How many people actually participated?	7	77	4	44	2	22	3	33
2c. Is the sample large enough?	3	33	0	0	3	33	3	33
2d. Could this sample reasonably lead to valid inferences about the target population?	0	0	0	0	0	0	2	22

Table 8. Frequency of critical questions usage related to the data collection dimension

	Article 1		Article 2		Article 3		Article 4
	f	%	f	%	f	%	f	%
3. How reliable were the instruments or measures (tests, questionnaires, and interviews) used to generate the reported data?	3	33	4	44	3	33	3	33

Table 9. Frequency of critical questions usage related to the data analysis dimension

	Article 1		Article 2		Article 3		Article 4
	f	%	f	%	f	%	f	%
4. How was the data distribution accomplished?	2	22	2	22	1	11	2	22
5a. Are the reported statistics appropriate for this kind of data?	2	22	1	11	3	33	2	22
5b. Could outliers cause a summary statistic to misrepresent the true picture?	0	0	2	22	1	11	1	11

Table 10. Frequency of critical questions usage related to the data presentation dimension

	Article 1		Article 2		Article 3		Article 4
	f	%	f	%	f	%	f	%
6. Is a given graph drawn appropriately, or does it distort trends in the data?	0	0	2	22	0	0	3	33
7. How was this probabilistic statement derived? Are there enough credible data to justify the estimate of likelihood given?	0	0	0	0	0	0	0	0

Table 11. Frequency of critical questions usage related to the data support dimension

	Article 1		Article 2		Article 3		Article 4
	f	%	f	%	f	%	f	%
9. Should additional information or procedures be made available to enable me to evaluate the sensibility of these arguments? Is something missing?	3	33	6	66	6	66	6	66
10. Are there alternative interpretations for the meaning of the findings or different explanations for what caused them?	2	22	4	44	3	33	5	55

Gal, I. (2004a). Statistical literacy: Meanings, components, responsibilities. In D. Ben-Zvi & J. Garfield (Eds.), The challenge of developing statistical literacy, reasoning and thinking (pp. 47–78). Dordrecht: Kluwer Academic.

 Google Scholar