Teaching, Learning and Assessing Statistical Problem Solving

Figures & data

Figure 1. The statistical problem solving approach

Table 1: Revised Taxonomy and the Problem Solving Approach

Stage of problem solving approach	Descriptor (from the QCA specification)	Level in taxonomy (from the revised taxonomy)
1. Specify the problem and plan	formulate questions in terms of the data needed, and consider what inferences can be drawn from the data; decide what data to collect (including sample size and data format) and what statistical analysis is needed	Remember: recognising; recalling Understand: interpreting; exemplifying; explaining Apply: executing Analyse: differentiating; organising Create: planning; producing; generating
2. Collect data	collect data from a variety of suitable sources, including experiments and surveys, and primary and secondary sources	Remember: recognising; recalling Understand: classifying; comparing Apply: executing; implementing Analyse: organising Create: planning
3. Process and represent the data	turn the raw data into usable information that gives insight into the problem	Remember: recognising; recalling Understand: interpreting; exemplifying; classifying; summarising Apply: executing; implementing Analyse: differentiating; organising
4. Interpret and discuss the data	answer the initial question by drawing conclusions from the data	Remember: recalling Understand: interpreting; exemplifying; summarising; inferring; comparing; explaining Analyse: differentiating; organising; attributing Evaluate: checking; critiquing Create: generating; producing

Figure 2. Schematic diagram of the problem solving approach

Figure 3 Introductory slide in “How safe is your area?”

Figure 4 Initial slide for the PLAN stage

Figure 5 Different approaches to the problem

Figure 6 Formulation of the PLAN

Figure 7 Consideration of data sources

Figure 8 The use of the PSA being reinforced

Figure 9 The questionnaire

Figure 10 The summary sheet

Figure 11 Processing the data

Figure 12 Comparisons

Figure 13 Discussion summary

Figure 14 Template for the first task in the assessment

Table 2: Questions in the on line assessment ‘Getting the Best Deal’

Question	Task
A. Overview of the problem solving approach
A0	The descriptions of the stages of the cycle must be dragged and dropped into the correct position.
A1	Explain why the process is called a cycle and why that description is important.
B. Ayesha's investigation Ayesha has been told she now has to pay for her own mobile phone. She wants to use what she has learned about the problem solving approach from her math teacher to achieve this. She has written up her investigation in rough using ten statements.
B0	The ten statements, describing Ayesha's investigation, must be dragged and dropped into their correct positions in the cycle.
With reference to the ‘plan’ statements in Ayesha's investigation
B1	Look at the four statements and then indicate whether someone else could read this plan and then carry it out for themselves (Yes, No or Don't know)†.
B2	If they answer ‘no’ to B1, describe what other information they would need (by typing into a text box)‡.
B3	Indicate which of the statements states the problem Ayesha is trying to solve (‘Don't know’ is one possible answer).
B4	Suggest another method Ayesha could use to choose a day at random (text box).
Question	Task
With reference to the ‘collect’ statements in Ayesha's investigation
B5	What is the mean length of Ayesha's phone calls (text box)?
B6	Comment on her choice of shop (text box).
B7	Would collecting more information on different deals help? Explain your answer (text box).
With reference to the ‘process’ statements in Ayesha's investigation
B8	The process statements used by Ayesha are presented showing her calculations. Why did Ayesha use the number 30 in the calculations (text box)?
B9	Do you think it is reasonable to expect she'll use her phone the same amount each month (Yes, No or Don't know)? Explain your answer (text box).
B10	Are there other things you think she should do with the data (text box)?
With reference to the ‘discuss’ statements in Ayesha's investigation
B11	Would you advise Ayesha to change to the deal given that the calculations are correct (Yes, No or Don't know)? Why (text box)?
B12	Ayesha thinks that £56 is a lot of money and 840 minutes is a long time. It is based on data from one day. What might have happened on this day for it to be higher than normal (text box)?
B13	What could Ayesha do to check if her use on this day is similar to other days or not (text box)?
C. Andy's investigation Andy considers advertisements for two phones that only differ in the amount of time and number of texts that are included in the package. The two adverts are displayed.
With reference to the ‘plan’ stage of Andy's investigation
C1	Andy's friends say that the deals are the same. Do you agree (Yes, No or Don't know)?
C2	Can you give an example of when one deal might be better than the other (text box)?
Andy decides to investigate how much he uses his phone to see which is the better deal. The candidates' task is to help him with his investigation. (After the candidates have committed to an answer they cannot return to it later in the light of what Andy actually decided. They are told that their responses will be judged on the basis of whether their reasons and explanations are correct, not on whether they choose the same course of action as Andy.)
C3	What data do you think he should collect (text box)?
C4	Over what number of days, weeks or months should he collect data? Explain your choice (text box).
C5	How could he collect the data (text box)?
A handwritten statement of Andy's problem and plan are then presented for consideration.
C6	Why does he decide to choose a bill at random (text box)?
C7	Why are his mobile phone bills a good way to collect the data (text box)?
With reference to the ‘collect’ stage of Andy's investigation
A table of the data that Andy decided to collect (number of calls, total call duration and number of texts for each of 31 days).
C8	The call durations are in seconds and look messy. Suggest how Andy could make this better (text box).
With reference to the ‘process’ stage of Andy's investigation
He decides to change the seconds to minutes and round them to the nearest half minute
C9	Andy has done the first few, can you do the rest below? (Four numerical data boxes are there for the answers to be entered and an on screen calculator, together with instructions for its use, is also available.)
Question	Task
C10	One of the rows in the table doesn't look right to Andy (the row is presented). Can you spot why (text box)?
C11	Can you explain why it's happened (text box)?
C12	Is it a problem (text box)?
C13	Complete the frequency table for the number of calls made each day for the month. (Numerical data boxes labeled 0, 1, …, 12 are there for the answers to be entered.)
C14i	Look at the chart below (line graph of daily call duration). Mark whether you think it is helpful or unhelpful (or not sure) in deciding which package to use.
C14ii	Look at the chart below (bar chart for number of calls per day). Mark whether you think it is helpful or unhelpful (or not sure) in deciding which package to use.
C14iii	Look at the chart below (pie chart for number of calls per day). Mark whether you think it is helpful or unhelpful (or not sure) in deciding which package to use.
C14iv	Look at the chart below (scatter plot of number of calls vs number of texts). Mark whether you think it is helpful or unhelpful (or not sure) in deciding which package to use.
C15	Candidates are asked to undertake calculations for the number of calls made daily. (Numerical data boxes labeled mean, median, mode and range are there for the answers to be entered.) On screen calculator is provided.
With reference to the ‘discuss’ stage of Andy's investigation
C16	What does the mode tell you about how Andy uses his phone (text box)?
C17	How long does he spend on the phone in the month in all (text box)? Candidates can use the data and charts on previous screens.
C18	Which package do you think he should choose and why (text box)? Superimposed line graphs of number of texts and number of calls for a different month are presented. An anomaly is highlighted.
C19	Can you give a possible explanation for what happened here (text box)?
Referring back to the ‘plan’ stage of Andy's investigation
Andy had noticed that lots of phone deals seemed to suggest that the more texts you send the fewer calls you make. Two sample deals are presented.
C20	Write a plan for an investigation to look at whether it is true that the more texts a person sends, the fewer phone calls they make.

† Where ‘Yes, No or Don't know’ is indicated there are three check boxes.

‡ Where ‘text box’ is indicated the candidates are invited to “Type your answers here”.

Table 3: Assessment sheet for Getting the Best Deal

Domain	Mark	Question	Description	Comment	Mark
Holistic view	0, 1, 2	A D&D	Places descriptions in correct locations.
	0, 1, 2	A1	Clear statement of what cycle means. and why it is important.
	0, 1, 2	B D&D	Places statements into correct stages. of the cycle.
Ayesha
Plan	0, 1, 2	B1 & B2	Gives a clear justification for choice of response to B1.
	0, 1, 2	B3	Correctly identifies the statement that is the problem to be solved.
	0, 1, 2	B4	Gives a clear description of an alternative method that could be used to choose a day at random.
Collect	0, 1, 2	B5	Correctly calculates the mean.
	0, 1, 2	B6	Clearly comments on the choice made for the shop.
	0, 1, 2	B7	Presents clear discussion of whether collecting more information would help.
Process	0, 1, 2	B8	Clearly states why the number 30 was used.
	0, 1, 2	B9	Gives a clear justification for choice of response to whether phone use could be expected to be the same in other months.
	0, 1, 2	B10	Clear statement of other uses that should be made of the data.
Discuss	0, 1, 2	B11	Gives a clear justification for choice of advice on whether deal should be accepted.
	0, 1, 2	B12	Gives a clear statement of possible reasons for data to be an outlier.
	0, 1, 2	B13	Makes a clear statement of how data should be checked.
Andy
Plan	0, 1, 2	C1	Selects the correct answer.
	0, 1, 2	C2	Clearly states an example of how one of the deals could be better.
	0, 1, 2	C3	Clearly states which variables should be measured.
	0, 1, 2	C4	Gives clear explanation of the choice of time period over which data should be collected.
	0, 1, 2	C5	Clearly describes how the data could be collected.
	0, 1, 2	C6	Gives a clear explanation of why a bill is chosen at random.
	0, 1, 2	C7	Gives a clear explanation of why phone bills are a good data source.
Collect	0, 1, 2	C8	Makes a clear suggestion of how data tabulation can be made less messy.
Process	0, 1, 2	C9	Correctly calculates call durations.
	0, 1, 2	C10	Correctly identifies mistake in data.
	0, 1, 2	C11	Gives clear explanation of why mistake is there.
	0, 1, 2	C12	Gives clear statement of whether this poses a problem.
	0, 1, 2	C13	Correctly completes the frequency table.
	0, 1, 2	C14i	Makes a sensible response to question about time series plot.
	0, 1, 2	C14ii	Makes a sensible response to question about bar chart.
	0, 1, 2	C14iii	Makes a sensible response to question about pie chart.
	0, 1, 2	C14iv	Makes a sensible response to question about scatter plot.
	0, 1, 2	C15	Correctly calculates summary statistics.
Discuss	0, 1, 2	C16	Gives clear interpretation of the mode.
	0, 1, 2	C17	Gives correct interpretation of results from frequency table.
	0, 1, 2	C18	Clearly states preferred choice of package and clearly explains reasons.
	0, 1, 2	C19	Gives a clear interpretation of the new graphical information.
Plan	0, 1, 2	C20	Constructs a clear plan for a further investigation.
A mark of 0, 1 or 2 is allocated for each question asked of the candidates. These correspond to the responses being incorrect (0), partially correct (1) or correct (2).

Table 4: Test scores by domain

Domain	Minimum	Q1	Median	Q3	Maximum
Holistic	0.00	50.00	50.00	83.33	100.00
Plan	0.00	27.27	45.45	59.09	90.91
Collect	25.00	50.00	68.75	87.50	100.00
Process	3.85	11.54	19.23	46.15	88.46
Discuss	0.00	12.50	21.43	42.86	71.43

Figure 15 Interaction plots for percentage question scores