Interview with Bob delMas

Beginnings

AR: Thanks very much, Bob, for agreeing to be interviewed for the Journal of Statistics and Data Science Education. Let’s start by taking you back to age 18: Where were you, and what were your career aspirations at that point?

RD: At age 18, I was in my first and second years of college as an undergraduate at the University of Minnesota. I always had a strong interest in science and an aptitude for mathematics. I was someone that my fellow classmates in grade school and high school sought out for help in these areas, so I also demonstrated an ability to explain and guide in understandable ways. When I considered going to college, I initially intended to major in a science area that combined my academic interests, such as biochemistry. However, I came to realize that I was very much interested in a type of epistemological question: Why do I and others seem able to learn science and mathematics, whereas others find these disciplines challenging? This led me to declare Psychology as a major as a first-year student, and then switch to Child Development in my sophomore year, a decision that set the stage for my academic future.

AR: Did you already have an academic career in mind as an undergraduate? Did that same epistemological question lead you to graduate school as soon as you finished college?

RD: I cannot say that I had an academic career in mind as an undergraduate, other than I was certain that I would go to graduate school. Neither of my parents went to college, and none of my relatives who graduated from college had an advanced degree or had a career in an academic setting, so I did not have any models to go on other than what I experienced in college. After earning my undergraduate degree in Child Psychology, I initially applied to several Child Development graduate programs. I was accepted at Cornell University but decided not to attend for financial reasons.

I then spent two years in jobs that helped me realize that my undergraduate degree did not credential me to pursue my interests. During that time, I researched a variety of graduate programs and realized that Educational Psychology programs had coursework and research opportunities that were aligned with my interests. I was accepted into the Educational Psychology program at the University of Minnesota, where I took coursework in learning theory, cognitive psychology and decision-making, and a supporting program in ethology. I also extended my coursework in advanced statistical methods and my experience in conducting statistical analyses through research positions.

AR: Did you do any teaching as a graduate student?

RD: I was the instructor for only one course as a graduate student, and that was a Master’s level introductory statistics course in the Educational Psychology Department. I also served as a teaching assistant and later the head teaching assistant for an introduction to educational psychology course that was taken primarily by students in the teacher preparation program.

Dissertation and Academic Career

AR: What did you study for your dissertation?

RD: My interests in advanced statistical methods, cognition, and decision making blended together when I learned about misconceptions students have about probability and statistics, such as the gambler’s fallacy and the so-called law of small numbers. At that time, I met Joan Garfield, who was ahead of me in the Educational Psychology graduate program and had similar interests. We both read Judgment Under Uncertainty: Heuristics and Biases. Through reading this collection and other literature in the cognitive sciences, along with many discussions with Joan, I started forming a thesis for instructional conditions that might help students to overcome some of these misconceptions. My thesis was titled “The Effect of Activity-Based Instruction and Directed Evaluation of Predictions on Misconceptions of Probability.”

As part of my dissertation research, students enrolled in an undergraduate introductory statistics course were randomly divided into two groups, an evaluation group and a no-evaluation group. We arranged for the two groups to meet in different classrooms during their scheduled class session. All students received the same instruction on probability through a take-home workbook that I prepared. I developed an activity about tossing a coin 100 times, and students in both groups were prompted to guess the outcome before each toss. A random sequence of heads and tails was generated, and that sequence was called out in both group sessions. Those in the no-evaluation group did not record their guess for a toss, but they did record the outcome. Those in the evaluation group recorded both their guess and the outcome of the toss. Based on an analysis of pretest and posttest responses, the study results provided strong evidence of an increase in misconceptions among students in the no-evaluation condition and a decrease in misconceptions among the evaluation group, consistent with predictions from cognitive theories of confirmation bias and conceptual change.

One thing I really liked about the study is that it demonstrated that active learning and experience are not necessarily enough to produce desired learning outcomes – awareness of misconceptions and relevant learning theory is also needed to design effective learning interventions. In this case, directing students to evaluate their beliefs and expectations was needed. Many of the learning interventions I have developed with colleagues include formative assessment opportunities for students to test and evaluate their understanding of statistical concepts that are prone to misconceptions and misunderstandings. These concepts include sampling variability, sampling distributions, confidence intervals, and p-values.

AR: I will come back and ask more about your learning intervention studies later, but for now let’s continue with your biography. Were you committed to an academic path as you completed your dissertation? Where did that lead?

RD: Yes, I was committed to an academic path as I completed my graduate studies. Toward the end of my graduate program, I was hired by Joan Garfield into a graduate research fellowship position in the Office of Research and Evaluation in the General College of the University of Minnesota. Upon completing her doctorate, Joan became a faculty member in the General College and served as the director of the office. My position turned into a postdoctoral position upon completion of my doctoral degree in 1988. During my three years in that office, I learned more about multivariate statistical models, such as multiple regression and repeated measures ANOVA, in order to conduct institutional research for the General College that looked at student and institutional factors associated with student attrition, persistence, academic performance, and degree completion.

My next academic position was as a faculty member of the Psychology Department at Northeast Missouri State University, which later changed its name to Truman State University. During my two years there in 1990-1992, I taught courses in introductory psychology and research methods, attempting to apply active learning methods in the courses. During that same time, the General College at the University of Minnesota decided to make the director of the Office of Research and Evaluation a full-time position, which they recruited me to fill. During my six years as director of that office, I continued to develop and extend my knowledge of research designs and statistical modeling methods, as well as co-develop and work on educational curriculum and research projects with Joan. I also taught a distance-learning version of an undergraduate statistics course for the University College.

Joan moved to the Department of Educational Psychology in 1995, which created a potential faculty position in the General College. I applied for and received a full-time assistant professor position, teaching courses in developmental mathematics and statistics. Joan and I continued to work on research projects together and I also taught advanced statistical methods courses from time to time for the Department of Educational Psychology. When Joan developed a statistics education research program, the first of its kind, in the Department of Educational Psychology, I applied for and was hired into an associate professor position in that program in 2006. That is where I remain to this day, working with colleagues and graduate students to develop active-learning courses in statistics and statistical methods at undergraduate and graduate levels, assessments for evaluating students’ statistical conceptual understanding, and research on the effect of the educational approaches we design.

AR: I have many follow-up questions, but a small tidbit in there caught my attention. I struggled to learn to teach courses remotely in 2020, so I’m wondering what your distance-learning course in the early 1990s was like.

RD: The distance learning course I taught in the 1990s was set up very differently from how we now teach remotely. It was essentially a correspondence course. The readings and assignments were designed to be identical to the corresponding in-person course. All assignments were carried out using a calculator. There were no video-recorded lectures. There was a course manual with printed lessons that the students purchased from the University bookstore and received by postal mail. Students could contact me by email (if they had an email account) or by phone with questions. All assignments were completed and mailed back to me by postal mail, which I then graded and sent back by postal mail. Students typically took six months to a year to complete the course.

Research on Learning Interventions

AR: Times have changed indeed. I’m very glad that COVID-19 was not COVID-91, as in 1991. Earlier you mentioned your dissertation finding about the importance of directing students to evaluate their beliefs and expectations. What other findings has your research into statistics education generated?

RD: Some of my earlier research, conducted with Joan Garfield and Beth Chance, focused on developing student understanding of sampling distributions. I developed simulation software that provided an interactive way for students to explore the relationships between sample size, sample distributions, and distributions of sample means, concepts related to the Central Limit Theorem. We originally developed an activity that had students generate distributions of sample statistics from different-shaped populations using different sample sizes, and then answer a set of guiding questions about making comparisons. While about 20% of students showed correct to good understanding about sampling distributions on a pretest, this increased to about 50% on a post-test. This improvement was encouraging, but many students were still demonstrating misunderstandings.

Literature on conceptual change indicated that people’s misconceptions can be stubborn and difficult to change, as people either ignore or discredit contradictory evidence. We modified the activities so that students directly tested their pretest responses. We also added instructions to direct their attention to all relevant information and comparisons needed to test their expectations. The new activity resulted in 72% of students demonstrating correct or good reasoning on the post-test. We had learned that it was not enough to have students interact with software and make observations; more students showed improved understanding when they confronted their misconceptions by making and testing predictions.

Under Joan’s leadership, we developed an entire introductory statistics curriculum based on these principles called Adapting and Implementing Innovative Materials in Statistics Courses Statistics (AIMS). Research on this curriculum demonstrated improvements in student understanding of other statistical inference concepts such as sampling error, p-value and confidence intervals.

AR: Please tell us more about this curriculum. What are its primary learning goals? What are some of its distinctive features? Could you give an example to illustrate some of these goals and features?

RD: The primary learning goal of the curriculum is the promotion of conceptual understanding of statistics (in contrast to treating statistics as a branch of mathematics, which it is not). The curriculum relies on the design of activities that engage students in the active production and exploration of data to develop conceptual understanding and statistical reasoning. As I described earlier, the lessons are designed to have students actively explore and test their understanding of statistical concepts, usually with the help of technology rather than through abstract mathematics.

I recommend that anyone who wants to get a sense of the AIMS curriculum read Developing Students’ Statistical Reasoning by Joan Garfield and Dani Ben-Zvi (Garfield and Ben-Zvi 2008, Springer). The first chapter states the learning goals and pedagogical approaches used in the curriculum. Part II presents many aspects of the curriculum we and others have developed. The chapters in Part II are organized around main areas of statistical conceptual understanding (e.g., Learning to Reason About Distribution, Learning to Reason About Variability, Learning to Reason About Statistical Inference). Each chapter in Part II first presents an example lesson activity, a rationale for the showcased activity, a review of relevant research literature on the conceptual area and implications for instruction, a proposed sequence of activities designed to develop conceptual understanding in the area, and descriptions of lessons designed to incorporate the activities. We used the same pedagogical principles to develop another curriculum based on randomization-methods called Change Agents for Teaching and Learning Statistics (CATALST; Garfield, delMas, and Zieffler 2012).

AR: What are some other findings from your research about teaching and learning statistics?

RD: One study I conducted with Yan Liu explored the difficulties students face in coordinating ideas of frequency, density, and center to develop an understanding of the standard deviation. We found that students initially presented simple, one-dimensional understandings of standard deviation that did not consider deviation about the mean. I designed a technology tool that let students explore the effects of the frequency of values relative to the mean on the size of the standard deviation. Using this tool, students moved toward a more mean-centered conceptualization that coordinated the effects of frequency (density) and deviation from the mean. This research is published in the Statistics Education Research Journal (delMas and Liu 2005).

Another project chronicled the development of a set of middle school activities with Helen Doerr and Katie Makar for supporting students’ understanding of distribution and variability. The development of the activities and of student understanding across the activities is detailed in an article we published in the Statistics Education Research Journal (Doerr, delMas, and Makar 2017). More information on the actual activities can be found in an article we published in Mathematics Teacher: Learning and Teaching Pre-K–12 (Makar, Doerr, and delMas 2020).

More recently I have been involved in a project led by my colleague Andrew Zieffler, in collaboration with several graduate students in our program, on the design of activities to foster an understanding of data science concepts among secondary mathematics teachers. Results from one study in this project are published in the Journal of Statistics and Data Science Education (Zieffler et al. 2021). We found that while teachers were able to read and build classification tree models, they had more difficulty evaluating models, indicating the need for further research on how teachers understand model evaluation and how to support the development of their understanding.

Graduate Program in Statistics Education

AR: You have worked with Ph.D. students who have specialized in statistics education. Please tell us how that program came to be.

RD: The specialization in statistics education is an emphasis within the Quantitative Methods in Education (QME) program within the Department of Educational Psychology at the University of Minnesota. The specialization was the brainchild of Joan Garfield, which she founded in 2002. Starting with coursework in advanced applied methods in measurement, statistics and evaluation required by the QME program, the program added additional courses such as: (1) Becoming a Teacher of Statistics, which introduces students to the statistics education research literature and teaching methods, and (2) Statistics Education Research Seminar, where students read and discuss current research literature within a focused area in statistics education. I became the second tenured faculty member in the program when I transferred to the Department of Educational Psychology in 2006, although I had served as co-advisor for a student in the statistics emphasis prior to my transfer. Students who have completed the Ph.D. program developed instruments to assess students’ understanding of areas such as statistical literacy, informal versus formal inferential reasoning, and concepts related to simulation-based inference methods, or developed curriculums for specific topics in statistics, such as understanding sampling variability and sampling distributions and understanding the purpose of and difference between random sampling and random assignment in research designs.

AR: Tell us more about the “Becoming a Teacher of Statistics” course. For example, what are some readings that you assign, and what kinds of assignments or projects do students complete?

RD: The Becoming a Teacher of Statistics course has about 30 assigned readings that are selected to introduce students to the field of statistics education research and to support their completion of assignments in the course. Here is a list of some of the assigned readings to provide a sense of the covered topics:

Chance, B., Ben-Zvi, D., Garfield, J., and Medina, E. (2007). “The Role of Technology in Improving Student Learning of Statistics.” Technology Innovations in Statistics Education, 1.

Cobb, G. W. (2007) “The Introductory Statistics Course: A Ptolemaic Curriculum?” Technology Innovations in Statistics Education, 1.

Garfield, J. (2013). “Cooperative Learning Revisited: From an Instructional Method to a Way of Life.” Journal of Statistics Education, 21.

Garfield, J., delMas, R., and Zieffler, A. (2012), “Developing Statistical Modelers and Thinkers in an Introductory, Tertiary-Level Statistics Course.” ZDM—The International Journal on Mathematics Education, 44(4), 883–898.

Garfield, J., Zieffler, A., and Fry, E. (2017). “What is Statistics Education?.” In International Handbook of Research in Statistics Education, eds. D. Ben-Zvi, J. Garfield, and K. Makar, Springer.

Garfield, J., Zieffler, A., Kaplan, D., Cobb, G., Chance, B., and Holcomb, J. (2011). “Rethinking Assessment of Student Learning in Statistics Courses.” The American Statistician, 65, 1–10.

Gould, R. et al. (2016). “Teaching Data Science to Secondary Students: The Mobilize Introduction to Data Science Curriculum.” In Proceedings of the Roundtable Conference of the International Association of Statistics Education (IASE), eds. J. Engel.

Horton, N. J. (2015). “Challenges and Opportunities for Statistics and Statistical Education: Looking Back, Looking Forward.” The American Statistician, 69(2), 138–145.

Mills, J. D., and Raju, D. (2011). “Teaching Statistics Online: A Decade’s Review of the Literature About What Works.” Journal of Statistics Education, 19.

Tintle, N., Topliff, K., VanderStoep, J., Holmes, V., and Swanson, T. (2012). “Retention of Statistical Concepts in a Preliminary Randomization-Based Introductory Statistics Curriculum.” Statistics Education Research Journal, 11, 21–40.

Prior to the weekly class meeting, each student is expected to complete the assigned readings and post a reflection. In addition to reading assignment reflections, there are five reflection assignments that ask the students to reflect on their statistics education experiences, watch and report on a webinar posted at the Consortium for the Advancement of Undergraduate Statistics Education (CAUSE) website, find a YouTube video that could be used to teach a statistical topic, observe someone teach a class for an introductory statistics course, and reflect on their philosophy of teaching statistics.

In addition to the readings, the course invites seven to eight guest experts to a virtual visit with the class during a class session. Past guests have included Michael Bulmer, Mine Çetinkaya-Rundel, Chris Franklin, Joan Garfield, Rob Gould, Ellen Gundlach, Nick Horton, Danny Kaplan, Robin Lock and Patti Frazer Lock, Nathan Tintle, and myself (when I was not the instructor). For sessions with guest experts, students are assigned readings and other resources related to the guest topic. For example, Chris Franklin was a guest expert on the topic Connections to K-12 Statistics Education. Students were assigned the GAISE K-12 report, an article by Franklin et al. (2011) in The American Statistician titled AP Statistics: Building Bridges Between High School and College Statistics Education, and links to information on the Common Core standards related to statistical concepts. Based on the assigned materials, students posted questions for Chris to consider and respond to during the class visit. Another assignment requires each student to explore a statistical software program or package and make a class presentation. Each student also participates in a small group assignment throughout the semester where the group identifies a dataset to use in teaching a statistical concept, creates activities to teach the topic using the dataset, and makes a class presentation at the end of the course.

AR: Now I’ll ask about preparing students in this program to become education researchers. What are some of the fundamental ideas and skills that you aim to help students develop toward that goal? What are some of the biggest challenges?

RD: Students who have entered the graduate program in statistics education research are interested in becoming better, more effective teachers of statistics. That motivation is necessary and welcome, but students need to also come with or develop questions that can be turned into researchable studies. While I do not want to say it is easy to become a better teacher of statistics, we have good evidence that all of the students who have gone through the program have done so. I believe this is primarily a result of the coursework and the mentoring they receive while serving as teaching assistants and instructors in our statistics courses. The bigger challenge is coming up with a researchable question related to how students learn statistics, why students have difficulty learning certain concepts and topics in statistics, or identifying effective ways to teach statistics. If a student does not come to the program with curiosity about these types of questions, it can be a challenge to help the student develop that interest and find a researchable question. However, students’ dissertation topics usually emerge as students read and discuss the statistics education research literature, especially by engaging in the statistics education research seminars. Students’ doctoral dissertations have developed high quality assessment instruments, effective curricular innovations, and important frameworks for researching students’ understanding of statistics. The dissertations can be found by going to the IASE website (iase-web.org), clicking on the Publications link, then clicking on the Dissertations link (or going directly to http://iase-web.org/Publications.php?p=Dissertations) and searching for “Minnesota.”

AR: What are some academic backgrounds of students who have come into this program? What are some typical careers that graduates of the program pursue?

RD: Students admitted into the Master’s degree program typically have a bachelor’s degree in mathematics or statistics. Students who enter directly into the Ph.D. program typically come with a master’s degree in statistics, but a few have had master’s degrees in mathematics education or a field where statistics coursework is prevalent. The majority of graduates from the Ph.D. program have faculty positions in statistics departments or biostatistics departments in 4-year colleges or universities. Some, however, have worked in other university settings such as a research and consulting center or a center for teaching and learning, in the assessment development industry, or for the U.S. government as a data analyst.

Pop Quiz

AR: Now let’s move to the “pop quiz” portion of this interview. I will ask several questions that move beyond statistics education, and I’ll ask that you keep your responses brief. First, please tell us about your family.

RD: My wife, Rose Gregoire, passed away in January 2021 after battling ovarian cancer for two years. We were married for 35 years and knew each other for 40 years. Rose had a wonderful, creative imagination, was very involved in community activities, was an avid gardener, and a great problem solver. I miss her dearly. We have two adult children. Nicholas, the oldest, has a degree in computer science from California Polytechnic University, works in the Bay area, and is an avid video game player and collector. Our daughter, Elise, has a master’s degree in mathematics from the University of Minnesota, is a musician and singer, and shares many things in common with Rose. And the final member of my family is Nilla Wafer, a very cute, black and white, high energy rat terrier who is a constant source of fun and comfort.

AR: My condolences on your wife’s passing. Next I’ll ask for some of your hobbies.

RD: I practice a Japanese style of karate (for almost 50 years), play guitar and harmonica, sing, love hiking/canoeing/camping, and enjoy doing home improvement projects.

AR: Please recommend a book for pleasure reading, a movie, a musical selection, and a place to visit.

RD: A book I read this summer and enjoyed was “Leonardo da Vinci” by Walter Isaacson. However, I also just reread “Dune” by Frank Herbert, and I highly recommend seeing the current movie directed by Denis Villeneuve. I often listen to and play and sing songs by James Taylor (I used to sing “Sweet Baby James” as a lullaby to my children). I would love to go back to Australia and highly recommend a visit to any part of the eastern coast or the Outback.

AR: Here is a fanciful question: You can travel in time to observe what’s going on in the world for one day. What time would you travel to – in the past or the future – and why?

RD: I have always been intrigued by Albert Einstein, both the scientist and the person, and have read several biographies on Einstein (the earliest being a book I checked out of the school library when I was in fifth grade) as well as accounts of his special and general theories of relativity. Einstein died the year I was born. So, if I can choose my fancy, it would be to travel back in time to a day in 1954 or 1955 to spend the day with Albert Einstein talking about science and humanity, and to perhaps experience the spontaneity, creativity, and childlike nature of his mind and personality. I would like to see the world at that time, just before I was born, through the eyes of a very brilliant but genuine human being, as I think his perspective would have much bearing on our present world.

AR: Here’s another silly one: You can have dinner anywhere in the world (all expenses paid) with three companions, but the only topic of discussion is to be statistics education. Who would you invite, and where would you eat?

RD: There really are so many people to choose from, but I can only choose three. Joan Garfield would have to be there because she is a good friend, research colleague, and well-known foodie. Another choice would be Rob Gould as he has a well-defined palate for both food and all things statistical. The third would be Dani Ben-Zvi, as I know he enjoys a fine meal, but would also help to stir up the pot around a discussion of the future of statistics education. As for a place to eat, the location I would pick is Paris, France, and I would call on Joan to recommend the restaurant.

AR: Now let’s collect some data: Do you consider yourself an early bird or night owl? On what day of the week were you born? How many of the 50 states have you set foot in? How many miles do you live from your birthplace?

RD: I am definitely an early bird; no matter what time I go to bed at night, I am up by 6am and often earlier, both on weekdays and the weekend. I was born on a Tuesday. I have set foot in 45 of the 50 United States. According to the most direct route by car, I live 1,871 miles from my birthplace (Long Beach, CA).

Favorite Course and Readings

AR: Back to statistics education: What has been your favorite course to teach?

RD: I truly do not have a favorite course, because I enjoy any opportunity I have to work with students, answer questions, and engineer and facilitate ways for them to explore concepts, especially statistical concepts. However, one of the most fun courses I ever taught was called “Psycho-Economics.” This was a course I co-developed at Northeast Missouri State University (now Truman University) with a colleague in the Economics department, David Gillette. The course introduced students to synergies between psychological studies and economic theories by having the students participate in classical research activities that demonstrate human behavior under conditions of uncertainty. Many of the activities were based on studies described in “Judgment under uncertainty: Heuristics and biases” by Kahneman et al. (1982), as well as related studies in the literature. I enjoyed figuring out how to carry out the research activities with the students, gather data, and display results all in a single class session, while still allowing time for discussion. It was fun to see the students’ expressions when they replicated results from the original studies, often demonstrating biases in their reasoning, and to engage in discussions with them about the implications for human behavior. The students were very engaged, and we as the instructors were equally engaged.

AR: Please recommend some readings (books or articles) that you recommend, first for teachers of statistics and then for statistics education researchers.

RD: For both teachers of statistics and statistics education researchers, I recommend:

Ben-Zvi, D. and Garfield, J., eds. (2004). The Challenge of Developing Statistical Literacy, Reasoning and Thinking. Kluwer Academic Publishers.

Gal, I. and Garfield, J. B. (1997). The Assessment Challenge in Statistics Education. IOS Press.

Garfield, J. and Ben-Zvi, D. (2008). Developing Students’ Statistical Reasoning. Springer.

Lock, R. H., Lock, P. F., Morgan, K. L., Lock, E. F., and Lock, D. F. (2020). Statistics: Unlocking the Power of Data (3rd ed.). John Wiley & Sons.

Lovett, M. C. and Shah, P., eds. (2007). Thinking with Data. Lawrence Erlbaum Associates.

National Academies of Sciences, Engineering, and Medicine. (2018). How People Learn II: Learners, Contexts, and Cultures. National Academies Press.

National Research Council. (2004). How Students Learn: History, Mathematics, and Science in the Classroom. The National Academies Press.

Rossman, A. J., and Chance, B. L. (2011). Workshop Statistics: Discovery with Data (4th ed.). John Wiley & Sons.

Scheaffer, R. L., Watkins, A., Witmer, J., and Gnanadesikan, M., Erickson, T. (2006). Activity-Based Statistics: Student Guide (2nd ed.). John Wiley & Sons.

Tintle, N., Chance, B. L., Cobb, G. W., Rossman, A. J., Roy, S., Swanson, T., and VanderStoep, J. (2020). Introduction to Statistical Investigations (2nd ed.). John Wiley & Sons.

For statistics education researchers in particular, I recommend:

Bakker, A. (2004). “Design Research in Statistics Education: On Symbolizing and Computer Tools,” Doctoral dissertation. https://dspace.library.uu.nl/bitstream/handle/1874/893/full.pdf?sequence=2.

Ben-Zvi, D. and Garfield, J. (Eds.) (2018). International Handbook of Research in Statistics Education. Springer International Publishing.

Conclusions

AR: What do you see as the biggest challenges and opportunities for the field of statistics education research? What are your hopes and expectations for the field’s development in the next decade or two?

RD: I hope that researchers continue to look at the challenges students face in developing a conceptual understanding of statistics and statistical practice and research educational methods to help students meet those challenges. We have made progress in helping students develop important concepts after several decades of research on how to help students understand measures of center and variability, sampling variability, correctly interpret confidence intervals, understand statistical modeling and randomization methods, and other important statistical concepts. Even so, some students in my courses still demonstrate incomplete understanding even when I use activities and teaching methods shown to be effective by research. Perhaps there are only so many students that can be brought along the path to understanding in a single course, but I hope that is not the case. The 2016 college-level Guidelines for the Assessment and Instruction of Statistics Education (GAISE) report points to promoting multivariate thinking as an important learning goal for a first course in statistics, as well as more advanced statistical learning. This is an area that could benefit from much exploration of what it means to think in a multivariate way, how to reliably measure multivariate thinking, and what types of learning experience promotes multivariate thinking. We have also witnessed the movement of data science into the curriculum, a new disciplinary area that presents the challenge of defining the disciplinary concepts and methods at the same time it is evolving. One challenge may be to develop a framework that identifies the stable concepts within data science that can currently be targeted for teaching and determining which of these concepts are suitable at an introductory level in order to carry out educational research that can inform instructional practice. I believe there is still much to do and learn and I look forward to watching how the field of statistics and data science education grows over the next several decades.

AR: I was planning to ask about data science in my next question, and you raised that topic before I did. Do you have thoughts or opinions about what aspects of data science, if any, should be taught to introductory students? Wait, that question is too broad, so let me ask more specifically about students who currently study introductory statistics to fulfill a general education requirement.

RD: I certainly do not have expertise in the area of data science, but it strikes me that there is a large overlap between data science and statistics, so, arguably, much of what is covered in an introductory statistics course is data science. Having said that, I think that one promising addition would be the inclusion of more experience in data wrangling. Based on my experience, introductory statistics textbooks and courses often include examples where the data is already clean and tidy. Students can just read in the data and run analyses. But real-world applications of statistics often involve data that need to be preprocessed before they are ready to be subjected to a particular data analysis method. Including more datasets that need to be cleaned or preprocessed before data analysis would better prepare students for applying statistical methods. I also expect that adequate instruction on machine learning methods requires separate courses that are dedicated to building up the knowledge needed to implement methods used in industry, so I would not recommend trying to fit them into an already crowded introductory statistics curriculum. The statistics reform movement dedicated time and effort to paring down topics in the introductory statistics course in order to provide more in-depth coverage of important statistical concepts. I would not want to see the introductory statistics course watered down simply to include exposure to additional content.

AR: Among all of your accomplishments in statistics education, can you identify one of which you are most proud, and explain your choice?

RD: I would have to choose the development of the Sampling SIM program. I named the original version of the program Sampling Distributions (see delMas, Garfield, and Chance 1999) and renamed later versions Sampling SIM (see Chance, delMas, and Garfield 2004). The design of Sampling SIM was based on research findings from cognitive science and recommendations for designing interactive software in order to support the development of students’ understanding of sampling, sampling variability, sampling distributions, hypothesis testing and confidence intervals. Sampling SIM was used in several research studies that I conducted with Joan Garfield and Beth Chance where we developed activity-based lessons to promote statistical understanding and novel assessment items to measure changes in students’ understanding. I believe these research studies have informed the research of others in the field of statistics education research, including some influence on current approaches that teach statistics through randomization-based methods.

AR: Before we wrap this up, would you like to give any answers to questions that I have not thought to ask? Would you like to take this opportunity to say anything to readers of JSDSE?

RD: Your questions have really allowed me to comment on so many aspects of my career and statistics education. Honestly, there is nothing coming to mind that I feel I would want or need to comment on. I hope that people who are passionate about teaching statistics will continue to document difficulties students have in learning statistical concepts, be creative and innovative in designing ways to help students surmount those difficulties, and conduct innovative research to provide evidence of whether or not the innovations are effective. And I hope to read about those innovative instructional approaches and research studies in future publications of JSDSE.

Additional information

Notes on contributors

Bob delMas

Bob delMas is an Associate Professor in the Quantitative Methods in Education program in the Department of Educational Psychology at the University of Minnesota.

He is the 2022 recipient of the George Cobb Award for Lifetime Achievement in Statistics Education. This interview took place via email on August 10, 2021 – February 20, 2022.