Setting the Stage: Statistical Collaboration Videos for Training the Next Generation of Applied Statisticians

Abstract

Collaborative work is inherent to being a statistician or data scientist, yet opportunities for training and exposure to real-world scenarios are often only a small part of a student’s academic program. Resources to facilitate effective and meaningful instruction in communication and collaboration are limited, particularly when compared to the abundant resources available to support traditional statistical training in theory and methods. Our work helps fill the need for resources by providing ten modern, freely-available videos of mock collaborative interactions, with supporting discussion questions, scripts, and other resources. Videos are particularly helpful for teaching communication dynamics. These videos are set in the context of academic research discussions, though the scenarios are broad enough to facilitate discussions for other collaborative contexts as well. The videos and associated resources are designed to be incorporated into existing curricula related to collaboration. The materials have been piloted with positive feedback from students and instructors. Supplemental files for this article are available online.

KEYWORDS:

• Collaboration
• Communication
• Education
• Resources
• Video

1 Introduction

Statistics students, both undergraduate and graduate, typically learn foundations of statistical inference, methods, and theory through a set of core courses. Beyond these foundations, statistics and data science education should include formal training in important aspects of effective collaboration and consulting, including communication skills (Parke 2008; Greenhouse and Seltman 2018). Exposure to common challenges and work on skills needed for collaborative work is part of a well-rounded statistics education. Despite this, collaboration-related educational opportunities get far less emphasis than theory, methods, and computing material covered in textbooks. Many students could benefit from additional time, effort, and resources directed at this crucial aspect of education in statistics or data science.

As data science-related professions continue to increase in visibility and demand, training of applied statisticians in collaborative skills, along with exposure to common challenges encountered in practice, is increasingly important (Hahn and Doganaksoyis 2012; Vance 2015; Greenhouse and Seltman 2018; Davidson, Dewey, and Fleming 2019; Gibson 2019). Resources to support such training are in demand, especially those allowing flexibility in format to accommodate a range of instructional types and student audiences.

One particular focus in statistics courses covering collaboration or consulting is communication in challenging situations frequently encountered by applied statisticians or data scientists. Available resources include publications describing a curriculum across an entire program (Parke 2008; Greenhouse and Seltman 2018), those describing best practices for teaching collaborative skills (Smith and Griffin 1990; Davidson, Dewey, and Fleming 2019; Vance and Smith 2019), as well as a few statistical consulting textbooks that can be used to help lead discussions on consulting skills. The books include James Boen and Doug Zhan’s 1982 book The Human Side of Statistical Consulting, Janice Derr’s 2000 book Statistical Consulting: A Guide to Effective Communication, Javier Cabrera and Andrew McDougall’s 2002 book Statistical Consulting, and, more recently, Doug Zahn’s 2019 book Stumbling Blocks to Stepping Stones: A Guide to Successful Meetings and Working Relationships.

Resources about communication skills also include helpful frameworks and collaboration practices, such as the ASCCR (Attitude, Structure, Content, Communication, Relationship) framework, POWER (Prepare, Open, Work, End, Reflect) process, and LEARN (Listen, Evaluate, Act, Review, Next) frameworks (e.g., Vance and Smith 2019; Zahn 2019). However, many students do not have the necessary background to productively engage in realistic mock meetings, making supplementary resources an important tool, particularly for students who will not have the opportunity to engage in a consulting or collaboration practicum. A valuable part of Derr’s (2000) book is the accompanying series of 13 videos illustrating communication between a collaborating researcher and statistician, using a storyline. In some of the videos, scenes with both positive and negative examples of communication are presented for a particular portion of a meeting.

Pedagogical research supports the value of videos in learning when the videos are integrated into the curriculum and accompanied by interactive collaboration and conversation (Karppinen 2005). Videos have been shown to be particularly helpful for showing the dynamics of face-to-face interactions using both verbal and nonverbal communication (Karppinen 2005). Providing discussion questions along with the videos has been shown to increase student learning (Lawson et al. 2006). In our own experiences as students and teachers, we found that Derr’s videos elicited student engagement, discussions, and reflection in a way that textbooks and mock meetings did not; the videos provided motivation and structure to facilitate active learning. These experiences are consistent with those attributed to the use of such videos in education research (Karppinen 2005). Unfortunately, Derr’s videos and text are now out of print and can be challenging to obtain.

With the goal of filling the need for additional, and easily accessible, collaboration-focused training resources, we developed and produced a collection of ten modern, open-access videos and accompanying materials to serve as foundations for rich discussion in statistical and data science coursework. Our videos are not meant as a remake of the Derr (2000) videos; we expect the Derr videos to have continued value for those with access to them.

With the exception of one video, each is under 5 minutes in length, shorter than the suggested time for best practices for student learning by Brame (2016). The video lengths leave ample time for viewing and discussion within a virtual or in-person class. The videos are closed-captioned for accessibility, as well as freely and publicly available on the American Statistical Association (ASA) website (https://community.amstat.org/statisticalcollaborationtrainingforappliedstatisticians/home).

In this article we briefly describe goals and potential uses for the videos; present the 10 scenarios and our motivation for choosing them; discuss the accompanying materials and provide recommendations for their use; and share initial feedback on the videos.

2 Goals and Flexibility for Use

The videos can support learning within different delivery formats, as well as across a variety of courses and student backgrounds. Each video is accompanied by a set of questions meant to motivate discussion and/or reflection, as well as a script of the dialogue allowing students to act out the scene themselves and experiment with making positive or negative changes. The videos are designed to facilitate active learning as a supplement within a course or seminar, where each video is stand-alone so an instructor can choose content, time commitment, discussion questions, and how to incorporate the materials into a larger course structure.

While we developed the video collection with mainly consulting and collaboration courses in mind, the materials can supplement traditional statistics courses (even those with non-statistics students) due to their focus on communicating statistical ideas and working with a statistician. The practice of combining the learning of core material with real-world application and collaboration skills continues to gain traction in statistics education, and these videos provide one way to support movement in that direction. Instructor feedback also notes that focusing on statistical concepts included in the video content can be used to motivate and direct learning, as opposed to focusing only on communication skills. The extent to which the videos are used for the conceptual content versus the lessons in communication will depend on the course; we find these to be integrally linked in practice and developed the videos with this mind.

The videos are not designed to model a perfect interaction or to blatantly point out ineffective communication; instead, the goal is to create rich scenarios with enough depth in statistical content and communication that students are afforded a realistic glimpse into challenges encountered in practice, as well as differences in communication styles. Ideally, the videos and elicited discussions allow students to think through difficult situations in a safe and supportive environment before confronting them in practice. Common themes across all videos include establishing expectations, clear and professional communication, and helping statisticians to convey their role as collaborators rather than service providers or technicians. The videos are helpful for demonstrating the power of asking good questions, listening actively, and negotiating carefully to influence the direction of a collaboration.

As previously mentioned, the interactions presented in the videos do not represent the best, or only, way to manage the scenarios presented. The setting for all videos is academic statistical collaboration, involving short-term projects. However, aspects of the conversations presented occur in other settings (e.g., consulting in private industry). Rather than providing an illustration of how to manage a situation, the materials motivate reflection about positive and negative aspects of communication shown in the video, as well as potential differences in opinion related to the statistical practice, concepts, or philosophy incorporated into the topic of a particular conversation. To motivate engaging discussion, some of the videos do not end with a clear solution and the supporting materials encourage consideration of other strategies.

The scenarios are meant to recognize and respect that many details of an effective response will depend on the specific context, such as the individuals involved, roles and expectations with the collaboration, or the individuals’ jobs. Overall, the videos provide the opportunity for students to interact with statistical concepts that tend to be difficult to discuss, while working on key collaborative skills including clear communication, negotiation, and organization.

3 Scenarios

The scenarios depicted in the videos were identified from our own extensive experiences as collaborating statisticians and common issues of negotiation described by others (e.g., Derr 2000). We also included input from statistics graduate students who worked as statistical collaborators. The scenarios and the accompanying questions can be loosely organized into several themes: general communication, planning, and meeting-related issues; challenges with statistical inference; and professional ethics. Most have more than one theme and there is overlap among videos. For example, multiple scenarios portray the challenges faced by a statistician when researchers approach the interaction as a straightforward transaction for a well-defined technological service or delivery of advice. A full list of video scenarios, lengths, and descriptions is provided (Table 1).

Table 1 List of video scenarios, lengths, and descriptions.

Video	Title (Length)	Keywords	Description
1	Countering stereotypical views of statisticians (3.50 min)	Statistician stereotype; project planning; study design; ANCOVA; active listening	Scientific collaborators may initially communicate as if a statistician is a technician, using phrases such as “I just need you to run the statistics” or “I just need you to calculate a sample size.” In this video, a researcher makes such a statement and the statistician responds. The scenario encourages discussion about next steps and how the conversation could have been handled differently, while also providing an opportunity to discuss the importance of asking specific questions about study design.
2	Defining roles and scope of the work (4.08 min)	Roles; expectations; repeated measurements; funding expectations; thesis or dissertation; graduate student research; active listening; teaching	For a statistician, negotiating one’s scope of work and role in a collaboration can lead to difficult conversations. This video presents a scenario portraying specific challenges that may arise when the project is the basis of a graduate student’s thesis or dissertation.
3	Turning down requests for new work from a current collaborator (2.34 min)	Understanding statistician workload capacity; funding; statistical support; declining requests; establishing boundaries; workload; negotiation	A statistician may need or want to decline requests for new or additional work from a current collaborator for many reasons, including a lack of time or even negative experiences with the collaborator. This video portrays a discussion between a collaborator who requests assistance with new projects and a statistician who turns down the work. The scenario encourages discussion about prioritizing projects and establishing boundaries.
4	Discussion about a realistic timeline for work (4.03 min)	Understanding statistician workload capacity; timeline for analysis; data management; exploratory data analysis; descriptive analysis; co-authorship; workload; negotiation	Collaborators may believe a statistician should be able to accomplish proposed work in an unrealistically short time frame. This video depicts a scenario where a researcher expresses an opinion that the statistician’s part of the work should not take much time, and the statistician responds. The video promotes discussion about negotiations, timelines, and expectations.
5	Beginning and ending a meeting effectively (3.48 min)	Meeting structure; initial meeting; ethics; enforcing professional boundaries; active listening	An effective beginning and ending are important to leading a productive meeting. This video depicts a scenario where the collaborator’s expectations for an initial meeting do not align with those of the statistician, and where the statistician finds it difficult to end the meeting on time. This scenario can be used to start a discussion about running an effective meeting and establishing expectations, as well as the ASA’s Ethical Guidelines for Statistical Practice.
6	Using statistical results to identify questions of interest (4.27 min)	Secondary data analysis; funding; identifying objectives; ethics; teaching	Statistical analysis may be used in an exploratory way to identify specific comparisons or questions for which to focus a report or future research. In this video, a collaborator requests that the statistician assist with “finding what’s significant and important” to drive the current research findings. This video encourages discussion about assuming good intentions, the importance of distinguishing between a priori and post hoc comparisons (or exploratory and confirmatory analysis), and the ASA’s Ethical Guidelines for Statistical Practice.
7	Statistically significant versus practically meaningful (4.18 min)	Sample size; statistical significance; practically meaningful effect; clinically significant; effect size	A researcher’s need to justify a sample size for a study often provides motivation for contacting a statistician. The process of carrying out a sample size investigation is usually more involved than expected by the researcher (who might think of it as a simple mathematical calculation). This video depicts one of the most challenging parts of the study planning process – discussing the difference between statistical significance and practical importance.
8	Advocating for data visualization (3.13 min)	Data visualization; data anomalies; exploratory data analysis; study design	Introductory statistical training typically does not emphasize the importance of creatively plotting data before summarizing and aggregating. Statisticians may be in a position to advocate for and assist with effective visualizations for a given project, particularly when the researcher may not have the necessary computing expertise. In this video, a researcher and statistician discuss data visualization as a first step in the analysis, and identify important design information that would have otherwise been overlooked. The scenario encourages discussion about the importance of thorough exploratory data analysis and considering data visualization as an integral part of the analysis process.
9	Navigating requests related to p-hacking and survey validation (4.07 min)	p-hacking; survey validation; clarifying research objectives; statistical significance; professional ethics	Understanding and reflecting on professional ethics related to being a statistician is an important aspect of statistical collaboration. This video presents a meeting where a scientist describes their plan to use statistical significance to decide what to report in a publication, putting the statistician in an ethical dilemma. The video also motivates discussion about issues related to pressure put on researchers to use “validated” instruments and obtain “significant” results.
10	Pseudoreplication and refusal to use a statistician’s advice (5.44 min)	Pseudoreplication; publication with atypical analyses; discipline norms; co-authorship; ethics; acknowledgement for contributions; establishing professional boundaries; professional ethics	Statisticians may contribute work that is reasonable and sound, only to ultimately have a collaborator ignore the contributions. This may be because the methods used do not align with researcher perceptions of norms in their discipline, or due to outside pressure from advisors, reviewers, funding agencies, or others. This video portrays a statistician in such a situation and elicits discussion of ethical questions, such as how a statistician should be acknowledged for work, particularly if they disagree with the approach ultimately used. The scenario can also lead to more technical discussions about pseudoreplication.

NOTE: The videos may be used as a set but may stand alone; the videos were designed to be used to augment an existing curriculum. The videos are not listed in any particular order and are numbered for ease of identification.

3.1 General Communication, Planning, and Meeting-Related Issues

Students are frequently nervous about structuring and running meetings with existing or potential collaborators. To help address these concerns, five videos directly address general communication, planning, and other meeting-related issues. The videos and discussion questions focus on ways to:

• work with a domain expert to move away from a technician role toward a more collaborative role relationship (Video Scenario 1),

• negotiate the scope and timeline of work in a mutually beneficial way (Video Scenarios 2 and 4),

• discontinue a collaborative relationship and decline future work (Video Scenario 3), and

• end a meeting politely, yet firmly (Video Scenario 5).

3.2 Challenges with Statistical Concepts

Discussing statistical concepts with a nonstatistical audience is an essential skill that requires practice (Derr 2000; Love 2016; Nussbaum 2017; Vance and Smith 2019; Zahn 2019). To quote Nussbaum (2017), “It’s not what we said, it’s not what they heard, it’s what they say they heard.” In the videos focusing on challenges with statistical concepts, students hear and critique conversations around technical statistical topics led by the statistician, and are encouraged to reflect on ways to discuss statistical concepts with non-statisticians. Common skills portrayed across these videos include listening and teaching in a respectful and professional manner. The videos show collaborative statisticians:

• responding to broad and ill-defined questions and requests (Video Scenario 6),

• discussing and explaining the difference between statistical significance and practical relevance (Video Scenario 7), and

• encouraging scientific collaborators to expand their understanding of statistical thinking to include data visualization and exploratory descriptive summaries (Video Scenario 8).

3.3 Professional Ethics

Ethics-related topics are important for future statisticians to consider and may not be taught as part of a regular statistics curriculum. A couple of scenarios lead naturally into discussions on ethics related to statistical analysis and reporting of results. Discussions can combine principles from the ASA’s Ethical Guidelines for Statistical Practice (2018) with awareness of underlying tone or demeanor to demonstrate that it can be important for a statistician to understand that requests for unprincipled analyses often come from a lack of understanding and not from ill-intent. The videos with topics related to professional ethics can also supplement the module on applied ethics described in Elliott, Stokes, and Cao (2018). In the ethics-related videos, scenarios include:

• navigating requests for p-hacking (Video Scenario 9),

• explaining pseudo-replication and its potential ramifications (Video Scenario 10), and

• addressing inappropriate and potentially unethical analysis choices (Video Scenario 10).

4 Supporting Materials

Two discussion questions are provided at the end of each video to prompt and facilitate immediate discussion. In addition, a larger set of discussion questions, a non-exhaustive list of resources for further reading, and a script of the interaction accompany each video.

4.1 Written Scripts

Written scripts are provided for each scenario (Appendix A, supplementary material). The scripts can be used for reference during a discussion or to have students carry out the roles while working in pairs during class (in addition to or instead of the videos). This role-playing is yet another way that these videos can be adapted to fit the context of an already-defined course or curriculum. Reading the scripts with a peer in class reinforces the ideas of reflective statistical practice that are built into models for collaborative best practices like the ASCCR framework and the POWER process for structuring a meeting (Vance and Smith 2019; Zahn 2019). Role-playing exercises are also part of the LEARN process (Listen, Evaluate, Act, Review, Next) for systematically improving collaborative communication (Zahn 2019). Compared to critiquing recorded interactions between students and collaborators in class, using scripted exercises may reduce self-consciousness and potential biases (even implicit) that may occur in training (e.g., Smith and Griffin 1990). One scenario (Video Scenario 1) has a script for additional interactions included in the discussion questions that were not included in the video due to length considerations.

4.2 Additional Discussion Questions

In addition to the two discussion questions at the end of each video, accompanying materials include at least four additional discussion questions (Appendix B, supplementary material). The questions, while not exhaustive, often add additional context to the material presented in the scripts and allow for discussion of topics not included in the actual videos. Instructors are encouraged to read through all questions and choose those of most relevance for their specific learning objectives, as well as reframe and add to the questions as appropriate for their students. The questions can be used as reflective assignments following viewing of the videos, or after students meet with clients/collaborators if that is a part of the course requirement. For online courses, discussion questions can be posed to a course discussion board for student input and reflection. Students could also be asked to pose their own discussion questions related to the scenarios.

4.3 Resources for Further Reading

Beyond providing additional discussion questions, many of the videos are also accompanied by suggested readings (Appendix B, supplementary material). These articles, guidelines, and books assist both students and instructors to think critically about the communication issues presented in the videos. The collection provided is meant to be a starting point and is restricted to resources relevant to the discussion questions included, not an exhaustive list of resources. These resources can be assigned prior to and/or after reviewing the relevant video.

5 Discussion

Drafts of the videos were piloted in two statistical collaboration courses in the fall of 2019 and the spring of 2020. The final videos were used in graduate-level statistical collaboration courses in a large statistics department in the summer and fall of 2020. The 2019 and early spring 2020 sections were taught in person, while the remainder were taught asynchronously online. Students were a mix of masters- and doctoral-level students in a statistics program. All students were required to work with at least one scientific collaborator as part of the course. Students were required to write a statistical report, give at least two oral presentations; lectures and class activities focused on collaboration and communication. Feedback about the videos was solicited and collected via course evaluations, in-class discussions, personal e-mails, and written reflection assignments.

In the in-person sections, the students were paired to read the scripts for several of the videos. In one activity, students were asked to read the script a second time with a different tone. The discussion centered on the importance of tone and nonverbal communication in a meeting. Students were very eager to discuss the issues of assuming ill-intent, as well as noticing increasingly negative reactions as both statisticians and clients grew more defensive.

In another activity, students identified parts of the interaction that led to a breakdown in communication and identified at least two alternative lines for the statistician’s script and where they would have reacted differently. This activity was successful both in-person and online. The identification of alternative statistician communication led to a rich discussion on differences in communication styles and the importance of paying attention to the other person’s reaction in a conversation. We have found it important for students to realize that while best practices and guidelines exist, there is no magic formula for communication or other aspects of statistical practice that applies to all people and every situation.

Preliminary student feedback to the videos, scripts, and discussion questions has been overwhelmingly positive. One student referred to the scripts in a written reflection about their first meeting: “I was a bit surprised by some of the client’s reactions to us. I felt that there was a little bit of defensiveness when we saw some of their previous work (though I think they had warmed up to us somewhat by the end of the meeting). I am starting to see how some of the scripts actually do come up in real life.” This review shows the effectiveness of the scripts in helping to prepare students for meetings – a challenging concept to teach without a resource such as these videos. Another student group had a client interaction that they described in an e-mail as “a video moment” when their collaborator objected to their presentation of results. The students credited the videos for improving their confidence within the discussion.

As previously mentioned, we have found in our work that discussing possible ways to respond to challenges before they arise can improve communication and lead to more successful and productive professional conversations and relationships. Likewise, instructor feedback from a graduate-level statistical consulting seminar course for master’s-level students included appreciation for the support around framing expectations for client meetings. In particular, the instructor was able to use the videos to discuss expected challenges in interacting with clients (M. Greenwood, Personal Communication, March 31, 2021).

A key responsibility of statistics educators is to encourage formal training of students in communication and effective collaboration, especially with the increased demand for these skills. These videos augment the training of the next generation of applied statisticians to be effective collaborators and to work in interdisciplinary teams. The topics covered are useful for statistics students, but also for professional training for working collaborative statisticians. Domain-science practitioners might also use the videos for training on how to interact with statisticians. This video collection serves as a valuable resource for training the next generation of quantitative researchers and practitioners.

Acknowledgments

The authors thank graduate students who assisted with idea development, Dr. Ann Hess for reviewing the scripts, questions, and videos, Ms. Mona Maser for videography, and the actors and production crew. The authors also thank Dr. Andrew Hoegh and Dr. Mark Greenwood for providing instructor feedback on the videos.

Supplementary Materials

Appendix A. Includes scripts for the 10 videos. Scripts are organized by scenario and labeled A.X Scenario Title where X represents the number of the video (1–10) corresponding to the listing in Table 1 of the manuscript. Appendix B. Includes discussion questions for the 10 videos. Discussion questions are organized by scenario and labeled B.X Scenario Title where X represents the number of the video (1–10) corresponding to the listing in Table 1 of the manuscript.

Additional information

Funding

The videos were created through funding provided by the American Statistical Association (ASA) Strategic Initiatives.