Iterative user and expert feedback in the design of an educational virtual reality biology game

ABSTRACT

This study focuses on an educational game titled Cellverse, a two-player cross-platform VR project intended to teach high school biology students about cell structure and function. In Cellverse, players work in pairs to explore a human lung cell and diagnose and treat a dangerous genetic disorder. Cellverse is being designed by the Collaborative Learning Environments in Virtual Reality (CLEVR) team, an interdisciplinary team consisting of game designers, educational researchers, and graduate and undergraduate students. Using a design-based research approach, we have enlisted the help of both subject matter experts and user testers to iteratively design and improve Cellverse. The objective of this paper is to share how user and expert feedback can inform and enhance the development of learning games. We describe how we gather and synthesize information to review and revise our game from in-game observations, semi-structured interviews, and video data. We discuss the input of subject matter experts, present feedback from our user testers, and describe how input from both parties influenced the design of Cellverse. Our results suggest that including feedback from both experts and users has provided information that can clarify gameplay, instruction, subject portrayal, narrative, and in-game goals.

KEYWORDS:

• Virtual reality
• educational games
• serious games
• K-12 education
• cellular biology
• collaborative learning

Acknowledgements

CLEVR is supported in part by Oculus Education. We also acknowledge the MIT students, staff, scientists, and individuals who gave us feedback on the game.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 Interview with Matthew Schnepps; Interview with Lourdes Aleman; Interview with Shane Tutweiler; Interview with David Walt.

2 Interview with Gail Jones; Interview with Connie Cepko; Interview with Daniel Kurtizkes; Interview with Tyler Krause.

3 Interview with David Goodsell.

4 Interview with Ian Cheeseman et al.

5 Interview with Hongmei Mou.

6 Interview with Brian Lin and Jaimee Elizabeth Hoefert, 26 July 2018.

7 To see gameplay videos of CLEVR, please visit education.mit.edu/project/clevr/.

Additional information

Funding

This work was supported by Oculus Education [Grant Number Gift].

Notes on contributors

Annie Wang

Annie Wang is a graduate student in the Comparative Media Studies Department at the Massachusetts Institute of Technology, working in the Education Arcade. Her interests lie in intercultural exchange, game design, and the power of new media technologies in disrupting and reshaping social science and STEM education inside and outside of the classroom. She graduated from Wellesley College with a degree in both Media Arts and Sciences (2D Design plus Computer Science) and History. Before joining the CMS program, she was cross-registered and later worked at the MIT Education Arcade and the Game Lab, where she worked to help design both touchscreen and virtual reality-based games for student learning. Outside of academia, she can usually be found researching and testing new recipes, getting lost in history museums, collecting pictures of dogs and seals, or debating the intricacies of video game lore.

Meredith Thompson

Meredith Thompson draws upon her background in science education and outreach as a research scientist and lecturer for the Scheller Teacher Education Program. Her research interests are in collaborative learning, STEM educational games, and using virtual and simulated environments for learning STEM topics. She has a bachelor’s degree in chemistry from Cornell, a master’s in science and engineering education from Tufts, and a doctorate in science education from Boston University. She has two current projects: the Collaborative Learning Environments for Virtual Reality (CLEVR) is creating a cross platform collaborative game about cellular biology, and INSPIRE is a group of education professors who are using games and simulations in teacher preparation. Thompson uses those games and simulations when she teaches the STEP course: “Understanding and Evaluating Education.” She coauthored Envisioning Virtual Reality: A toolkid for educators, published by CMU's ETC press. In her spare time, Meredith writes and sings music with her twin sister Chris (www.cmthompson.com), spends time with her two boys, writes poetry, and enjoys hiking in the woods.

Dan Roy

Dan Roy is a research scientist at the Education Arcade and the Teaching Systems Lab, designing playful learning experiences for teachers and students alike. He is the lead game designer on the CLEVR project, inviting high school biology students to explore a cell in VR and collaboratively diagnose and treat a genetic disorder. He directs the ELK project, helping teacher candidates practice understanding what students know through roleplay conversations. Dan is also the founder of Skylight Games, a social enterprise inspiring a love of learning through play, starting with languages (Lyriko). Before his current roles, he worked with the Learning Games Network on games to teach language (Xenos) and science (Food Fight, Guts and Bolts), and with the Education Arcade, helping middle-schoolers build curiosity, intuition, and comfort in math through puzzles (Lure of The Labyrinth). He has an SM in Comparative media studies from MIT and a BS in computer science from UMass Amherst.

Katharine Pan

Katharine Pan is an MIT student pursuing a major in mechanical engineering and a minor in business. She is on MIT’s varsity swim team, and is a mentor to high school students through the Amphibious Achievement team. Her hobbies include sailing and exploring different cultures and cuisines.

Judy Perry

Judy Perry oversees design, development, and research for several projects involving educational games and simulations, as well as their integration into formal or informal learning settings. Her work includes overseeing the TaleBlazer software platform, an online toolkit and mobile app for making and playing location-based augmented reality (AR) games. Perry is also project manager for CLEVR (Collaborative Learning Environments in Virtual Reality), which explores design and development of new approaches to educational applications of VR. Her research interests include playful learning, constructionism, design-based research (DBR), location-based games, participatory simulations (pSims), ubiquitous “casual” games, virtual reality (VR), augmented reality (AR), mobile games, and more generally, digital materials that foster engagement with STEM. Perry leads professional development training workshops for educators who want to implement STEP lab projects. She enjoys collaborating with other institutions (including zoos, nature centers, libraries, museums and science centers, living history museums, and K-12 classrooms) to develop experimental learning offerings.

Philip Tan

Philip Tan is the creative director for the MIT Game Lab. He teaches CMS.608 Game Design and CMS.611J/6.073J Creating Video Games. For six years, he was the executive director for the US operations of the Singapore-MIT GAMBIT Game Lab, a game research initiative. He has served as a member of the steering committee of the Singapore chapter of the International Game Developers Association (IGDA) and worked closely with Singapore game developers to launch industry-wide initiatives and administer content development grants as an assistant manager in the Media Development Authority (MDA) of Singapore. Before 2005, he produced and designed PC online games at The Education Arcade, a research group at the Massachusetts Institute of Technology that studied and created educational games. He complements a Master’s degree in Comparative Media Studies with work in Boston’s School of Museum of Fine Arts, the MIT Media Lab, WMBR 88.1FM and the MIT Assassins’ Guild, the latter awarding him the title of “Master Assassin” for his live-action roleplaying game designs. He also founded a DJ crew at MIT.

Rik Eberhart

Richard (Rik) Eberhart as Studio Manager for the MIT Game Lab, He spends his days playing Tetris: with people, boxes, tasklists, equipment, money, and time. When not staring at a spreadsheet trying to fit in another computer purchase, a last minute event budget, or placing undergraduate researchers on a Game Lab project, he’s chipping away at spreadsheets on his DS, reproducing pixel-art in Picross and Picross 3D, or managing the ultimate spreadsheet, a game of Sid Meier’s Civilization. He is also an instructor for two MIT Game Lab classes on game production and has served as a mentor and director for multiple game development projects including elude, a game about depression produced in the summer of 2010. He holds a Bachelor of Arts degree from the College of William & Mary, is a Certified Scrum Master, a PMI Agile Certified Practitioner, and is currently working towards a Serious Games MA Certificate from Michigan State University.

Eric Klopfer

Eric Klopfer is Professor and Director of the Scheller Teacher Education Program and The Education Arcade at MIT. He is also a co-faculty director for MIT’s J-WEL World Education Lab. His work uses a Design Based Research methodology to span the educational technology ecosystem, from design and development of new technologies to professional development and implementation. Much of Klopfer’s research has focused on computer games and simulations for building understanding of science, technology, engineering and mathematics. He is the co-author of the books, Adventures in Modeling, The More We Know, and the recently released Resonant Games as well as author of Augmented Learning. His lab has produced software (from casual mobile games to the MMO The Radix Endeavor) and platforms (including StarLogo Nova and Taleblazer) used by millions of people, as well as online courses that have reached hundreds of thousands. His work has been funded by federal agencies including NIH, NSF and the Department of Education, as well as the Gates Foundation, the Hewlett Foundation, and the Tata Trusts. Klopfer is also the co-founder and past President of the non-profit Learning Games Network (www.learninggamesnetwork.org).