Participatory Design of a Hybrid Kinect Game to Promote Collaboration between Autistic Players and Their Peers

ABSTRACT

We report on the participatory design of a hybrid two-player Kinect game to encourage complex emotion recognition and collaboration between autistic people and their peers. From its inception, autistic college students have been involved in an iterative process to design, evaluate, and redesign the game. The emotion recognition game has two playing phases. In the first phase, the players independently assemble pieces in a digital puzzle. In the second phase, players communicate in-person to agree on the appropriate emotion for the context and construct the emotional face for the body they have assembled together. We also designed collaborative reward games that require the two players to cooperate and one that encourages players to look at each other. In order to assess the level of in-game cooperation, we added a face tracking component that automatically quantifies collaboration and can replace time-consuming hand-coded evaluations. We report on how this game was designed and built by a team of autistic students, their peer mentors, a psychologist, computer scientists, and a graphic artist. Preliminary observations show that modeling of emotion recognition and collaboration by peers with stronger social skills is emerging as a central aspect of the effectiveness of our game. The participatory process has led to several design changes including one that dramatically increased player collaboration. We share insights and lessons learned that can guide others working in participatory design.

Notes

1. www.digitalbubbles.org.uk.

2. Mentee 3 suggested using dragons and supplied images that inspired the dragon theme.

Additional information

Funding

This project has been supported by a CUNY Interdisciplinary Research Grant (#80209-02 23) awarded to Kristen Gillespie-Lynch and Deborah Sturm, a PSC-CUNY Award (RF-CUNY award # 69235-00 47), jointly funded by the Professional Sta  Congress and the City University of New York, awarded to Deborah Sturm and a grant awarded by the FAR Fund (# 72105-03 06) to Kristen Gillespie-Lynch to support Project REACH, the mentorship program from which we are recruiting autistic collaborators.

Notes on contributors

Deborah Sturm

Deborah Sturm is a Professor of Computer Science at the College of Staten Island. She designed and teaches three gaming electives and introduced an area concentration in game development. She is on the steering committee of the CUNY Games Network and has helped organize four game-based learning conferences. She collaborates with members of the Psychology Department to design and develop research games for players on the Autism spectrum.

Michael Kholodovsky

Michael Kholodovsky is an undergraduate Computer Science major at the College of Staten Island and is part of the team that coded parts of the Unity Kinect game. He is graduating and plans to work as a software developer and study for a masters in Computer Science.

Rayan Arab

Rayan Arab is an undergraduate Psychology major at the College of Staten Island. She is a student in the Verrazano Honors Program and a mentor in Project REACH.

David Shane Smith

David Shane Smith is an artist and musician based in Brooklyn, NY.

Pavel Asanov

Pavel Asanov graduated college with a degree in Electronic Engineering. He is a founding member of Sound Facets LLC and Play Fitness Corporation and is an active member of the NYC startup community and a mentor at the Founder Institute startup incubator.

Kristen Gillespie-Lynch

Kristen Gillespie-Lynch is an Assistant Professor of Psychology and autism specialist at the College of Staten Island and the Graduate Center CUNY. She developed a mentorship program for college students with autism and other disabilities, Building Bridges Project REACH, and directs the Advanced Certificate Program in ASD at CSI.