Simple Interactions: A randomized controlled trial of relational training for adults who work with young people across settings

Abstract

Research across multiple fields finds that adult-youth relationships are key influencers of development; however, professional learning about relational practice is limited. This suggests the need for targeted, efficient ways to help adults improve relational practice. We present a randomized controlled trial of Simple Interactions, a strength-based professional development approach during which participants reflect on short videos of themselves interacting with young people using a dialogic protocol. Participating staff expressed high satisfaction with Simple Interactions, rating an average of 4.56 out of 5.00 on a 7-item composite measure. We saw an experimental effect for belief change, with treatment group staff after the workshops rating relational practice as more important than control group staff. We did not see an experimental effect for relational practice, perhaps due to substantial challenges around workshop attendance. The findings are promising for the innovation and suggest more engagement may be necessary to see change in practice.

Data availability statement

The data that support the findings of this study are available from the corresponding author, upon reasonable request.

Notes

1 We use the following terms interchangeably: afterschool programs, organized activities, out-of-school time programs, out-of-school learning programs, youth programs.

2 Prior to site recruitment, we conducted a power analysis using Optimal Design software (Raudenbush, 2011). The design achieved 80 percent power to detect effects between 0.55 to 0.59 standard deviations assuming 30 sites with 4 staff members per site (120 total staff). We based study design assumptions on our previous study which found a large effect size for the intervention (Hedge’s g of 1.14; see [Akiva et al., 2017]), and available resources to carry out the present study.

3 Data collection for the first wave of the exploratory sample also occurred before and after the intervention. During data collection visits, a researcher collected two five-minute video clips of staff interacting with children. Staff also completed a survey prior to and after the intervention.

4 Staff from one program attended workshops at a local funding agency one block from the afterschool program. A facilitator provided transportation to this site.

5 To ensure reliability, a total of 11 raters were trained for 4 hours. During training, raters discussed scoring with master coders (two raters with over one year of experience using the SIT) and then independently coded 10 videos. After the training session, coders rated 10 additional videos. Raters that met an acceptable interrater reliability score, an intraclass correlation (ICC) of 0.80, began coding. We assessed interrater reliability using a one-way mixed, consistency, average-measures ICC (Mcgraw and Wong, 1996). An ICC of 0.80 indicates a high level of consistency across raters (Cicchetti, 1994).

6 The sample sizes vary by model in Table 5 due to missing data. See the section on Missing and Balance for more details.

7 The sample sizes vary by model in Table 6 due to missing data. See the section on Missing and Balance for more information.