Developing and using matrix methods for analysis of large longitudinal qualitative datasets in out-of-home-care research

ABSTRACT

Managing and analysing large qualitative datasets pose a particular challenge for researchers seeking a consistent and rigorous approach to qualitative data analysis. This paper describes and demonstrates the development and adoption of a matrix tool to guide the qualitative data analysis of a large sample (N = 122) of interview data. The paper articulates the theoretical and conceptual underpinnings of the matrix analysis tool and how it was developed and applied in a longitudinal mixed methods out-of-home-care research study. Specific illustrations and examples of data integration and data analysis are provided to demonstrate the benefits and potentials of constructing matrix tools to guide research teams when working with large qualitative data sets alone or in combination with quantitative data sets.

KEYWORDS:

• Out-of-home-care
• methodology
• longitudinal design
• qualitative analysis

Introduction

Managing and analysing large qualitative datasets poses a particular challenge for researchers seeking a consistent and rigorous approach to qualitative data analysis (Lichtenstein & Rucks-Ahidiana, 2021). For example, large qualitative data makes it difficult to adopt traditional manualised coding techniques that are context sensitive and attentive to nuance, detail, and depth of analysis. Further, maintaining a consistent approach to analysis is particularly challenging for projects that utilise teams of coders and analysts alongside strict completion times (Lichtenstein & Rucks-Ahidiana, 2021; White et al., 2012).

The rise of qualitative big-data collected from multiple sources and across different time points has led researchers to develop innovative approaches to data organisation and analysis (e.g. Abraham et al., 2021; Brooks et al., 2015; Grossoehme & Lipstein, 2016; Lichtenstein & Rucks-Ahidiana, 2021; White et al., 2012; Winskell et al., 2018). For example, Lichtenstein and Rucks-Ahidiana (2021) used a contextual text coding method on qualitative data to produce quantitative trends, which were then used ‘ … to focus qualitative analysis on specific questions or trends of interest’ (p. 11). Quantifying the data served as a data organisation technique to give focus for contextual, nuanced and focused qualitative analytical depth (Lichtenstein & Rucks-Ahidiana, 2021). Likewise, in their research of veterans access to health services in the US, Abraham et al. (2021) developed a template and matrix to manually analyse 205 semi-structured interview transcripts and recordings. They used initial interviews to develop a template ‘ … organised by domains and categories’ (p. 143). In their research, domains are a priori constructs based on the study aims and interview questions, and categories are short inductive summary themes derived from interview data. These inductive categories were then organised into a rubric to facilitate consistent data coding (Abraham et al., 2021).

Similarly, Brooks et al. (2015) established a coding template based on a subset of data to give a ‘ … high degree of structure in the process of analysing textual data with the flexibility to adapt it to the needs of a particular study’ (p. 203). White et al. (2012) qualitatively analysed 236 interviews with nurses on their practice. The wide ranging and context rich data presented a particular challenge in establishing analytical rigour, consistency, and trustworthiness. Finally, Winskell et al. (2018) had the daunting task of analysing 75,000 narratives, which were amassed over an 18-year period as part of sub-Saharan African script writing competition on HIV related topics. A stratified random sampling of 2,000 narratives from eight time points were analysed quantitatively, narratively, and thematically. As they state, the challenge was to conduct data analysis that preserves its ethnographic, longitudinal and cross-national richness (Winskell et al., 2018).

These examples demonstrate new innovations in analytical techniques and data organisation methods to undertake qualitative analysis of large datasets. This article contributes to this literature by describing the development and application of a matrix data organisation and analysis tool used by a team of interviewers and coders, to bring a systematic focus to a large scope of qualitative interview data as part of a mixed methods longitudinal study of Out-of-home-care (OOHC). This project’s researchers faced similar challenges to those previously described. In brief, the sampling target for this study is N = 338 interviews with young people in care and leaving care. The interviews are conducted over five longitudinal waves across a 2-year period, with a maximum target of N = 1,690 interviews. This target is based on over sampling to account for attrition, however, it is likely that the final number of interviews will be in the several hundred. A full research protocol for this study is published elsewhere.

Suffice to say, a significant challenge for this project is the analysis of hundreds of qualitative interview transcripts from different sample groups, and across different time points and localities. Research teams often manage limited resources, and this is particularly pertinent with time intensive qualitative analysis (Abraham et al., 2021; Lichtenstein & Rucks-Ahidiana, 2021). The project discussed herein is no exception. Furthermore, many studies that draw on large-N qualitative data fail to transparently articulate their analytical methods (Lichtenstein & Rucks-Ahidiana, 2021). Hence, the aim of this paper is to describe and critically discuss the development and adoption of a matrix tool to guide the qualitative data analysis of large interview data.

Below we describe the study context and design, before articulating the theoretical and conceptual underpinnings of the matrix analysis tool and how it was developed in this study. Specific illustrations and examples of data integration and data analysis are provided to demonstrate the benefits and potentials of constructing matrix tools to guide research teams when working with large qualitative data sets.

Study context

Researching out-of-home-care

OOHC is accommodation, care and support provided to children unable to live with their parents due to experiences of abuse, or neglect, or a substantial risk of harm (AIHW, 2021). International evidence consistently shows many young people who have been in OOHC have poorer long-term outcomes than their peers (Kääriälä & Hiilamo, 2017; Mann-Feder & Goyette, 2019; Mendes & Snow, 2016; V Paulsen et al., 2020; A. D. P. Van Breda & Frimpong-Manso, 2020). Compared with peers in the general population, young people leaving care have higher rates of non-completion of school and overall lower levels of educational attainment (Kim et al., 2019; Refaeli & Strahl, 2014), lower employment rates (Zinn & Courtney, 2017), unstable living conditions (Bender et al., 2015; Tam et al., 2016), lower incomes and experiences of financial instability (Zinn & Courtney, 2017), poorer physical and mental health (Lehmann et al., 2013), and higher levels of crime and substance abuse (Clausen & Kristofersen, 2008; Dixon, 2008).

While the poor outcomes for care leavers are well established in the literature, there is a lack of quality research around how young people’s transitions can be supported most effectively. Much of the existing research targeting care leavers’ outcomes is retrospective, based on tracking young people entering service systems after leaving care (Mendes & Snow, 2016). A systematic review of leaving care programs concluded program effectiveness cannot be determined, as the studies lack methodological rigor (Everson-Hock et al., 2011). As there appears no significant reduction in children entering OOHC, the required evidence remains much needed.

Navigating through life: project overview

In Australia, the number of young people in OOHC has increased, up 7% between 2017 and 2020 to 46 000(AIHW, 2021, p. vi). Australian OOHC takes the form of kinship care, foster care, or residential care, while the legislation, policy, service provision and funding are the responsibility of each State in Australia (AIHW, 2021). In Western Australia, the context of this study, 4,839 children and young people aged 0–17 years were living in OOHC on 30 June 2020, a rate of 7.9 per 1,000 (AIHW, 2021).

Given the limitations of current leaving care research, policy decision-makers and practitioners have little evidence on which to implement programs that would result in better outcomes into adulthood for young people transitioning from care. To improve existing evidence, the Navigating Through Life study is a longitudinal, mixed-method, population-based study. The project aims to: 1) map the pathways and lived experience of young people transitioning out of and who have left OOHC; 2) identify key factors associated with meeting the cultural, social and developmental needs of young people transitioning out of OOHC; 3) gain a population-based understanding of the association between the multiple and varied transition pathways, and the outcomes for young people both in and who have left OOHC; and 4) identify Aboriginal family and community perspectives on important barriers and enables for young peoples’ achievement of developmental milestones (Parsons, Chung, Cordier, Hodgson, Lund, Mendes, O’Donnell, et al., 2020). The research employs a multifaceted mixed-method approach and contains three studies, reported in detail in by Parsons, Chung, Cordier, Hodgson, Lund, Mendes, O’Donnell, et al. (2020) in their research protocol.

Longitudinal prospective cohort study

The focus of this paper is Study 2, a longitudinal prospective cohort study. Employing a mixed method design, the study collected quantitative and qualitative data from two cohorts of young people: an in-care cohort aged 15–17, and an exited-care cohort aged 18–25. Data were collected at five time points in a two-year period. Young people were recruited through leaving care service networks and via Department of Communities Child Protection and Family Support District Offices. One hundred and twenty-two young people have completed the first wave of data collection. Young people participated in semi-structured, in-depth qualitative interviews, either face-to-face, over the phone or via video conferencing. Participants completed quantitative outcome measures using an iPad or via an online Qualtrics survey.

The interviews with young people addressed key life domains including: 1) current living situation; 2) planning for independent living; 3) friends, family, and connections; 4) school and education; living costs; 5) health and other services; and 6) background and identity. Additional domains were covered depending on the participants’ age and parental status including post school education and parenting. The quantitative measures were developed by the research team utilising questions from standardised psychometric measures. These measures covered the following constructs: 1) social inclusion; 2) independent living skills; 3) resilience; 4) self-determination; 5) wellbeing; 6) adverse childhood events; 7) child interaction; and 8) relationships.

Constructing and piloting a matrix analytical framework for qualitative data analysis

Theoretical and conceptual framework

OOHC literature contains multiple, at times overlapping theoretical constructs and conceptualisations of transitions into adulthood (see, Figure 1). Stein (2006) highlighted the lack of theorising in the area, suggesting potential theoretical directions to increase the incorporation of theory into the field, which has resulted in increased efforts to theorise OOHC research (Glynn, 2021). Perhaps due to Stein’s own theorising, resilience theory features heavily in studies (Daining & Depanfilis, 2007; Jones, 2012; Pinkerton, 2011; Stein, 2005, 2008; Sulimani-Aidan, 2017; Van Breda, 2015, 2017) and is used to explore personal and interpersonal factors associated with increasing resilience in transitions into adulthood. Social inclusion (Cordier et al., 2017; Kääriälä & Hiilamo, 2017), social capital (Pinkerton, 2011; Singer et al., 2013), social network (Blakeslee, 2012), identity capital and identity formation theories (Dima & Skehill, 2011; Lee & Berrick, 2014; Ward, 2011) are also prominent frameworks through which OOHC transitions are understood. Other theoretical constructs in OOHC care research include participation, particularly in education and employment (Hollingworth, 2012; Tilbury et al., 2011), attachment (Dima & Skehill, 2011; Stein, 2005), reflexivity (Hung & Appleton, 2016), cumulative disadvantage theory (Singer & Berzin, 2015), and development theories (Mulkerns & Owen, 2008). More recently, recognition theory (Glynn, 2021; V. Paulsen & Thomas, 2018) has been employed to understand how relationships, legal relations and solidarity impact transitions into adulthood.

Figure 1. Theories of out-of-home-care.

Figure 1 depicts the theoretical constructs informing the field of OOHC research. Most research has been concentrated in the areas of individual and personal theories, with some development occurring in the areas of macro and structural sociological theories. Emerging theoretical perspectives includes rights-based approaches to OOHC.

Mike Stein, in his work exploring resilience and young people leaving care, categorised care leavers into three groups – Moving on, Survivors, and Struggling – and posited the groupings may be helpful in understanding how young people experience leaving care transitions (Stein, 2005, 2006). According to Stein, the Moving on group is characterised by stability and continuity, gradual preparation for leaving care, and having a ‘post-care normalising identity’, which might include participating in higher education, having a desirable job or becoming a parent (Stein, 2005, p. 19). Those in the Surviving group experienced more instability, movement and disruption compared to the Moving on group, while also believing challenges had contributed to their maturity and self-reliance. They often saw themselves as independent, while also relying heavily on agencies for accommodation, financial and personal support. Out of the three groups, the Struggling group experienced the most disadvantage and the highest number of care placement moves while in care. They were also more likely to experience unemployment, homelessness, feelings of isolation, and mental health challenges.

It should be pointed out that Stein’s conceptual framework is an analytical heuristic that can be used to organised young people’s experiences into groupings, rather than a deterministic or predictive model of leaving care. Many young people who have poor care and care leaving experiences may flourish after leaving care; the converse may also be true. Likewise, some in the Moving on group may experience health, mental health, employment, and other challenges that are a normal part of life. Hence, the categories should not be treated as deterministic, nor are they exclusive.

Stein (2008) argues the categories are closely related to resilience. He noted placement stability promoted resilience by providing warm and redeeming relationships with a caregiver, and continuity of care in young people’s lives. Conversely, placement instability was seen as a barrier to resilience hindering young people’s ability to form relationships with helpful adults and peers. Helping young people to develop a positive self-identity was also noted as important to promoting their resilience (Stein, 2008).

Domains of inquiry in navigating through life

Navigating Through Life has used Stein’s categorisation with data from the Longitudinal Prospective Cohort study’s interview tool. The eight domains in the tool are underpinned by key theoretical constructs in the OOHC literature (see Supplementary Table S1).

The current living situation domain asks questions about young people’s living arrangements, giving an indication of placement type and levels of transience or stability. Planning for independent living domain explores conceptualisations of independent living and access to practical planning support. The friends, family and connections domain addresses social networks and supports to which the young person is connected. The school and education domain (and post-school) explores barriers and facilitators to education attainment and plans for further education, training or work. The living costs domain explores young peoples’ economic inclusion, if and how this impacts social inclusion. The health and other services domain addresses both access to and knowledge of health systems and services, while the background and identity section asks about young people’s sense of identity, knowledge of family history, and desire to learn more about this area.

Some constructs may appear in multiple domains. For example, information about relationships and social connections are indicated in the friends, family, and connections domain, as well as living costs, and health and other services domains as these prompt young people to identify a ‘go-to’ person through whom they might access support.

The matrix was developed using the aforementioned theoretical constructs as a framework for organising interview information. The interview tool domains run along the matrix’s x-axis while Stein’s care leaver categorisations provide a continuum on the y-axis. The framework allows for interview data to be condensed into a score and summative comment within each domain with the intention of facilitating comparisons between domains and within the dataset.

Developing the matrix

The matrix was developed to align with the qualitative interview schedule’s domains, including the post-school and parenting domains. Each domain contains a descriptor for a Struggling (1 or 2), Surviving (3 or 4), and Moving on (5 or 6) score (see Supplementary Table S1). For example, regarding living arrangements, the matrix encapsulates the descriptors of the Struggling category as ‘insecure, unsafe accommodation; high transience; limited resources or support for accommodation; history of homelessness (primary, secondary of tertiary), no fixed address, temporary living arrangements; feel unsafe; very dissatisfied with current living situation’, through to the Moving on descriptors of ‘stable and secure accommodation; low transience; accommodation support/resources available; young person satisfied with living situation; in control of their living situation; have sufficient income to meet accommodation and living costs’. Likewise, in respect to planning for independent living, the descriptors for the Struggling category move from ‘no evidence/recollection of leaving care planning; leaving care planning poor quality; young person very unprepared to leave care; limited to no appreciation of what is involved in independent living; young person does not recall or indicates that there has not been any leaving care planning or actions; very dissatisfied with lack of leaving care support; ambivalent about what improvements can be made; strong and critical views over lack of support’ through to the descriptors of the Moving on category ‘clear evidence of quality leaving care planning; detailed planning and support; strong understanding of independent living and how to attain it; young person makes direct links between leaving care planning and being prepared for independent living.’ The matrix therefore allows for the interviewer and analyst to make sense of the data by coding and organising the transcript in accordance with the various descriptors associated with OOHC.

The process to create the matrix required several steps. Firstly, five audio interviews were randomly sampled from the first wave of data collection, and each was scored using the matrix by five researchers within the team. Second, each interview was reviewed in terms of how well the matrix captured the nuance of the young person’s situation. Consequently, an additional in-care matrix was developed, omitting the post-school section and descriptors adapted to respond to young people in care’s generally higher dependence on carers and others for financial and other support compared to care leavers. Both matrices were altered to increase focus on knowledge about finances, services, and systems and options to access these. In the living costs section, descriptors were included on the impact of a young person’s financial situation on their social participation. Finally, a one-page guide for interviewers was created, which outlined key foci for scoring within each domain.

Overall, the team deciding scoring within each domain should account for both what the young person said and the interpretations of the interviewer. For example, if a young person interviewed had a generally positive experience of school but did not complete school and did not go into further education or training, both factors were taken into account. Scores also considered context by providing information in the summative comment of the young person’s present situation. That is to say, the interviewer would summarise the interview in writing, paying attention to salient aspects of the participants story, including background or contextual factors that accounted for their current circumstances, any emotionally significant components that seemed important to the participant, and other nuances or narrative details that would give depth of meaning to the matrix scores. Overall vulnerability in transitional experiences and circumstances were to be considered, and emphasis placed on qualitative interpretation rather than numerical scoring. Thus, the benefit of the matrix is to function as an analytical tool to organise qualitative data in a quantitative form, but also to work as a structured tool for representing interview data into qualitative summaries.

Data management and retrieval

The prospective longitudinal study collects, stores, and manages all quantitative data using REDCap electronic data tools hosted at our University. REDCap (Research Electronic Data Capture) is a secure web-based platform designed to support data capture in research studies (Harris, Taylor, Minor et al., 2009; Harris, Taylor, Thielke et al., 2009). Data collection forms for the outcome measures were built within REDCap and then organised into the five waves of data collection. Using the REDCap app young people complete the outcome measure data collection forms on an iPad™, and researchers complete the matrix scores and comments within REDCap after each interview.

Incorporating the Matrix into the REDCap platform where the quantitative data is also collected and managed has several advantages. Matrix scores and comments are collated electronically and organised into waves against the same unique participant identifier (ID) as all other data collected in the study, simplifying data management across all aspects of the study. As Matrix scores are electronically linked to other participant data (e.g. outcome measures, demographics, care system interactions), matrix scores for each wave of data collection can be easily extracted for subsamples of participants identified as critical for qualitative data interrogation. Furthermore, matrix data can be extracted either on their own or in conjunction with the other data collated within REDCap, in formats compatible with a range of statistical packages, allowing for matrix scores to be utilised in statistical modelling.

Application and validation to data analysis

Here, we will detail how the matrix has been used in a mixed methods study to address the aims of the Navigating Through Life project. The objective of this exemplar study was to identify factors that predicted young people’s mental health, resilience, self-determination and social inclusion, and the experiences that might explain young people’s outcomes in these domains. The objective was addressed using a fully mixed, sequential, quantitative methods dominant design (Nastasi et al., 2015). The data for this example were drawn from all care experienced young people who participated in Wave 1 of Navigating Through Life’s prospective longitudinal study (N= 122). Quantitative and qualitative data were available for all participants. While the large sample size was advantageous for our quantitative analysis as it provided increased statistical power, it posed a challenge for developing a qualitative analytical approach with nuanced and focused analytical depth. This example describes how the matrix was used in combination with the quantitative data analysis to narrow the qualitative data sample for a more consistent and rigorous approach to the qualitative data analysis.

Data preparation

The quantitative dataset included the child protection administrative data, demographic information, and outcome measure scores for all young people who participated in wave 1 of the study. Outcome measures administered included the Connor-Davidson Resilience Scale (Connor & Davidson, 2003), AIR Self Determination Scale (Wolman et al., 1994), Strong Souls (Thomas et al., 2010), and a measure of social inclusion developed specifically for this study. Each measure provided a single score representing the constructs of resilience, self-determination, mental health, and social inclusion respectively. All quantitative variables were collected using REDCap and exported to SPSS Statistics for Windows Version 27 (IBM Corp, 2020) for analysis. Interviews from the same 122 participants were recorded and then transcribed verbatim to form the qualitative dataset, with transcript documents saved on a secure server using the same participant IDs as those in REDCap. Interviewers scored the matrix for each participant after completing each interview, and matrix ratings were stored in REDCap and exported to SPSS for analysis along with the quantitative data.

Data analysis

In alignment with the sequential design of this mixed methods study, quantitative data analysis was conducted first, to identify factors that were significant predictors of young people’s scores on the self-report outcome measures of self-determination, social inclusion, mental health, and resilience (the dependent variables). First, univariate linear regression modelling was used to explore independent variables that predicted young people’s scores for each outcome measure. Independent variables were related to care experiences (i.e. placement stability, most recent type of placement, longest placement type, total time in care), demographics (i.e. gender, Aboriginal status, location), adverse childhood experiences, and care status at the time of data collection (i.e. in care vs already left care). Outcome measure scores were also used as independent variables for univariate analysis, except when there was evidence of collinearity between outcome measures. For example, self-determination, social inclusion and resilience were entered as independent variables, against the dependent variable of mental health.

Next, independent variables that were significant predictors (p< 0.05) of the four dependent variables at a univariate level were entered into multivariate linear regression models for each of the four dependent variables. Using a process of backwards elimination, independent variables that were not significant in the multivariate models were removed, until one final model per dependent variable was determined. The quantitative results consisted of four multivariate models representing the combination of independent variables that best predicted young people’s resilience, self-determination, mental health and social inclusion.

Finally, qualitative data analysis was used to explore young people’s life circumstances during their transition from care in relation to their resilience, self-determination, mental health and social inclusion as a way of explaining the quantitative results. The matrix tool was utilised to select a sample of transcripts that would explore the breadth of young people’s experiences, while also remaining of a manageable sample size. Univariate regression modelling was used to determine whether initial qualitative interpretation of a young person’s broader life circumstances during their transitioning from care (i.e. participants’ overall matrix score) was associated with the constructs explored in the quantitative analysis. Results showed that the overall score obtained from the matrix was a significant, positive predictor of the constructs measured by the outcome measures (see, Table 1). Placement towards the Moving on end of the continuum of leaving care experiences operationalised in the matrix (based on our interviewer’s initial qualitative analysis) predicted more positive self-reports of resilience, self-determination, mental health and social inclusion. Conversely, self-reports of poorer resilience, self-determination, mental health and social inclusion were predicted by placement towards the Struggling end of the continuum of leaving care experiences.

Table 1. Univariate linear regression modelling of the association between matrix scores and outcome measures.

	Unstandardised Coefficients
Model	β (95% CI)	Standard Error	p	r^2
1. Dependent variable = CDRS (resilience)
Overall Matrix Score	0.324 (0.141–0.508)	0.093	<0.001	0.098
2. Dependent variable = AIR Self Determination Scale
Overall Matrix Score	0.419 (0.250–0.588)	0.085	<0.001	0.178
3. Dependent variable = Strong Souls* (mental health)
Overall Matrix Score	−0.444 (−0.624 – −0.265)	0.091	<0.001	0.176
4. Dependent variable = Social Inclusion measure
Overall Matrix Score	0.378 (0.263–0.494)	0.058	<0.001	0.271

Note. CDRS = Connor-Davidson Resilience Scale; *A higher Strong Souls score indicates poorer mental health.

Results from the regression modelling provided a data-driven way of identifying a relevant subsample of the 122 interviews to analyse to meet the objectives of the study. We opted to draw a random subsample of 32 transcripts from either end of the leaving care continuum; 16 from young people with an overall matrix score in the Moving on range, and 16 from young people with a score in the Struggling range of the overall matrix continuum. The sample was also stratified by care status (i.e. half of the young people were in care at the time of their interview, and half had left care), as this was also a significant predictor of all four dependent variables in the quantitative analysis stage. A random number generator (random.org) was used to select the random sample of transcripts (Haahr, 2010), and a thematic analysis approach (Braun & Clarke, 2019) was taken to identify key themes from young people’s experiences that were associated with resilience, self-determination, mental health and social inclusion. An initial sample of 32 was selected as an inductive thematic approach to saturation (Saunders et al., 2018), as we thought it unlikely that new codes and themes would be identified in a larger sample.

Discussion

This longitudinal cohort study is one of three sub-studies comprising the Navigating Through Life study. This study is aimed at addressing questions about pathways and outcomes of young people with care experience that will provide evidence for decision makers to improve policies and programs aimed at meeting the complex and intersecting needs of care experienced young people.

Our findings revealed the capacity of a matrix tool to meaningfully manage large-scale, longitudinal qualitative data. Set against a backdrop of similarly innovative approaches to organisation and analysis of large datasets (e.g. Abraham et al., 2021; Brooks et al., 2015; Lichtenstein & Rucks-Ahidiana, 2021; Winskell et al., 2018), this matrix is a fine-grained tool that can handle non-homogenous cohorts so that both trends within and between subgroups can be followed. Moreover, because the matrix incorporates qualitative data from the young person’s interview, the participants’ own voices – as opposed to linked or administrative data – provide the basis for leaving care classification (i.e. Struggling, Surviving and Moving on). Similar processes of matrix development and testing could be applied to a range of qualitative data studies.

One of the benefits highlighted was the link between theory and the matrix, which opens possibilities for critical evaluation of OOHC theorising. The matrix used Stein’s three classifications of Struggling, Surviving and Moving on for initial data management and analysis, and we are still working on developing other theoretical approaches that could be used for the matrix in this study. Social and individual level theoretical analyses are central to the development of evidence about young people in OOHC, so this will be the basis of the next iteration of the use of the matrix. From Stein’s work (Stein, 2005) on resilience theory, his conceptual framework was used as a helpful analytical heuristic for the matrix’s three grouping classifications (i.e. Struggling, Surviving, and Moving on). Where Stein suggests a link between young people’s placement stability/instability and their ability to form relationships, which he defines as a function of resilience, these findings may also have implications for theoretical approaches and fields of study (e.g. attachment theory, social inclusion, recognition theory). With the additional domains used in the matrix to look at various aspects of young people’s lives, our study enables a more nuanced examination across the components of their lives and over time. Thus, as an organisational and analytical tool, this matrix may provide a key contribution to the development of theory and constructs with respect to young people experiencing adversity, especially within OOHC theory.

While this example demonstrates the use of the matrix in relation to the overall score it yields and the data collected in Study 2, there are other potential and intended uses of the matrix within Navigating Through Life. A similar approach could be used to sample transcripts for research questions pertaining to particular outcomes found in the linked administrative data (e.g. education, housing, early parenthood). Domain specific ratings within the matrix would inform a sampling frame for transcripts based on the domain of interest, and qualitative findings would provide context to the quantitative analysis, and also enhance the interpretation of data from the linked datasets (Chikwava et al., 2021; Harron et al., 2017).

Another research application for the matrix is to inform extreme case sampling (Patton, 2015). The matrix scores, and their trends over time (per domain, or overall), can be used to inform purposive sampling of young people to approach for further in-depth narrative interviews (Holloway & Jefferson, 2000). The matrix allows us to identify young people within the sample who have either very poor or very positive outcomes in relation to their care-experienced peers, or young people whose outcomes appear to improve or decline over time. Further in-depth interviews with these young people can enable a deeper understanding of factors which may have contributed to young people’s outcomes after leaving care.

Finally, as the matrix tool helps inform further quantitative analysis within structural equation modelling (SEM) of longitudinal data. Within SEM, applications such as Latent Growth Model (LGM) and Growth Mixture Model (GMM) can face difficulties when statistical assumptions are not satisfied, but this matrix can help to address these difficulties. For instance, the matrix can help us to identify where individuals have deviated from the longitudinal growth trajectories predicted by a given statistical model, such as when they change classification across the lifetime of the study. Where hardships or life events associated with a particular classification may be expected to adversely affect an individual, the matrix allows us to make meaning of divergences from this prediction, thus illustrating where resilience allows them to overcome these issues and thrive over time.

The matrix development is a pilot study that is still being used and tested by the researchers. The initial sample size was limited due to time and resource constraints. Secondly, there are always limitations to the generalisability of qualitative findings using one set of data from one country. It is therefore difficult to know if these findings will be applicable outside of the present context even though it is likely given the similarity of existing international evidence from OOHC research.

Conclusion

The matrix was developed specifically for this mixed methods study, so it requires further testing and replication. This matrix has been developed at the population level; however, the population is not representative for either in or OOHC populations, which suggests that it should be tested with other population groups. Additionally, the process of developing, applying, and validating this matrix (i.e. utilising a fully mixed, sequential, quantitative methods dominant design) also needs to be replicated within populations other than those in OOHC. After further testing and replication, the method and matrix can be published more widely, thus garnering wider acceptance within the field.

The use of technology developments has enabled researchers to develop far more efficient methods of analysing qualitative data sets. The challenge is to ensure that the quality and rigour of the analysis is not compromised in such processes. A key benefit from the development of the matrix in this study of OOHC outcomes has been that data collection technology enables us to use the language and expressions of participants to support an accurate representation of their accounts. The next iteration of the matrix in this study is to draw on a range of theoretical frames to analyse the qualitative data.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/13645579.2022.2159323

Additional information

Funding

The Navigating Through Life project is funded by an Australian Research Council Linkage Grant (PROJECT ID: LP170100044) in partnership with the Western Australia Department of Communities and Wanslea Family Services. The funding sources have no role in the study design or data collection and have no role in data analysis, data interpretation or writing.

Notes on contributors

David Hodgson

David Hodgson is Senior Lecturer in Social Work at Curtin University, Perth, Western Australia. David has conducted research in the areas of compulsory school leaving policy, school attrition, services for families and children, mental health, social justice theory, and social work curriculum. David was Associate Editor for the journal Australian Social Work (2018-2022), and he has co-authored two theory texts: Key Concepts and Theory in Social Work (2017, Palgrave) and Social Justice Theory and Practice for Social Work: Critical and Philosophical Perspectives (2019, Springer).

Reinie Cordier

Reinie Cordier is a professor at the Department of Social Work, Education and Community Wellbeing, Northumbria University, Newcastle. His main research interests lie at the intersections between health and social care, and a major focus for his research is on how to ensure that people from different populations and social strata experience participation, remain engaged, socially connected and live healthy lives.

Lauren Parsons

Lauren Parsons is a Research Fellow at Curtin University at the Curtin School of Allied Health in the Faculty of Health Sciences. Her research has focused on promoting the participation of vulnerable children and young people through the development and evaluation of psychosocial interventions, mapping lived experiences with later outcomes, and finding new ways for marginalised families to access allied health services. Using a range of research methodologies Lauren collaborates with colleagues across the disciplines of speech pathology, occupational therapy, social work, and psychology, and is keen to promote evidence-based practice and policy decision making within the health and human services.

Brontë Walter

Brontë Walter is a social work PhD candidate and sessional academic at Curtin University, with an interest in critical social work research which promotes social justice and change. She has worked with Navigating Through Life since 2019, and other research in the areas of out of home care, family domestic violence, disability, and housing.

Fadzai Chikwava

Fadzai Chikwava is a PhD student at Curtin University. She is studying resilience and mental health among young people transitioning from care in Australia. She is also working part-time at the Mental Health Commission as a Senior Evaluation Analyst. Prior to this, she worked for various organisations where she conducted research in public health and higher education programs. She has written a systematic review paper on research using population-based administration data integrated with longitudinal data in child protection settings and a paper on the relationship between homelessness and mental health among young people transitioning from care.

Lynelle Watts

Lynelle Watts is an Associate Professor in Social Work at Curtin University. Lynelle has conducted research in the areas of family and domestic violence, extended care for care experienced young people, reflective practice and critical reflexivity in social work, teaching and learning in higher education, social work curriculum, and assessment tools for carers of people experiencing mental distress. At the time of writing Lynelle is the serving President of Australia New Zealand Social Work Welfare Education & Research (ANZSWWER) - a peak body for social work and welfare educators. Lynelle is Social Media Editor for the journal Australian Social Work. Lynelle is the co-author of two books Key Concepts and Theory in Social Work (2017, Palgrave) and Social Justice Theory and Practice for Social Work (2019, Springer). Lynelle is active on Twitter – you can find her @watts_lj

Stian Thoresen

Stian Thoresen is an experienced researcher around vulnerable and excluded populations, including young people leaving state out-of-home care and persons with disabilities. Most of the studies have focused on social and economic inclusion as well as transition to adulthood. He has extensive research experience with clients from NGOs, businesses, government agencies, and the Norwegian and Australian Research Councils. He has led several projects in Norway, Australia, and Southeast Asia.

Matthew Martinez

Matthew Martinez is a Postdoctoral Research Fellow at Northumbria University, where they are involved in various research projects in the Department of Social Work, Education and Community Wellbeing.

Donna Chung

Donna Chung is the John Curtin Distinguished Professor of Social Work at the Curtin School of Allied Health. Her research focuses on male violence against women, this has included victim-survivor experiences and outcomes as well as research about perpetrators of violence, abuse and coercion. She works closely with governments on legislation, policy and service design to reduce violence against women.