Construct validity of the simplified Chinese version of the instrument ‘Picture My Participation’

Abstract

Background

Preliminary evidence of the content validity of the simplified Chinese version of ‘Picture My Participation’ (PMP-C; Simplified) items and reliability of the subscale attendance for the effectiveness of the use with children and youth in mainland China has been collected. However, evidence of construct validity for the instrument is not yet available.

Aim

To explore the construct validity of the attendance scale in PMP-C (Simplified).

Methods

A cross-sectional study using convenience sampling was conducted using PMP-C (Simplified) with a picture-supported interview for 290 children and youths aged 5–21 with and without ID in urban and rural areas of mainland China. Exploratory factor analysis (EFA) was performed using the principal component analysis (PCA) to analyse the resulting data.

Results

The EFA extracted five factors with eigenvalues greater than one and the cumulative contribution rate of factors accounted for 51.62% of the variance. All items had factor loadings above 0.50. The five subcomponents included: organised activities, social activities, taking care of others, family life activities and personal care and development activities.

Conclusion

The results of the factor analysis support the construct validity of the PMP-C (Simplified) attendance scale. It provides further psychometric evidence that PMP-C (Simplified) is a sound measure to assess participation for children and youths in mainland China.

Keywords:

• China
• disabled children
• participation
• everyday activities
• self-report

Introduction

In the report Global Disability Action Plan 2014–2021 of the World Health Organisation (WHO), it states that all persons with disabilities and their families should have the right to build a world where they can fully achieve and live with dignity, equal rights and opportunities [1]. Participation in everyday activities is as an important outcome for children, especially for children with intellectual disability (ID) [2,3]. Participation offers children opportunities to develop skills and build self-esteem, contributing to a positive effect on mental and physical well-being [4].

According to the biopsychosocial approach to disability provided by the International Classification of Functioning, Disability and Health (ICF) and the Family of Participation-Related Constructs (fPRC) framework, participation varies by culture and setting and is a complex phenomenon, including attendance and involvement [3,5,6]. Measurement of ‘frequency of attendance’ in life situations or activity settings may be deemed as ‘objective evidence of engagement’, while the involvement is the subjective feeling when attending in a variety of activities [3]. Attendance is a necessary but not sufficient condition for active involvement. Studies have shown that these objective and subjective constructs are not conflicting but complementary [7] and should be evaluated with the national setting in which there are applied [8].

ID is a prevalent form of non-progressive neurodevelopmental impairment [9], with an estimated prevalence rate of approximately 1% around the world [10]. Previous studies indicate that participation restrictions are a result of the dynamic interaction between health conditions and the environment [6]. Specifically, it is known that the participation of children with ID is limited compared to their peers with typical development (TD) [11–13], especially for the activities outside the family, and present personal and environmental barriers to play [2,14]. The United Nations Convention on the Rights of the Child (UNCRC) underlines child’s rights to express their own opinions and perceptions of participation [15], even though they have an intellectual impairment [16]. With reference to the Second China National Disability Sample Survey statistics, it showed the prevalence of ID based on the national population reached 7.5 per 1000 in China [17]. However, there is a shortage of good instruments measuring participation for children with ID in mainland China, let al.one children’s own self-reports.

Children with ID which are considered a vulnerable population group are characterised by limitations in intellectual functioning and adaptive conduct [9]. They often have limitations in sensation, perception, cognition, communication, and behaviour, and may have other health conditions (i.e. epilepsy, obesity, asthma, and congenital heart defects) that influence participation. The important aspect is that cognitive/intellectual impairment can make it difficult to understand terms and constructs in non-adapted instruments. Therefore, it is challenging to have a standardised, self-report measure of participation in everyday activities for children with ID in mainland China context. Previous studies demonstrated that most instruments used to assess participation are developed and validated in western high-income countries and belonged to parent-/caregiver-report questionnaires and thus do not give a voice to children themselves.

‘Picture My Participation’ (PMP) is unique as it is a self-report measure of participation and environment that focuses on children and youths aged 5–21 from low socio-economic backgrounds [18]. It is novel in form, drawing on the experience of ‘Talking Mats’ method [19], using a clear, simple culture and contextually appropriate language and pictures, which provide both an interactive and a visualised way to enable children and youth with ID express participation experiences and goals. It is based on the ICF and UNCRC in collaboration with the United Nations Children’s Fund (UNICEF) [20], as an adaptation of the Participation and Environment Measure for Children and Youths (PEM-CY) [21] and the Paediatric Activity Card Sort [22,23]. Participation measures should meet requirements for the socialisation of children within their different cultural contexts [24]. The instrument has previously been verified in Sweden, South Africa and Taiwan, China as having sound psychometric properties as a promising universal utility screening tool for children with and without ID in everyday contexts [5,25,26].

Under the guidelines for the cross-cultural adaptation process of self-reported measures, the English version of PMP was translated into PMP-C (Simplified) [28]. After the transcultural adaptation and adjustment, five items were slightly modified, and eight pictures were revised to improve their fit with the culture of mainland China [27]. The PMP-C (Simplified) as a paper-based questionnaire format for self-report or structured interview is supported by pictures to help the children with ID understand the item content. The initial psychometric study of the PMP-C (Simplified) with 20 items showed good face validity and content validity, and excellent internal consistency and test-retest reliability of the subscale attendance for children and youths with and without ID in mainland China [27]. However, for the PMP-C (Simplified) to be used in mainland China, the construct validity of the subscale attendance in the PMP-C (Simplified) must be examined in a large sample in mainland China. The Consensus-Based Standards for the Selection of Health Status Measurement Instruments (COSMIN) [29] defined ‘Construct validity’ supports a particular interpretation of scores from a measure [30]. The construct of participation is comprehensive, encompassing a true reflection of the actual functions of children in real daily life. Participation in ICF [31] is defined as ‘involvement in a life situation’ and thus is contextualised. For individuals, it is best seen as a profile where participation vary between contexts. The instrument therefore requires that subcomponents that share important contextual features can be identified, e.g. family activities or social activities. The scores of subcomponents will facilitate the selection of what type of activities to aid for when setting participation goals for intervention [25]. Because of the dependency of context, the actual content of activities can vary between countries/cultures and therefore the construct validity must be tested for mainland China. As the attendance scale and the involvement scale of PMP-C (Simplified) with the same items, and attendance is a necessary step for involvement, it may be sufficient to only assess the attendance aspect of participation for screening purposes in children’s rights context [18]. Thus, the main aim of the current study was to extend our prior psychometric work in a large sample only focus on exploring the attendance aspect of PMP-C (Simplified) regarding construct validity by identifying and describing possible subcomponents.

Materials and methods

Design

A cross-sectional validation study design was used to explore the construct validity of the attendance aspect of the PMP-C (Simplified).

Settings

To obtain more representative data of the PMP-C (Simplified), the survey took samples from Tianjin and Hebei provinces, the Guangxi Zhuang and the Xinjiang Uygur Autonomous Region in mainland China. In 2021, the total population of Tianjin was 13.73 million, and the registered population of Hebei Province was 74.48 million. The permanent population of the Guangxi Zhuang Autonomous Region at the end of the year was 50.37 million, while for the Xinjiang Uygur Autonomous Region it was 25.89 million.

Participants

Children and youths with and without ID were recruited using a convenience sampling method of students at special needs schools and mainstream schools in Tianjin and Hebei provinces, the Guangxi Zhuang and the Xinjiang Uygur Autonomous Region in mainland China. Data collection took place from September 2019 to May 2021. A sample of 96 children and youths with ID and 194 children and youths with TD aged five to 21 years were recruited; for details see Table 1. Children and youths with ID were eligible for inclusion if they attended a special school or had been historically diagnosed at a hospital with ID according to the Chinese Wechsler Intelligence Scale (C-WISC) or the Infant-Junior Middle School Students’ Social Life Ability Scale (revised version of the Japanese S-M Social Life ability test) and the impairment was considered mild or moderate. Children and youths with either ID or TD needed to meet the following criteria to be included: (i) aged between five and 21 years; (ii) able to understand Mandarin, pass three trial items of understanding questionnaire items and express their thoughts verbally; (iii) Participants and their caregivers had to give informed consent and participate voluntarily. Children and youths with uncorrected visual or hearing impairments, and those with unstable health conditions such as cancer, who had had surgery within the previous three months, had infections or other active medical conditions that affected their daily lives, or had a progressive disease (e.g. neuromuscular disease) or a degenerative disorder (e.g. degenerative seizures), were not included. Participants were not included if they had a missing data of attendance scale of PMP-C (Simplified).

Table 1. Demographic characteristics of children and youths.

		Children with ID (n = 96)	Children with TD (n = 194)	All participants (n = 290)
Gender: n (%)	Girls	34 (35.4%)	105 (53.1%)	165(56.9%)
	Boys	62 (64.6%)	91 (46.9%)	125(43.1%)
Age, yr (SD)	Min-Max	6-21	7-18	5-21
	Mean (Sd)	13.8 (3.9)	13.6 (2.6)	13.7(3.1)
	Median	14.0	14.0	14.0
Area	Urban: Tianjin	64	49	113
	Rural: Hebei	26	55	81
	Rural: injiang	3	47	50
	Rural:Guangxi	3	43	46

Instrument

The PMP-C (Simplified) contains 20 items and uses culturally sensitive pictures to help children and youths identify participation in home, social and community activities. The items were chosen by reassessing existing participation measures and matching items to the UNCRC [26]. There are 12 pictures/symbols freely available from BildstodTM (www.bildstod.se) and eight pictures that were newly drawn by computer to improve their fit with the culture of mainland China [27]. It is comprised of three trial items with graphic symbols that were used to see if the children could interpret al.l the instructions and items on the PMP-C as intended and could make clear distinctions when applying the 4-point rating scale. After the trial items, four sections are following to [18]: (1) frequency of attendance for each item, (2) level of involvement when performing each item, (3) choice of three important activities determined independently, and (4) evaluation of perceived barriers to and facilitators of participation. The interview takes about 10–20 mins and varies depending on the communication skills of each child or youths. In the current study, construct validity was explored using data from only the first section of the scale (frequency of attendance).

Data collection

All the participants and their caregivers were informed about the purpose of the study. A semi-structured interview with picture-supported, only use the ‘Talking Mats’ format as an inspiration [32] was administered with the children. This combined format uses clear, plain culturally and contextually appropriate language understandable to children and does not require reading or writing skills. To begin with, the interviewer sat next to the child at a table and placed one mat in front of the child and showed the pictures of ‘frequency’, which was divided into four columns using a four-point Likert scale represented by four pictures of apples and baskets: ‘Never’ (empty basket), ‘Seldom’ (basket with two apples), ‘Sometimes’ (basket with five apples) and ‘Always’ (basket filled with apples) [26]. Frequency ratings reflect how often participation in this activity has occurred over the past two weeks (ranging from ‘1’ to ‘4’), such that higher scores indicate greater participation. Then, three trial items were used to see if the child understood the content of PMP-C (Simplified) and the process of rating the activity. The children were asked ‘How often do you participate in daily routine activities’. At the same time, the picture of the specific activity was shown to the children and then they were required to place it on the suitable column representing that best represented the frequency of their participation in this activity. Finally, if the child was included, pictures corresponding to 20 items were shown by the interviewer to help the children to show the frequency with which they engaged in each activity, and then the responses to each item were recorded on separate score sheets. Four researchers who had experience working with children with ID or who were trained in interviewing children using PMP-C (Simplified) conducted the interviews. Information about the child and family demographics, including date of birth, gender, and level of perceived disability, were assessed by a short questionnaire completed by primary caregivers.

Data analysis

The data were processed using SPSS Statistics 27.0. For nominal data, codes were assigned on an arbitrary basis (e.g. Male ‘1’, Female ‘2’). The responses on the four-point Likert scale in PMP-C (Simplified) were coded from 1 to 4: 1 = never, 2 = seldom; 3 = sometimes, 4 = always. An EFA performed using the principal component analysis (PCA) and the varimax rotation method was used as the extraction method to explore the construct validity of the PMP-C (Simplified). The Kaiser-Meyer-Olkin (KMO) and Bartlett’s test of sphericity coefficient were used to assess the suitability of the factor solution [33]. Bartlett’s test should be significant, and the KMO should exceed a value of 0.6. The number of factors obtained from parallel analysis was compared to that of the factors obtained from the eigenvalue [34]. An eigenvalue of over one was regarded as the criterion for factor extraction. Factor loadings greater than 0.50 were considered significant.

The naming of a factor based on its conceptual underpinnings remains subjective. A factor must be labelled to capture the whole conceptual meaning of each variable explicating a particular latent dimension [35]. As the factor loadings of the PMP-C (Simplified) were similar to the PMP’s, the researchers drew on the theoretical and research literature of PMP supporting the interpretability of the statistical result. Considering the similarities and the differences within the five subcomponents based on cultural differences between different countries, researchers decided regarding how to name each of the five factors and keep the stability and the utility of the PMP to a certain extent.

Ethical considerations

The study adhered to the principles of the Declaration of Helsinki and was approved by the Tianjin Medical University Ethics Committee (TMUhMEC2016017) as well as the relevant school principals. The participants and their caregivers received written information in a envelope including a consent form, a demographic form and an introduction letter regarding the aim and procedures of the study. They were informed that data was confidential and that they were free to withdraw from the study without giving any reason.

Results

The Bartlett Sphericity test of the scale reached a significant level (χ2 = 1088.094, p < 0.001), indicating that data from the scale was suitable for factor analysis. The statistical value of KMO (Kaiser Meyer Olkin) was 0.734, indicating that the partial correlation between variables was very weak, and the effect of factor analysis was good [35]. The parallel analysis suggested a five-factor solution, where eigenvalues greater than one were retained and the cumulative contribution rate of factors was 51.62%, as presented in Table 2. The EFA extracted five subcomponents from the children’s responses and the factor loadings of all the items are presented in Table 3, there was no item with a factor loading of less than 0.50 or cross loading, so all 20 items were retained. Items were retained on the factor with the strongest theoretical alignment, five-dimensionality was supported. Hence the result of the EFA in PMP-C (Simplified) indicated five subcomponents, including organised activities, social activities, taking care of others, family life activities and personal care and development activities.

Table 2. PCA extraction with total variance explained, initial eigenvalues, and after rotation.

Component	Initial Eigenvalues			Rotation Sums of Squared Loadings
	Total	% of Variance	Cumulative %	Total	% of Variance	Cumulative %
1	3.442	17.209	17.209	2.793	13.965	13.97
2	2.249	11.246	28.455	2.194	10.971	24.94
3	1.829	9.147	37.602	1.873	9.363	34.30
4	1.629	8.145	45.747	1.773	8.867	43.17
5	1.174	5.872	51.619	1.691	8.453	51.62
6	.968	4.839	56.458
7	.929	4.645	61.103
8	.852	4.261	65.364
9	.816	4.081	69.445
10	.749	3.743	73.189
11	.700	3.502	76.690
12	.647	3.236	79.926
13	.601	3.005	82.931
14	.574	2.869	85.800
15	.566	2.828	88.628
16	.532	2.659	91.287
17	.492	2.459	93.746
18	.466	2.332	96.078
19	.417	2.086	98.164
20	.367	1.836	100.000

Table 3. Factor loadings for five factor EFA solution.

Variables Component
	1	2	3	4	5
C4: Gathering supplies	.710	.030	.076	.056	−.079
C6: Cleaning at home	.697	.109	.100	.118	−.010
C15: Shopping: Shopping and errands (market)	.696	.077	−.073	.059	.034
C12: Organised leisure	.636	−.046	−.008	.120	.136
C19: Overnight visits and trips	.604	−.133	.211	−.033	.208
C17: Health centre	.574	−.040	.012	.321	.047
C11: Playing with others	−.096	.798	.019	.031	.084
C10: Celebrations	.028	.783	.004	−.026	.000
C14: Spiritual activities	.211	.675	−.142	.022	−.151
C16: Social activities	−.059	.633	.129	−.139	−.027
C8: Caring for animals/pets	.148	−.050	.783	.091	.088
C7: Caring for family	.170	−.025	.739	−.029	.044
C20: Volunteer activities	−.116	.126	.739	.052	−.129
C5: Meal preparation	.159	.075	−.100	.734	.157
C9: Family time	.072	−.024	.035	.677	.130
C2: Family mealtime	.190	−.112	.191	.565	−.154
C13: Quiet leisure	.061	−.092	.028	.506	.351
C3: My own health	.061	.022	−.041	.039	.710
C1: Personal care	−.028	−.005	.098	.110	.681
C18: School	.152	−.061	−.057	.121	.638

Subcomponent 1: Organised activities

This subcomponent includes the following six activity items with all items having a component loading higher than 0.5: ‘overnight visits and trips’, ‘organised leisure’, ‘cleaning at home’, ‘attendance at the health centre’, ‘gathering supplies and shopping’.

Subcomponent 2: Social activities

This subcomponent includes the following four activity items, with all items having a component loading of 0.6 or higher: ‘playing with others’, ‘spiritual activities’, ‘celebrations’ and ‘social activities’.

Subcomponent 3: Taking care of others

This subcomponent includes the following three activity items, with all items having a component loading higher than 0.7: ‘caring for family’, ‘caring for animals or pets’, and ‘volunteer activities’.

Subcomponent 4: Family life activities

This subcomponent includes the following four activity items, with all items having a component loading of over 0.5: ‘family time’, ‘family mealtime’, ‘meal preparation’ and ‘quiet leisure’.

Subcomponent 5: Personal care and development activities

This subcomponent includes the following three activity items: ‘school’, ‘personal care’ and ‘my own health’.

Discussion

Given the limited availability of tools to assess the participation of Chinese children with and without ID in everyday activities, this study examined the construct validity of the subscale ‘attendance’ for the PMP-C (Simplified) in a mainland China setting. The evidence of construct validity was adequate. The PMP-C (Simplified), with 20 items, can fully capture the five-factor structure of children’s participation, as conceptualised in this study.

Validity in research is defined as how well an instrument measures what it sets out to measure [36]. Construct validity is defined as the adherence of an assessment to the existing knowledge and theory of the concept which is being measured [30,37]. Previous study has stressed the important role of construct validity by using scale scoring to reach proper conclusions [30] and it is indeed a crucial concept used in assessing the PMP-C (Simplified)’s accuracy. In a previous study, as an improvement and application of the original scale PMP, the wording of a few items in the PMP-C (Simplified) was changed to make the items more suitable for Chinese [27]. Since one item ‘volunteer activities’ only exists in PMP-C (Simplified) and not in the original PMP-C, an exploratory PCA was chosen in the present study. Based on the results of PCA, the one label of the subcomponent social activities and taking care of others in PMP was changed into two labels in the PMP-C (Simplified): one was called social activities, the other was called taking care of others, while other labels were kept the same as PMP. In other words, it is concluded that the results of the study to a large extent overlap with the item-factor distribution defined in the original PMP scale. In the opinions of the integrated expert panel, the five subcomponents specified represent appropriate clusters of participation for children in mainland China.

The construct validity of the PMP-C (Simplified) was supported. Firstly, the subcomponent organised activities contains six items with all items having a high-order factor as the best fit. This label named organised activities was in accordance with the finding of factor analysis in the original study [25]. These activities of all six items have a clear aim and a distinct beginning and end. Participation in the organised activities which is formal or recreational may be strongly influenced by the environment and culture, e.g. as found in rural and urban areas in mainland China. These activities are often organised by the primary caregiver and children can attend them. In this way, the child is a member of or formed into a temporary group and can be assigned tasks of different complexity, based on the skills and level of functioning required by the activities, eg.ID and TD.

The second subcomponent, social activities, contains four items. These activities are characterised by having groups of people living together or enjoying life in communities or organised groups that as a theme are characterised by people communicating with others and forming closer relationships with them. The items ‘playing with others’ and ‘celebrations’ are the most susceptible to the context- and environment-related constructs of the fPRC model [3].

Thirdly, the subcomponent taking care of others contains three items: caring for family, caring for animals or pets, and volunteer activities, The revised item ‘volunteer activities’ fits very well with the other two activities, as most of the volunteer activities for children are often organised by schools in mainland China aim to strengthen the ideological and moral education of children. For example, activities include interacting in welfare homes to take care of the elderly or other jobs to help others. The children are responsible for or deal with a situation or task and these activities are all about helping and considering others and require a certain self-care ability and communication skills [14].

The fourth subcomponent, family life activities, contains four items. The item ‘meal preparation’ had higher factor loadings on this subcomponent, which is typical for the Chinese family culture, but was not consistent with the original PMP study [25]. Previous study had demonstrated that family circumstances are crucial determinants of children’s participation [14]. All items describe routines activities that occur in home environments and contain social interactions with family members who live together and share common custom and beliefs.

The last subcomponent, personal care and development activities, contains three items, all focused on taking care of your own person (health, personal care and school). All three items have a component loading higher than 0.6, which indicates relatively sound psychometric support for the subcomponent. This finding supports previously reported results on the original PMP’s validity [25]. These activities are all about developing skills for living independently and learning new knowledge. This is a process in which children over time pass to more advanced stages. Self-care ability and health condition are important indicators to judge whether a child can enter school. In China, if they cannot attend school, the teachers can come to their home to teach the child. Thus, this subcomponent also seems to have clinimetric and special education relevance because it may inform clinicians and educators about what activities to target for intervention by discussing the result of the assessment with children and primary care givers.

Strengths and limitations

In this study, considering a reduction in the risk of bias, sampling was adequate and the participants with and without ID were selected from four centres with a representative population of different ethnic groups in rural and urban areas. However, due to the limited sample size, confirmatory factor analysis could not be implemented to confirm the construct validity of the PMP-C (Simplified).

Future directions

The development of PMP-C (Simplified) based on factor scores for variables is useful for developing a profile of a child’s participation pattern to provide insight into the everyday activities [25]. It is a brief, feasible way to gather the child respondents’ own views about their participation in everyday activities and guide intervention priorities for practitioners/clinicians when measuring participation of children and youths [38]. The use of the PMP-C (Simplified) to measure or quantify the level of attendance in different types of activities is a crucial step in promoting the participation of children with ID in rural and urban areas of mainland China. When the PMP-C (Simplified) with distinct subcomponents is combined with other measures of a child’s language or motor performance, it can be used to help families and therapists plan intervention goals or changes in activities for children with and without ID. Psychometric testing of the engagement scale needs to be validated in future research.

Conclusion

In summary, this study provides empirical support for the construct validity of the psychometric properties for the PMP-C (Simplified), making it a useful measure that can be administered to screen the participation of children with and without ID in mainland China and provide good information both for clinical practice and research.

Acknowledgements

The authors gratefully acknowledge all primary caregivers, children and participating schools, for their willingness to participate in this study.

Disclosure statement

The authors report no conflicts of interest.

Additional information

Funding

This research was funded by Tianjin philosophy and Social Science Planning Project [#TJSR17-009].