Assessment in people with PIMD: Pilot study into the usability and content validity of the Inventory of the personal Profile and Support

Abstract

Providing appropriate support for people with profound intellectual and multiple disabilities (PIMD) is challenging, since valid and reliable instruments that can be used for assessment in persons with PIMD are scarce. Therefore, this study analyzes the usability and validity of an instrument developed for persons with PIMD, the Inventory of the personal Profile and Support (IPS). This instrument is part of a person-centered, goal-oriented, interdisciplinary intervention for persons with PIMD, which is called the “support program.” A first step in the support program is to draw up a personal profile, on which a long term goal for the person with PIMD can be based. When the IPS is used in combination with another instrument, the Behavior Assessment Scales (BAS), a support profile can be written. However, the IPS has not been studied on psychometric qualities yet. To look into the usability and content validity of the IPS, two questionnaires that were developed for this study were used for eight direct support persons of three care facilities. Although filling in the IPS is time-consuming and is therefore sometimes considered to be a burden, in general usability and content validity were rated as good. This study underlines the importance of using assessment instruments during different stages of support in a consistent, systematic way, using the input of all involved persons in support of the person with PIMD.

Keywords:

• assessment
• people with profound intellectual and multiple disabilities (PIMD)
• support profile
• developmental assessment instruments

Public Interest Statement

For people with profound intellectual and multiple disabilities (PIMD), sharing information is of crucial importance in order to provide appropriate support. This group consists of people with a profound intellectual disability and a severe or profound motor disability. Often, they have several additional impairments and general health problems. They are dependent on others in all areas of life and experience extensive communication problems. For this group, providing appropriate support is challenging, because reliable and valid assessment instruments that can be used in the support of people with PIMD are scarce. Therefore, the current study focuses on the usability and content validity of an assessment instrument that was developed for people with PIMD, the Inventory of the personal Profile and Support (IPS). In general, direct support persons of people with PIMD-rated usability and content validity of the IPS as good.

Competing Interests

The authors declare no competing interest.

1. Introduction

In order to give direction to support or intervention for people with intellectual disabilities, assessment is of great importance (Hogg & Langa, 2005). Moreover, according to Mansell and Beadle-Brown (2004), assessment should be tailored to the circumstances and individual needs. However, for people with profound intellectual and multiple disabilities (PIMD), this is particularly challenging. The group persons with PIMD consists of people with a profound intellectual disability in combination with a severe or profound motor disability (Nakken & Vlaskamp, 2007). Moreover, they have a number of additional sensory impairments such as visual impairments (Woodhouse, Griffiths, & Gedling, 2000) and general health problems, such as gastroesophageal reflux disease (Bohmer et al., 1999) and epilepsy (Hogg, 1992; Van Timmeren et al., 2016). People with PIMD experience extensive problems in communication. Language is limited or non-existent and communication may include very subtle cues including physiological signals such as a rise in body temperature or change in level of alertness (Vlaskamp, 2005).

Persons with PIMD are dependent on support professionals to be sufficiently knowledgeable about their needs, possibilities, and preferences to be able to provide them with appropriate support (Carnaby, 2007; Forster & Iacono, 2014; Lyons, Walla, & Arthur-Kelly, 2013; Vlaskamp, Hiemstra, & Wiersma, 2007). However, providing appropriate support for persons with PIMD is difficult, since valid and reliable instruments that can be used for the assessment in persons with PIMD are scarce (Carnaby, 2007; Vlaskamp, Van der Meulen, & Zijlstra, 2002). Due to the physical disabilities of people with PIMD, standardized tests that are commonly used such as the Bayley Scales (Bayley, 1967) or the Vineland Adaptive Behavior Scales (Sparrow, Cicchetti, & Balla, 2005) do not lead to a valid estimation of a developmental level because they rely heavily on motor functioning (Carnaby, 2007; Visser, Ruiter, van der Meulen, Ruijssenaars, & Timmerman, 2014).

In order to develop a sound personal profile about the wishes, needs, functional (dis)abilities, and preferences of the individual with PIMD, a multidimensional framework is necessary (Carnaby, 2007; Vlaskamp, 2005). Therefore, several instruments were developed, including the Inventory of the personal Profile and Support (IPS) and the Behavior Assessment Scales (BAS) (Vlaskamp, Van Wijck, & Poppes, 2015). Using the IPS, information can be gathered about the person with PIMD and about the person in relation to persons that are important for him or her. Furthermore, the BAS can be used to measure the functional abilities of a person with PIMD in five developmental domains (Vlaskamp, Van der Meulen, & Smrkovsky, 1999). The instrument can be used in practice to make clear in what developmental domains the person is functioning better in comparison to other developmental domains. This way, strengths and weaknesses of the person with PIMD can be described (Vlaskamp et al., 1999). This approach is in accordance to a model such as the ICF (World Health Organization, 2001), which has a focus on establishing profiles of strengths and weaknesses as well (Buntinx & Schalock, 2010). The BAS and IPS supplement each other by collecting different information from different sources. This corresponds to the approach as described in Lyons, De Bortoli, and Arthur-Kelly (2016), who use the triangulated proxy reporting of data: a technique to improve communication between people with PIMD and their support persons. The IPS and BAS can be used in combination to collect as much information about the person with PIMD as possible, to be able to write a support profile, which contains all available information about the wishes, needs, preferences, and functional abilities of a person with PIMD. The BAS and IPS are both part of a person-centered, goal-oriented, interdisciplinary program for persons with PIMD, called the “support programme,” which was evaluated as effective by the admission committee of support of persons with a disability (Vlaskamp, Poppes, & Van der Putten, 2015; Vlaskamp & van der Putten, 2009). Vlaskamp and van der Putten (2009) found that implementing the program resulted in more individualized support that is tuned to the wishes and needs of the individual and an increased knowledge of staff about the individual with PIMD.

The BAS has adequate psychometric properties (Vlaskamp et al., 2002). However, the IPS has not been studied on psychometric qualities yet. Moreover, assessment with the IPS should be done by interviewing health care professionals and parents, which is time consuming and partly inapplicable in practice. As a result, health care professionals often fill in the IPS without an interview which could have consequences for the accuracy of the information gained. Therefore, a new version of the IPS was developed, which has a different way of gathering information and can be filled in without an interview (Vlaskamp, Van Wijck, et al., 2015).

This version of the IPS has not been studied for psychometric qualities or usability yet. Therefore, this study will focus on the usability and content validity of the IPS in combination with the BAS, by looking at the support profile based on the combination of the instruments. This is important, because without reliable and valid instruments, offering people with PIMD appropriate support is challenging. When the content validity of the support profiles based on the BAS and IPS is high, it is possible to better adjust the support of direct support professionals to the needs and possibilities of persons with PIMD, which could lead to knowledge and strategies that increase their quality of life (Lyons et al., 2016; Maes, Lambrechts, Hostyn, & Petry, 2007).

2. Method

2.1. Participants

A convenience sample was used consisting of eight persons with PIMD and their eight direct support persons from three residential care facilities. Moreover, two health care psychologists of two of the persons with PIMD were involved as well. Inclusion criteria for the persons with PIMD were (Nakken & Vlaskamp, 2007):

•	An estimated developmental age of 24 months or lower.
•	Severe or profound motor impairments.
•	Written informed consent from family or legal representatives to participate in the current study.

For the persons with PIMD, exclusion criteria were:

•	Having a severe progressive illness or disorder.
•	Having recently experienced a fundamental change in their environment (such as a movement to another care institution or surgery).

In Table 1, the demographics and additional impairments of the participants can be found.

Table 1. Additional impairments of the participants

Person	Residential home	Gender	Age	Impairment
				Visual	Auditory	Epilepsy	Eating and drinking problems	Sleeping problems	Contractures
1	1	Male	26	Yes	No	Yes	Yes	Yes	No
2	1	Male	22	Yes	No	Yes	Yes	Yes	Yes
3	1	Male	66	Yes	Yes	No	No	No	Yes
4	1	Male	68	Yes	Yes	No	Yes	No	Yes
5	1	Female	45	Yes	No	Yes	Yes	Yes	Yes
6	2	Female	63	Yes	Yes	Yes	Yes	No	Yes
7	2	Male	Missing	No	No	Yes	Yes	Yes	Yes
8	3	Female	33	Yes	No	Yes	Yes	Yes	Yes

For informed consent, all legal guardians of the involved participants received an information letter, which contained information about:

-	the BAS, IPS, and the aim of the study;
-	processing all personal information anonymously;
-	how information about the participant from the assessment instruments would be used by the researcher for the aim of the study, but that it would also be available to the direct support persons and the legal guardian;
-	the training the researcher who would do the assessment had;
-	how participation was completely voluntary and could be stopped at any time;
-	contact information of the researcher.

Ethical approval for this study was granted by the Ethics Committee of Pedagogical and Educational Sciences of the University of Groningen. Moreover, two of the involved residential care facilities discussed the study in an internal meeting and approved the study during that meeting. The other involved residential care facility had an ethical board who approved the study.

All participants with PIMD experienced severe problems in communication. Language was limited or non-existent and communication included reflex responses, sounds, facial expressions, and bodily movements (Vlaskamp, 2005). Of the eight direct support persons and the two health care psychologists who filled in the questionnaires about the IPS, there were eight women and three men. They were all involved as direct support persons in the living facility and all working with the person with PIMD for at least a year.

2.2. Instruments

2.2.1. IPS

The IPS focuses on the support that the person with PIMD receives and the context of support. It measures the possibilities of the person with PIMD and in relation to important persons for the person with PIMD (Vlaskamp & Van Wijck, 1997). A new version of this instrument was developed which can be filled in by a person who is involved in the support of the person with PIMD (Vlaskamp, Van Wijck, et al., 2015).

There are different versions for the different disciplines and persons involved in the support of a person with PIMD, including a version for parents, direct support person, health care psychologist, occupational therapist, speech therapist, physical therapist, and physician. The content consists of three parts: the development history of the person, characteristics of the person, and future goals for the person. In the development, the focus is on motor, cognitive and communicative skills, and independency in daily life of the person. Moreover, the test and observation results and diagnoses are included in this part. The characteristics of the person concerns questions about emotions, needs, relations with others, basic mood, and changes in the mood of a person. The future goals part considers questions about the needs, wishes, and goals that the involved person has for the person with PIMD (Vlaskamp, Van Wijck, et al., 2015).

2.2.2. BAS

The BAS measures the functional abilities of a person with PIMD on five different domains: emotional and communicative behavior (3 items), receptive language skills (9 items), general communicative behavior (15 items), visual behavior (36 items), and explorative behavior (37 items) (Vlaskamp et al., 1999). Each domain consists of dichotomous items: the answers consist of a yes/no construction, thus indicating that the higher the total score per domain, the more often the person scores a yes on that domain (Vlaskamp et al., 2002). Part of the BAS, 28 items, is based on knowledge of a proxy of the person with PIMD and is scored by an interview of that proxy. These items are, for example, about whether the person with PIMD recognizes other familiar persons and how this can be seen. The other items are scored by observation of the person with PIMD. Using different objects and materials it is tried to provoke behavior of the person. For instance, these items are about reaction to auditory and visual stimuli. The BAS is designed to give the person with PIMD maximal opportunities to show his or her functional possibilities. It has no age boundaries or cut-off scores and contains small steps between different items, making it possible to detect small differences in development. There is no time limit for the person with PIMD to show the behavior, nor a strict order of the items (Vlaskamp et al., 1999). Raw scores are converted to quartile scores on the five domains which results in a personal profile about the higher and lower possibilities of a person.

The BAS was evaluated as being practical useful by health care psychologists and support staff working with persons with PIMD (Visser, Oldenburger, & Van der Meulen, 2014). Factor analysis showed that the internal consistency of the BAS was very high (Vlaskamp et al., 1999). Moreover, the inter-observer reliability was good and the instrument was evaluated as reliable. Content validity was evaluated as good (Vlaskamp et al., 2002).

2.3. Procedure

Various facilities were asked by phone and e-mail to participate in the study. Health care psychologists were asked to select participants and their direct support persons.

2.4. Analysis

2.4.1. Usability of IPS

All eight direct support persons were asked to fill in the IPS and they were asked about the usability of the IPS. The health care psychologists were not involved in this part due to time constraints. For measuring the usability of the IPS, a questionnaire was constructed. Usability was defined by four quality components: time to fill in the IPS, time filling in the IPS compared to the previous version, comprehensibility of the IPS and information covered by the questions of the IPS. The questions about the usability of the IPS were:

-	How much time did it take to fill in the IPS?
-	Did you work with the previous version of the IPS? If yes, did filling in the new version take more or less time than the previous version?
-	How many questions or concepts were difficult to understand? Which ones were difficult to understand?
-	Is there any information about the person with PIMD that is not included by filling in the IPS? If yes, what information did you miss?

Personal information of the participants with PIMD and their direct support persons was coded to guarantee anonymity.

Descriptives concerning gender, age, and additional impairments of the participants were computed. The analysis was descriptive, by describing for every direct support person if he or she worked with the previous version of the IPS, how much time filling in the IPS was indicated by the direct support persons and whether there was enough space for filling in the questions. The analysis was qualitative, using descriptive qualitative analysis (Sandelowski, 2000) to analyze what items where difficult to understand and why and to analyze extra comments that were made.

2.4.2. Content validity IPS

For the eight participants, the BAS was also filled in, to analyze content validity according to Tadema, Vlaskamp, and Ruijssenaars (2007). Part of the BAS was filled in by observation of the person with PIMD and part of the BAS was filled in by interviewing the direct support persons. All eight direct support persons filled in the IPS. Moreover, in order to write a support profile, for some persons with PIMD, parents, physicians, physiotherapists, and health care psychologists were also asked to fill in the IPS. Based on the information of both the IPS and BAS, a support profile was written. A second questionnaire was used consisting of five questions about the support profile. The questions that were asked were:

-	To what degree is the description of the person in this support profile consistent with your own impression of who this person is?
-	What impression does the support profile give of the functional abilities of this person with PIMD?
-	What impression does the support profile give you of the wishes and needs of this person?
-	What impression does the support profile give of the difficulties in the support of this person?
-	Does the support profile contain enough information to formulate goals in the support of the person?

The questions were scored on a five point Likert Scale. The first question ranged from very inconsistent to very consistent (1 = very inconsistent, 2 = inconsistent, 3 = neither inconsistent nor consistent, 4 = consistent, 5 = very consistent). Question two till four ranged from very bad impression to very good impression (1 = very bad impression, 2 = bad impression, 3 = neither good nor bad impression, 4 = good impression and 5 = very good impression), and the last question ranged from “very bad possible” to “very good possible” (1 = very bad possible, 2 = bad possible, 3 = neither good nor bad possible, 4 = good possible, 5 = very good possible). After every question, a question for clarification was asked, i.e. “Can you explain why you think this is a good/bad impression of the functional abilities?”

Using a Likert Scale, direct support persons could score the quality of the support profile and using the open questions it was possible to further explain why they would give a specific score. The analysis was partly quantitative, by computing mean score, standard deviation, minimum and maximum score on the five-point Likert Scale for the five questions computed. Moreover, the answers on the open questions were analyzed using descriptive qualitative analysis (Sandelowski, 2000). Due to the sudden death of one of the participants and time constraints of two other direct support persons, the questionnaire about the support profile was filled in by five persons. Four were the direct support persons who filled in the IPS and one was the involved health care psychologist of one of the persons with PIMD. Moreover, the other involved health care psychologist gave feedback on the support profile, together with the direct support person, by filling in one content validity questionnaire together. All questionnaires and instruments that were used in this study were in Dutch.

3. Results

3.1. Usability of the IPS

Of the eight direct support persons, five filled in the time spent on filling in the IPS and three values were missing. According to five direct support persons, filling in the IPS took 30 min–120 min (M = 78, sd = 32.52). Six of the eight direct support persons had already worked with the previous version of the IPS, and two did not. Of these six direct support persons, one said that filling in the new version took more time than the previous one, two said that it took about the same time, one direct support person said it took less time and one value was missing. One direct support persons mentioned concerns regarding the list for parents, indicating that not all parents may understand all the questions.

For part one of the list, the development part, all questions were understood, but two direct support persons indicated that some questions could better be filled in by another expert, such as a physician and a physical therapist for motor development. One direct support person indicated that there was not enough space for answering the questions, due to problems with the digital form of the IPS.

For part 2 of the IPS, the personal characteristics, all the questions were understood by the participants. Again, one direct support person indicated that there was not enough space for answering the questions, due to problems in the digital version. Moreover, one direct support person suggested that it would be better to fill in the questions about personal characteristics by two persons instead of one.

For the last part of the list, about the future goals for the person with PIMD, all the questions were understood and there was enough space to fill in the questionnaires. All the direct support persons (N = 8) felt that they were the right person and had the necessary knowledge to answer the questions.

3.2. Content validity of the IPS

In Table 2, the mean scores and standard deviation on the five-point Likert Scale of the questionnaire of the support profile can be found. In this table it is shown that the mean score on the five-point Likert Scale is high in general and the range is small.

Table 2. Mean score, standard deviation, minimum, and maximum score on the support profile questionnaire

Question	Mean	Standard deviation	Minimum	Maximum
1	4.20	0.45	4	5
2	3.90	0.60	2	4
3	3.60	0.55	3	4
4	4	0	4	4
5	4	0	4	4

Notes: Question 1 = To what degree is the description of the person in this support profile consistent with your own impression of who this person is?; Question 2 = What impression does the support profile give of the functional abilities of the person with PIMD?; Question 3 = What impression does the support profile give you of the wishes and needs of this person?; Question 4 = What impression does the support profile give you of the difficulties in the support of this person?; Question 5 = Does the support profile contain enough information to formulate goals in the support of the person?

In general, all direct support persons indicated that they were content with the support profiles and said that the support profiles were well written. One of the involved health care psychologists noted that it was remarkable that a support profile that is solely based on a IPS and a BAS and was written by someone who barely knew the person with PIMD, could describe the characteristics, needs, and wishes of a person with PIMD so accurately. On question one to four, there were, however, some comments made.

At the first question, one person indicated that information regarding what the person likes and what goals were reached in the past year was missing in the support profile. However, this was included in the answers on the IPS. Moreover, information about the motor activation, the way the person structures his or her world and what support the person needs was missing according to one person who filled in the questionnaire. Whereas the question of what support a person needs is included in the IPS, specific questions about motor activation and the way a person structures his or her world are not. The direct support person who filled in the IPS did not indicate it at other questions. Moreover, one direct support person mentioned that there was no information about the diet the person with PIMD had because of diabetics and this information was also not part of the questions in the IPS.

At the question about functional possibilities, one direct support person mentioned that there was too little information about the auditory impairments and the motor impairments of the person with PIMD. Also, the health care psychologist of one of the persons indicated that the information that was mentioned regarding functional disabilities, was limited.

At the third question, one direct support person noted that determining the wishes and needs was difficult for all involved persons, so answering these questions was difficult in the IPS and thus difficult to find in the support profile. Another direct support person mentioned there was not enough information about the need the person with PIMD had to always have a familiar support person close to him. A health care psychologist mentioned that it was important to fill in this part with several involved persons instead of one.

At the question about difficulties in support of a person with PIMD, one direct support person mentioned that there was not enough information about the alertness of the person with PIMD. One direct support person said that there was behavior of the person with PIMD mentioned in the support profile that the person rarely showed and which did not have a large influence in daily life anymore.

4. Conclusion

This study focused on the usability and content validity of the IPS. In general, usability and content validity were good according to the majority of the direct support professionals. Concerning usability, all questions were clear and understood and there was enough space to fill it in. This was according to expectations, since the new version of the IPS was developed to be filled in by the involved persons without an interview with a health care psychologist (Vlaskamp, Van Wijck, et al., 2015). Scores on the questionnaire about the support profile, that were an indication for content validity, ranged from 3.60–4.20 (on a scale of 1–5), indicating a range from a neither good nor bad impression of the support profile to a (very) good impression of the support profile. In all, the majority of the scores indicated a good impression of the support profile.

5. Discussion

Although in general usability and content validity of the instrument were rated as good, some comments should be made. First of all, a concern of one direct support person was about the usability of the IPS for parents; a direct support person indicated that probably some parents may experience the list as a burden and may have difficulty with understanding the terms and questions. Since parents can be considered to be experts in the care of persons with PIMD and are essential in formulating their wishes and needs (Jansen, Van der Putten, & Vlaskamp, 2013), it is of great importance that they are able to fill in the questionnaire as well.

By filling in the instruments, all the available information about the person with PIMD is gathered and support can be specifically adapted to the highly complex support needs of the person with PIMD (Vlaskamp & van der Putten, 2009). This can increase quality of support, which is strongly related to the quality of life of people with PIMD (Lyons et al., 2016). According to the majority of the participants, the support profile was consistent with their own impression of who the person is. It gave enough information about the difficulties in the support of the person with PIMD and it contained enough information to formulate goals in the support of the person with PIMD.

Some methodological limitations of this study should be taken into account. The qualitative study of the IPS concerned a small sample, and one usability questionnaire was not filled in, but feedback was sent by the direct support professional without using the format. One of the questions about usability concerned the time it takes to fill in the IPS, which is not an entirely objective indicator of usability as some direct support persons are willing to invest more time in assessment than others. However, the question still yields valuable information, since there is a limit as to how much time direct support persons can invest in filling in instruments. Moreover, the time-related question of the IPS is especially interesting in combination with the questionnaire about content validity: investing time in filling in the IPS will be valued more if filling in the instrument yields the information that is needed in support of the person with PIMD. Furthermore, support profiles were written by a researcher and not someone who knew the person with PIMD well. Moreover, for one of the persons with PIMD, the BAS was filled in by someone else than the researcher, meaning that the researcher who wrote the support profile, never saw the person with PIMD, which may have caused a lower rate of the profile because there was no other information about the person with PIMD available but the information of the IPS and BAS. This was also the person who rated the support profile lowest. In addition, due to time constraints, for seven of the eight profiles, only the BAS and the version of the IPS filled in by the direct support person was used to write a support profile and not the versions of other involved persons. Precisely this is of great importance for writing a support profile. A support profile is more than just a summary about whom the person with PIMD is, it is a complete profile containing all the information of all the involved persons and discrepancies between different persons are of special interest (Vlaskamp, Poppes, & Zijlstra, 2005). This could have influenced the results, since, for example, the health care psychologist who filled in the questionnaire about the support profile did not fill in the IPS. If this version was integrated, this opinion may also have been better reflected and the profile would contain more information. Therefore, the comments that were made about the content validity of the IPS could have been explained by these limitations. In conclusion, in further studies regarding the IPS a recommendation is basing the support profiles on the IPS filled in by different persons who know the person with PIMD well. Moreover, it is important to look at the usability of the list for parents. Recommendations regarding practice concern the importance of using good assessment instruments during different stages of support in a consistent, systematic way, using the input of all involved persons in support of the person with PIMD. Assessment is crucial to learn about the wishes, needs, and preferences of persons with PIMD in order to deliver individualized support (Lyons et al., 2013).

Funding

The authors received no direct funding for this research.

Additional information

Notes on contributors

Marleen D. Wessels

The research group is part of the centre for special needs education and youth care, which has a main research focus on assessment and interventions for vulnerable groups of adults and children, including the development of assessment instruments and improving support of people with profound intellectual and multiple disabilities (PIMD). The focus of the current article is on assessment procedures in the support of persons with PIMD, aiming to tune the support to the preferences and needs of people with PIMD.