Validation of the Inventory of Professional Functioning (IPF): Occupational Health of Social Workers

Abstract

This article reports on the validation of the Inventory of Professional Functioning (IPF), a four-dimensional scale designed to measure the professional functioning of social workers. The IPF was developed as an outcome measure to be used with independent measures of substance use, depression, distress, relationship problems, and organizational variables, as well as protective factors such as social support, self-esteem, and coping strategies. Data from a random sample of 215 social workers were collected in an anonymous Internet survey that measured factors related to social workers' health. Using exploratory and confirmatory factor analysis, the new measure was found to have acceptable reliability and construct validity at the scale level. Convergent and discriminant validity were established at the scale level of analysis by examining correlations of the items of the IPF and its subscales with Class I, II, and III Criterion variables. The IPF is a useful measure to assess employee strengths in the workplace.

KEYWORDS:

• impairment
• organizations
• professional functioning
• scale development
• social workers

Working in challenging organizational environments with vulnerable and underserved populations places some social workers at risk to develop problems that may affect their personal and professional lives (National Association of Social Workers [NASW], 2006). Social workers who develop problems that interfere with their ability to work with clients have impaired practice, and measuring impaired practice has been a challenging task to this point. Social workers are valuable human resources who need tangible support when they have problems that affect their practice, and identifying social workers who need increased support is important for the profession. The purpose of this article is to report on the validation of the Inventory of Professional Functioning (IPF), a scale that was developed to measure professional functioning among social workers and other helping professionals.

Many definitions of impairment exist in the literature, with no consensus on one (Sherman, 1996). This ambiguity is evidenced by the plethora of definitions offered by several state NASW chapters that have attempted to define the problem (see Negreen, as cited in Siebert, 2001, p. 4). Impairment is sometimes addressed in terms of its causes (Sherman & Thelen, 1998) and sometimes in terms of the problems associated with antecedent conditions (Lamb et al., 1987; Reamer, 1992).

Some scholars address the etiologies of impairment by noting the prevalence rates of mental health problems and substance abuse problems among social workers and other mental health professionals (Deutch, 1985; Siebert, 2003). Others have theorized that family-of-origin problems and dysfunctional patterns originating in families create a vulnerability to impairment (DiCaccavo, 2002; Lackie, 1983; O'Connor, 2001; Vincent, 1996). Problems associated with impairment include depression (Siebert, 2004), substance misuse (Sherman, 1996; Siebert, 2003), difficulty managing distress (Strozier & Evans, 1998), extreme working conditions (Schaefer & Moos, 1996), family and relationship problems (Reamer, 1992), and prior trauma or abuse (Pooler, Siebert, Faul, & Huber, 2008). Some studies cite interference in functioning and diminished work performance as indicators of conditions mentioned above (Sherman, 1996; Sherman & Thelen, 1998).

The psychology literature uses the term distressed in conjunction with impairment (Good, Thoreson, & Shaughnessy, 1995; Kilburg, 1986; Kilburg, Nathan, & Thoreson, 1986; McCrady, 1989; Sherman, 1996; Sherman & Thelen, 1998; Thoreson, Miller, & Krauskopf, 1989; Thoreson & Skorina, 1986). The medical field suggests that “any physical, emotional or educational deficiency that interferes with the quality of … professional performance, education or family life” is evidence of impaired practice (Boxley, Drew, & Rangel, 1986, p. 50).

Tinius (1988) defined impairment as “a marked inability to perform competently and to take effective action while in a professional role because of chemical dependency, mental illness, or personal conflicts” (p. 25). Although dated, Lamb et al. (1987) offered the most comprehensive definition of impairment:

An interference in professional functioning that is reflected in one or more of the following ways: (a) an inability and/or unwillingness to acquire and integrate professional standards into one's repertoire of professional behavior, (b) an inability to acquire professional skills in order to reach an acceptable level of competency, and (c) an inability to control personal stress, psychological dysfunction, and/or excessive emotional reactions that interfere with professional functioning. (p. 598)

Bissell (1983) made a clear distinction between people whose practice is impaired versus those who are simply incompetent or unethical that offers some valuable clarification. Those who are incompetent are poorly trained and/or unable to help clients. Good gatekeeping and quality control help remedy incompetence. The unethical clinician is one who is dishonest or uncaring about the welfare of clients, and should be disciplined. The impaired practitioner is one who is ill or sick, but not malicious. Most people with impaired practice could be expected to recover and improve with help. Bissell suggested that people whose practice is impaired should not be punished.

For impairment to be present there must be a relationship with one or more antecedent conditions, as impairment is akin to a treatable illness with a diagnosable cause. Even though incompetence and unethical behaviors may not always be linked with impairment, the literature suggests that social workers with impaired practice are at a higher risk for incompetent and unethical practice than those whose practice is not impaired (Berliner, 1989; NASW, 2006; Reamer, 1992).

Social work has addressed the issue of impairment only very slowly and with apparent reservation. Reamer (1992) encouraged the profession to give increased attention to the issue of impairment, but there has not been any widespread response. More recently, Siebert has addressed important factors associated with the health and well-being of the social work workforce in several published papers—looking at depression (2004), substance misuse (2003), barriers to help seeking (2005a), the prominence of a caregiving identity (Siebert & Siebert, 2005), factors related to burnout (2005b), and vulnerability to emotional contagion (Siebert, Siebert, & Taylor-McLaughlin, 2007). Siebert and Siebert (2007) applied role identity theory as a framework for understanding why professionals develop problems that interfere with their functioning. When professionals primarily identify and see themselves as helpers or caregivers, it is not congruent with that role to have problems similar to their clients. When problems do arise in the professionals' lives, they are easily overlooked or minimized in an attempt to enhance their identity as a helper/caregiver. The social worker may minimize depression, substance misuse, and relationship problems and may have difficulty identifying and asking for help with their problems.

Social work has had a public policy on impairment since 1987, but it was not until 1994 that the issue was included in two sections of the NASW Code of Ethics—impairment of self and impairment of colleagues (NASW, 1999). In the 1996 revision of the Code of Ethics these two sections were expanded. In the current Code, revised in 1999, Section 4.05 clearly describes that social workers must seek appropriate professional help when they have problems that affect their practice to ensure that clients are not harmed. Section 2.09 states that social workers who have direct knowledge of a colleague's impairment must consult with them and assist them with obtaining appropriate help and support.

To operationalize impairment, a definition that was congruent with the extant literature and the NASW Code of Ethics was developed. Impairment is evidenced by deficits in one or more of the following areas performance: (1) professional judgment (ability to make decisions by incorporating professional standards and values into professional conduct), (2) competence (providing quality interventions/services), (3) ethics (not harming clients), and (4) reliability (trustworthiness and collegiality). These deficits are caused by an inability or unwillingness to address the effects of abuse or trauma (past or current events), family and relationship problems, distress, mental health problems, or substance abuse problems (Lamb et al., 1987; Reamer, 1992; Sherman & Thelen, 1998; Siebert, 2003, 2004).

PURPOSE

In the study of impairment among professionals there is a lack of a standardized measuring tool or scale. Negative consequences associated with impairment have been measured using an index comprising personal and professional impairment variables (Pooler et al., 2008; Siebert, 2001). Siebert first measured the negative consequences associated with mental health and substance abuse problems among social workers in methodologically sound way, and her work laid the foundation for further research.

A review of the literature suggests the primary way that researchers attempt to analyze problems among helping professionals is by measuring distress, burnout, depression, or substance misuse. A weakness is that studies have not adequately measured the impact of these problems on work with clients and relationships with colleagues. Therefore the development of a reliable and valid self-report measuring device that can capture the impact of deficits in practice is a logical next step in this research. A professional functioning measure is needed so that it is possible to examine relationships with risk factors and protective factors, and other variables of interest.

METHOD

The scale was developed by defining and specifying each construct to be measured (Faul & Van Zyl, 2004), and the definition of impairment mentioned earlier was used to create operational definitions. Four dimensions of professional functioning are operationalized here.

1. Judgment: Disruption in one's ability to make sound professional judgments about clients' welfare, or making decisions that do not contribute to a client's recovery, healing, or growth process. Evidence of this would be making decisions about client care that are not based on current research, or are ill informed, or have not been subjected to critical reasoning skills. It could also include a flippant, superficial, or judgmental stance toward clients. Clinicians may be preoccupied with their own issues and fail to give reasonable thought to the needs and care of clients.

2. Competence: Not staying abreast of current research, lacking appropriate skill in assessment and treatment of clients. This could include failing to refer a client to a needed service, failing to use best practices with a particular client, or spending time with clients that does not lead to solving the problems of the clients.

3. Ethics: Breaking confidentiality when not appropriate, being sexual with a client, engaging in a dual relationship, or failing to maintain appropriate professional boundaries with a client.

4. Reliability: Performing at a level less than one's best, as evidenced by missing work, being late for work, being late for appointments, poor documentation, and contributing less than expected in the work environment.

Item Creation

The operational definitions listed in the preceding section were developed after critically reviewing the impairment literature, the NASW Code of Ethics, and an existing impairment index (Siebert, 2001). The individual items were created by using these definitions and consulting experts in occupational social work. The domain sampling model of measurement (Faul & Van Zyl, 2004) suggests an infinite number of possible items can measure a construct. The skill of the scale developer is to choose those items that will best measure the domain and lead to high levels of content validity. Statements rather than questions were used in the item creation process.

The list method (Hudson, 1991) was used to generate the items. This method involves (1) writing down one attribute (behavior) associated with professional functioning and (2) writing down an item based on that attribute. This was continued until a small pool of items, about 15, was created for each of the four dimensions (Nurius & Hudson, 1993; Springer, Abell, & Nugent, 2002). Reliability increases with scale length, but there are diminishing returns on reliability when the scale moves from 11 to 20 items, and the returns are even smaller when the scale moves from 21 to 30 items (Nunnally & Bernstein, 1994). Between 13 and 15 items were developed for each dimension, and items were positively and negatively worded. After the items were developed they were scaled on 5-point Likert scale. The anchors chosen for this scale were from never to always, to capture the frequency of the behaviors being measured.

Sample

The IPF and 10 other measures examining risk and protective factors of social worker well-being were a part of a research package administered to a random sample of licensed social workers in Kentucky. The study used a cross sectional contextual Internet based survey design. The random sample was drawn from a total of 3,789 licensed social workers in the database of the Kentucky Board of Social Work. Social workers who did not have Kentucky mailing addresses were excluded from the sampling frame, reducing the sampling frame 11% from 3,789 to 3,366 licensed social workers.

SPSS 11.0 was used to choose a random sample of 1,250 social workers. The licensed social workers in Kentucky were chosen for two reasons: (1) licensed social workers work in diverse settings all over the State representing urban and rural areas, and (2) the Kentucky Board of Social Work was open to and supportive of the study. As an incentive all respondents were offered a $20 discount on one continuing education course. The study received Institutional Review Board approval.

Three waves of postcard invitations were mailed to the sample inviting them to complete the online research package. The postcards were mailed between March 7 and April 30 of 2005. The postcards contained information about the study, the incentive, and the URL of the survey. Respondents were asked to use the Internet to complete the survey. About 70 social workers responded to each invitation wave, yielding a response rate of 17% (N = 215). The data were collected online using a secure server and PHPSurveyor and analyzed with SPSS 14.0. With the exception of demographic questions, all the measures in the package formed a response set to aid in quicker completion and easier evaluation of data.

Reliability and Validity

The scale completed by respondents contained 55 items to measure four different domains of professional functioning: (1) Judgment, 14 items; (2) Competence, 15 items; (3) Ethics, 13 items; and (4) Reliability, 13 items. Analyses of item total correlations and Cronbach's alpha were used to identify poor items and test psychometric properties of the scale. Exploratory factor analysis was used to examine hypothesized factors, establish content and construct validity at the scale level, and identify poor items. Factorial validity of the final scale with 17 items was confirmed using structural equation modeling.

Convergent and discriminant construct validity at the scale level of analysis were established by developing and testing three a priori hypotheses (Faul & Van Zyl, 2004; Hudson, 1991): (1) the IPF should correlate the lowest with social background variables (Class I Criterion variables) which provides an indication of discriminant construct validity at the scale level and (2) the IPF should correlate moderately with variables the literature suggests should be related (substance use, depression, distress, etc.). These are Class II Criterion variables and are evidence of convergent construct validity at the scale level; (3) there will be variables that will have the highest correlations with the new scale (other dimensions of the professional functioning scale). These are Class III Criterion variables and further support convergent construct validity.

The items of the subscales were analyzed and screened by creating a correlation matrix. Faul and Van Zyl (2004) suggested removing all items with item total correlations less than .45. The mean of the corrected item-total correlations were treated as a coefficient of content validity, which should be higher than .50. Items were removed from the subscales if a gain in reliability (Cronbach's alpha) was obtained by removing the item and it made sense to remove the item.

The Ethics subscale was discarded in its entirety. There was almost no variance on the responses of this subscale because people responded never to almost every question. In hindsight it was clear that this was a poor subscale simply because many of the items asked about issues that have legal ramifications. These items asked about sexual contact and sexual attraction that are intrusive and uncomfortable for the respondent. All negatively worded items were removed because reliability increased when they were removed. A total of 38 items including the Ethics subscale with 13 items were removed from the IPF. Seventeen items remained with three hypothesized dimensions (Judgment, 6 items; Competence, 8 items; Reliability 3 items).

A principal components factor analysis using promax rotation was conducted on these 17 items. The Kaiser-Meyer-Olkin measure of sampling adequacy was excellent at .899. Barlett's Test of Sphericity was 1,454, significant at p < .001 with df = 136. Instead of finding three hypothesized dimensions or factors, there were four factors. Some items were double loaded on the Judgment and Competence subscales, and three items from the Competence subscale loaded higher on the Judgment subscale—highlighting that judgment and competence are closely associated. In addition, two items on the Judgment subscale loaded on their own factor. This new factor was named commitment because the items dealt with commitment to clients more so than professional judgment.

Principal axis factoring with varimax rotation was used to examine the items and factors again. The items loaded on the same four factors but with clearer item loadings on each factor (e.g., items that were not a part of the factor had lower loadings) which was the best model. Cronbach's alpha was used to examine the reliability of the new subscales after labeling the new factor and moving the items noted above (Judgment α = .88, Competence α = .81, Reliability α = .74, and Commitment α = .84). The IPF was very reliable (α = .92). The coefficient of content validity was .61, above the .50 criteria (see Table 1).

TABLE 1 Subscale Analysis of the Inventory of Professional Functioning (α = .92)

Item		Loading
Judgment (α = .88)
1	I know what I am doing with clients.	.78
2	I make good decisions about clients.	.79
3	I use good judgment with my clients.	.75
4	I feel good about decisions I make about clients.	.43
5	I use the best practices with clients.	.44
6	I know how to help my clients.	.56
7	I do accurate assessments on my clients.	.54
Competence (α = .81)
8	Clients know I am skilled.	.48
9	My clients are making progress.	.43
10	Clients think I do a good job.	.64
11	The time I spend with clients helps them get better.	.67
12	My work with clients is excellent.	.52
Reliability (α = .74)
13	My colleagues trust me.	.62
14	I work well with my colleagues.	.82
15	I return phone calls.	.49
Commitment (α = .84)
16	My clients are important.	.82
17	I value my clients.	.78

Factorial validity was determined by entering the 17-item scale with four factors (Judgment 7 items, Competence 5 items, Reliability 3 items, Commitment 2 items) into M Plus version 5.1. Any cases with more than nine missing responses on the IPF were discarded adjusting the size to 193. To adjust for nonnormal variables the maximum likelihood robust (MLR) estimator in M Plus was used (Enders, 2001). The four latent factors were allowed to correlate. The initial model had a chi-square value of 218, df = 113, p < .001. The root mean square error of approximation (RMSEA) = .069 and standardized root mean square residual (SRMR) = .058. Other fit indices for this initial model, the Comparative Fit Index (CFI = .913) and Tucker-Lewis Index (TLI = .896), indicated a poor model fit to the data.

The M Plus output suggested improvement in the model fit if error terms of several observed variables were allowed to correlate. Items 1 (“I know what I am doing with clients”) and 4 (“I feel good about decisions I make about clients”) were allowed to correlate. However, after controlling for the factor the error terms were negatively correlated. A possible explanation is that when social workers are effective with clients, they may have a healthy skepticism about their decisions. In other words their decisions are more likely to be subjected to critical analysis. The error terms of 2 (“I make good decisions about clients”) and 3 (“I use good judgment with my clients”) were allowed to correlate as well as error terms of 2 (“I make good decisions about clients”) and 5 (“I use the best practices with clients”). Error terms of 2 and 5 were negatively correlated. People who try to use best practices may be more likely to analyze their decisions, which may explain the negative correlation. The error terms of 10 (“clients think I do a good job”) and 11 (“The time I spend with clients helps them get better”) and 10 (“Clients think I do a good job”) and 16 (“My clients are important”) were allowed to correlate. Even though Item 16 was a part of the Commitment subscale, the conceptual linkage between 10 and 16 are obvious at face value. After the above iterations, the final model (see Figure 1) had a chi-square value of 153, df = 108, p = .003. Fit indices (Hu & Bentler, 1999) showed a good model with room for additional improvement (CFI = .963, TLI = .953, RMSEA = .046, SRMR = .05).

FIGURE 1 Confirmatory factor analysis of the Inventory of Professional Functioning (IPF) (N = 193).

Discriminant and convergent validity at the scale level of analysis were established by looking at correlations of the IPF with Class I, II, and III Criterion variables. The IPF had low correlations with basic social background variables such as gender, number of children, and household size (Class I Criterion variables). The low correlation shows discriminant construct validity at the scale level. The mean of correlations of the IPF with Class I Criterion variables is displayed in Table 2.

TABLE 2 Correlations with Class I Criterion Variables (M = .12)

	Gender	Number of children	Number in household
Inventory of Professional Functioning	−.16	−.13	−.09

The IPF had moderate correlations with constructs thought to have a relationship with professional functioning: (i.e., substance abuse, depression, distress, social support). The substance abuse, depression, and social support measures are subscales of the Corporate Behavioral Wellness Screening Inventory (Faul, 2002), a tool used to measure strengths of workers in occupational settings. These subscales have been validated with samples of workers in South Africa and a sample of social workers in the United States (Faul, 2002; Pooler, 2008). Distress was measured using the K6 Symptom scale. This scale was validated in the United States with the general population and has been used annually in the National Survey on Drug Use and Health (NSDUH) administered by the Substance Abuse and Mental Health Services Administration (2007). Moderate correlation of the IPF with Class II Criterion shows evidence of beginning convergent construct validity at the scale level. The mean correlation of the IPF with Class II Criterion variables is seen in Table 3.

TABLE 3 Correlations with Class II Criterion Variables (M = .46)

	Substance use	Depression	Distress	Social support
Inventory of Professional Functioning	.57	.50	.36	−.40

The IPF had the highest correlations with its own subscales. Higher correlations give evidence of convergent construct validity at the scale level. A correlation matrix detailing the correlations between the subscales of the IPF is seen in Table 4.

TABLE 4 Correlation Matrix with Class III Criterion Variables (M = .53)

	Competence	Reliability	Commitment
Judgment	.74	.55	.45
Competence	—	.56	.45
Reliability	—	—	.40

Summary of the Reliability and Validity of the IPF

Face validity and content validity were established during the process of defining the constructs to be measured and using experts to develop the items to be used in the scale. Exploratory factor analysis, Cronbach's alpha, and item-total correlation matrices were used to identify and remove poor items and identify hypothesized factors. A coefficient of content validity was determined using the mean of the corrected item-total correlations. Structural equation modeling was used to confirm factorial composition and validity. Through this process construct validity at the scale level was established. Then convergent and discriminant validity were established at the scale level by examining correlations of the IPF with Class I, II, and III Criterion variables.

RESULTS

Most of the people who responded to the invitation to participate in this research were female (85.4%, n = 182). The average age of the sample (n = 215) was 44.5 (SD = 11.9) and the median age 45. Ages ranged from 22 to 77. The sample was not racially diverse with 96.2% being White, and small percentages representing African Americans (1.9%), Native Americans (.9%), and persons who were biracial (.9%). Two thirds of the sample (64.6%, n = 137) were married and 8% (n = 17) lived with a partner.

The average annual personal salary of the respondents was $38,811 (SD = 14,135) with a median of $37,000. The average annual household income was $76,824 (SD = 39,282) with a median of $70,000. Respondents had been working in the field between one-half and 44 years, or an average of 14.8 years (SD = 9.3) and a median of 12 years, and at their current job an average of 7.1 years (SD = 6.9) and a median of 4½ years. Sixty-five percent (n = 140) characterized their work settings as being in urban areas, leaving 35% (n = 75) who worked in rural areas. The largest percentage of the sample worked in mental health (31%), followed by 21% with children and families. Most of the sample hold MSWs (85.2%, n = 178). The Kentucky Board of Social Work does not keep demographics of its licensees so there are no data to ensure a representative sample. However the sample has demographics characteristics similar to NASW members of the North Carolina chapter (Siebert, 2003) and NASW National (Practice Research Network, 2003). The most notable difference characterizing this sample is the underrepresentation of minorities. Other demographics are notably similar (age, marital status, years in profession).

The Inventory of Professional Functioning (IPF)

The IPF has 17 items with possible scores ranging between 17 and 85. The sample had scores that ranged between 17 and 68, with a mean score of 30.18 (SD = 6.4) and a median of 30. Most of the sample had low scores, suggesting that most of the sample did not have problems that affected practice. About 15% of the sample scored 35 or higher, which would indicate clear problems with professional functioning. Twenty-five percent of the sample scored 32 or higher. Clinical cutting scores have not been developed for this measure, and it is not clear what score indicates clinical significance. Future work on this measure will establish levels where professional functioning deficits are significant enough to harm clients and determine the sensitivity of the measure.

DISCUSSION

The development of the IPF and reporting on the initial steps of the validation process at the scale level fill a gap in understanding professional functioning in social work. Greater attention to professional functioning is needed so that resources and supports can be allocated toward workforce development (Hoge et al., 2007). As social workers who need help are valued and supported they are better able to provide services that promote social justice for vulnerable clients. Previous findings of research on social workers suggests that at any given time between 20% and 35% have some form of impaired practice (NASW, 2006; Pooler et al., 2008), and this measure will further aid in understanding workforce problems. In the workplace social workers must manage their internal world, client issues and behavior, colleague and supervisor relationships, workplace productivity demands, and adverse organizational issues such as poor pay or unsupportive administrators. Some social workers have difficulty doing that because they have problems that affect their ability to manage these demands and stressors or the working environment may be hostile or unsupportive.

Resources can be allocated for prevention, earlier identification, and development of tangible supports for social workers experiencing impaired practice. Resource allocation of this nature depends on a fundamental valuation of employees. Many organizations already manage limited and scarce resources and moving more resources to support employees seems challenging. However, if supports are not developed for social workers and other human service professionals, impaired practice may affect client outcomes, the reputation of an agency, and even the profession. In an era of evidence-based practice, the profession can consider how the health of social workers affects client outcomes and offer increased attention to this important issue. Workforce health is not just a productivity or fiscal issue in social work, it is an ethical issue because social workers whose practice is impaired can harm clients.

The discussion should be considered in light of the study's strengths and limitations. After three mailed invitations to complete an Internet survey, 215 social workers responded—about 70 per mailing, yielding a response rate of 17%. This is less than many survey response rates; however there was no drop-off in responses, which suggests that additional waves of invitations could have increased the response rate. Of note, there were no differences found between early and late responders, suggesting that the nonresponders were no different than those who completed the survey (Siebert & Siebert, 2007). As a caution, the findings should not be generalized beyond social workers in Kentucky.

The N of 215 was adequate for the data analysis and validation of the new instrument with 17 items. However, a rule of thumb was not followed in this scale development process. Generally, scale developers suggest having 10 respondents per original item, which would have been 550. The sample used here was only half of that: there were five respondents per item. However there were 10 respondents per item for the critical confirmatory factor analysis stage of the validation process.

The IPF can be improved, and it is a measure under development, but currently the IPF can be used to measure problems and strengths in professional functioning. As a standalone measure the IPF can be used by organizations to develop a picture of employee health. Or it can be used with other measures to identify factors associated with impaired practice such as depression, distress, or substance abuse. The measure is also useful to identify relationships with protective factors that improve and support functioning in positive ways.

The development of the IPF is a next step in promoting and measuring social worker health in the workplace. This research is a response to Reamer's (1992) call for more research on impaired practice and another step in building on Siebert's ongoing work (2003, 2004, 2005a, 2005b) in understanding social worker health. Workforce health and development ought to be a priority for the profession.

The author would like to thank Jinseok Kim, PhD, Seoul Women's University, for offering his critical feedback on the manuscript and contributing his expertise in structural equation modeling.