The impact of contract characteristics on the performance of public–private partnerships (PPPs)

Abstract

Four significant features of public–private partnership (PPP) contracts are analysed to understand their impact on performance. These are whether the contract allows sanctions to be imposed; its complexity; its flexibility; and whether renegotiation is possible. The effects of these characteristics were investigated by surveying participants in all of the PPP projects in The Netherlands. The only feature considered to have any significant impact on perceived performance was the possibility of imposing sanctions. The authors’ findings cast doubt on earlier research into managing PPP performance and suggest that researchers, governments and the private sector need to look beyond contract terms to properly understand and manage PPP performance.

Keywords:

• Contracts
• performance measurement
• PPP
• public–private partnerships

Pursuing public–private partnerships (PPPs) has become a common governance strategy in many countries to improve public service delivery and for large public infrastructure projects. PPPs involve the private sector, which is assumed to be better and more efficient at managing infrastructure construction, financing, maintenance and operation (for example by the use new technologies, new ideas or new ways of doing things). PPP contracts are meant to incentivize private sector partners to use their skills to produce better infrastructure-based public services. The academic literature on design, build, finance, maintain and operate (DBFMO) contracts in PPPs stresses that contracts are crucial for success (NAO, 2002; ODPM, 2002, 2004; Pollitt, 2002; Koppenjan, 2008).

The length of the contract period and the possibilities for imposing sanctions when services or products do not comply with a contract are generally thought to be especially important. Long-term contracts enable private partners to employ more innovative techniques and delivery methods. Sanctions are necessary to anticipate and respond to opportunistic behaviour by contract partners and to improve their performance during contract implementation.

From the perspective of relational contracting (Deakin and Mitchie, 1997), the flexibility of the contract and the room it leaves for negotiation are emphasized. Another characteristic that is considered important is the contract’s complexity. Drafting and implementing complex contracts requires more attention and higher transaction costs and therefore may influence the performance of partnerships (Williamson, 1996).

This paper presents a study that addressed the following research question:

What is the influence of PPPs’ contract characteristics, especially the length of the contract period, the possibility of sanctions, the complexity, the possibility for negotiation and flexibility, on the performance and innovativeness of these partnerships?

To answer this question, we conducted a survey of PPP projects in The Netherlands in which we asked respondents to assess contract characteristics and outcomes.

PPPs and contractual arrangements: theoretical considerations

Arguments in favour of PPPs usually relate to the terms of the contracts. Their relatively tight character, with clear provisions for monitoring and sanction is seen as a strength (Weihe, 2009). A long contract period allows private parties to spread their risk and innovate. Both of these assumptions are inspired by (neo)economic institutional theory and by new public management ideas (which are strongly influenced by economic theories). Other disciplines emphasize PPPs’ relational character and thus the possibilities for renegotiation and building trust (Williamson, 1996; Deakin and Michie, 1997; Brown et al., 2007).

PPP contract characteristics

Length of the contract period: In the literature on DBFMO contracts, contract length is mentioned as important because private partners need time to recover their initial investment (see Weihe, 2009; Hodge et al., 2010). Long contracts may also contribute to the overall quality of the product or service. Because these projects integrate a number of phases, contractors are able and incentivized to invest in better materials in the construction phase in order to have fewer maintenance costs later on (NAO, 2002; ODPM, 2004). So, long contract periods can be associated with a good overall performance: lower costs (cost-efficiency), better quality services and products, and more innovative solutions and products.

The length of the contract period also creates conditions for innovation by providing private partners with incentives to come up with new, innovative solutions regarding the way they organize processes and the products and services they provide. The extra investment needed by these innovations will be more affordable with long contract periods during which there is a guaranteed cash flow (Pollitt, 2002; Koppenjan, 2005; Lenferink et al., 2013). Therefore, we hypothesize that:

H1: A longer contract term relates positively to a better PPP performance.

H2: A longer contract term relates positively to innovation in PPPs.

Sanctions: Another feature of PPP contracts highlighted in the literature is the use of sanctions (Hodge and Greve, 2005; Van de Velde et al., 2008). Given the possibility of unanticipated future developments, a government needs to have instruments to influence the behaviour and performance of the private partner during the contract period. In order to do this, contracts may include the option of applying positive and negative sanctions during contract implementation. This means that performance indicators and a monitoring system need to be specified in the contract (Williamson, 1979, 1996; Koppenjan, 2015). The core ideas about these features come mostly from neo-institutionalist economics. From a neo-institutionalist perspective, the specific investments (knowledge, money, material) made by partners in a PPP make those partners vulnerable (Deakin and Mitchie, 1997; Nooteboom, 2002). Specific investments are investments in activities or products that cannot easily be used in other projects. This makes the investing partner dependent on the other partner, which may lead to opportunistic behaviour for example taking short cuts or economizing on quality. It is crucial, therefore, that the contractor (the principal) is able to monitor and sanction the contractee (the agent) if the latter fails to deliver the promised product and the promised quality. So the threat of sanctions are assumed to contribute to the performance of a PPP. The literature does not provide any clues about the relationshipbetweensanctionsandinnovation (Pollitt, 2002; Koppenjan, 2005). We therefore hypothesize on the influence on overall performance only:

H3: A contract with possibilities for applying sanctions relates positively to the PPP performance.

Complexity of the contract: The role of transaction costs and the type of relationship are important issues to consider in choosing the type of contract, or, as Williamson (1979) says, the type of governance. When transactions are not frequent and investments are undertaken by specific partners, there is a need either for complex contracts that specify all possible eventualities, or for additional governance mechanisms (for example more investment in contract management).

According to Williamson (1979):

Whenever investments are idiosyncratic in nontrivial degree, increasing the degree of uncertainty makes it more imperative that the parties devise a machinery to ‘work things out’— since contractual gaps will be larger and the occasions for sequential adaptations will increase in number and importance as the degree of uncertainty increases.

As a solution, he proposed that:

Two possibilities exist. One would be to sacrifice valued design features in favor of a more standardized good or service. Market governance would then apply. The second would be to preserve the design but surround the transaction with an elaborate governance apparatus, thereby facilitating more effective adaptive, sequential decision making.

Of course, choosing his first option would imply renouncing the potential of added value (actually it implies choosing another type of solution/product). The second option entails additional transaction costs.

Complex projects addressed by PPPs require specific transactions. Consequently, more complex contracts are needed to govern these projects or, alternatively, different forms of governance (for example more relational contracting). A complex contract has the advantage of arranging many different things, but its disadvantages include that it is costly to draft (because a lot of information and negotiation is needed), it is less flexible and it will lead to high transaction costs for monitoring and implementation. Therefore, it can be assumed that the more complex the contract, the less its overall effectiveness. The relationship between complexity and innovation is less easy to understand, with few clear ideas in the literature. Therefore, we did not formulate an a priori hypothesis on this relationship:

H4: Contract complexity relates negatively to the overall performance of a PPP project.

Flexibility and renegotiation: Neo-institutionalist theory posits that different situations may require different forms of contracts. Other writers on contract theory emphasize that relational contracting is more appropriate in complex partnerships, where innovation and specific transactions are required, than in more classical contracts, which have fixed performance indicators (see Deakin and Mitchie, 1997). Relational contracting refers to contracts with global agreements that expect partners to act according to the spirit of an agreement in concrete situations that cannot be specified in advance (Williamson, 1996; Brown et al., 2007; De Bettignies and Ross, 2009). Several authors emphasize that, particularly when relationships are becoming more complex, it is essential that contracts are flexible. ‘Flexibility’ means, for instance, that performance specifications can be changed and contract terms renegotiated (Verweij, 2015) to allow for unknowable changes in conditions during the implementation of a contract. A rigid contract that does not allow renegotiation may result in suboptimal outcomes, or even in partnerships failing. Also, a rigid contract may not allow parties to implement new ideas and solutions (Athis and Saussier, 2007; Verweij, 2015). These considerations led us to the following hypotheses:

H6: Contracts that are more flexible and allow room for negotiation relate positively to the overall performance of a PPP project.

H7: Contracts that are more flexible and allow room for negotiation relate positively to innovation in PPPs.

Research methodology: respondents and variables

Our respondents were public sector officials and project managers, and representatives of private consortia or people in consultancy firms involved in PPP projects. Almost all of the PPP projects in The Netherlands (about 90) were included in our survey.

In total, 343 people received a request to fill in the survey questionnaire. We included as many respondents as possible because the average response to most surveys is about 30%. Of the 343, 10 were not reached (mail undeliverable), and 24 either did not want to participate in the survey or were not involved in the PPP project. This left 309 respondents that were actually approached. In total, 157 people responded to the survey, 13 returned the questionnaire without responding to any of the questions. This left 144 people who actually filled in at least part of the questionnaire—a response rate of 46.6%, which is quite high. The 144 respondents were involved in a total of 68 PPP projects in The Netherlands. Thus, the survey covered 73% of PPP projects in The Netherlands. More than one respondent from each project was included in the dataset, therefore we needed to do a multilevel analysis (MLA). Respondents were employed in private consortia (27.1%), consultancies (13.2%), public organizations (45.8%) and other organizations (11.8%) such as non-profit organizations and law firms. On average, the survey respondents had been involved in projects for 14 years; this indicates their considerable experience with such projects (also indicated by the average age of 48 years). Each respondent was asked to answer the survey questions with a specific PPP project in mind, which they also had to mention in the survey.

Dependent variables: project performance and project innovativeness

Measuring the performance of complex PPP projects is difficult. One way to operationalize performance is to look at the extent to which goals formulated ex ante are realized. However, PPP projects include a variety of actors and multiple goals. This makes it difficult (and arbitrary) to pick one main goal to assess a project’s performance. Furthermore, projects take quite some time to complete, so goals will evolve (Klijn and Koppenjan, 2016). Additionally, we used a survey; so we did not measure the actual outcomes of partnerships but, rather, our respondents’ perceptions of these outcomes. Thus, we used perceived project performance as a proxy for outcomes. We took this approach following earlier work by Steijn et al. (2011), whose measurement scales built on different dimensions of project performance—see table 1. We found, as Steijn et al. did, that the items formed a good scale (as indicated by the Cronbach’s alpha, which was over 0.7). The mean score for project performance, as rated by the respondents, was 4.00 (SD = 0.51) on a five-point Likert scale, indicating a high satisfaction with the performance of their project.

Innovativeness was measured by having respondents rate three items—see table 2. Cronbach’s alpha for these items was 0.78, which is a good score. The mean score for project innovation, as rated by the respondents, was 6.38 (SD = 1.69) on a 10- point Likert scale, indicating high levels of perceived innovation within projects.

Contract characteristics

To measure the main contract characteristics, we included four items (scored on a 1–10 scale between two extremes) in the survey. They were:

• Sanctions: The contract has no possibility of imposing sanctions if the contract terms are not met (the contract has many possibilities for imposing sanctions if the contract terms are not met). The mean value was 2.82 (standard deviation: 1.6).

  Table 1. Measurement of perceived project performance.

  Table 2. Measurement of perceived project innovation.

• Complexity: The contract is simple to understand (the contract is difficult to understand). The mean value was 4.9 (standard deviation: 2.2).

• Flexibility: The contract is characterized by fixed target values and norms regardless of the circumstances (the contract is characterized by flexible target values and norms that can be reduced or enhanced under certain circumstances). The mean value was 4.9 (standard deviation: 2.4).

• Negotiation: The contract offers very little space for renegotiation (the contract offers much space for renegotiation). The mean value was 4.95 (standard deviation: 2.2).

We also measured the length of the contract by asking the following question: ‘What is the length of the contract (in number of years)?’ The mean of this variable was 19.9 years and the standard deviation nine years.

Control variables

As control variables, we used the size of the network around the PPP project (coded in five categories: 0–4, 5–9, 10–14, 15–19, 20+ organizations) and the respondents’ organizational background. The last variable was coded in three categories: public organization, private organization, and consultancy firm.

Common method bias

Measuring both independent and dependent variables from the perceptions of the same survey respondents entails the risk of commonmethod bias. There are several tests to examine whether common method bias is a problem. The best known is the Harman one-factor test. The Harman test assesses whether one single factor explains the correlations in the data. For that, an unrotated factor analysis had to be performed with the items. We did this, and no single factor emerged. (Actually, we performed two separate factor analyses with the two dependent variables. The first factor explained 28% or 29% of the variance. If we put all the variables in a factor analysis, the first factor explained only 26% of the variance.)

Another test advocated by various researchers is the Lindell and Whitney (2001) test. This test uses a theoretically-unrelated construct as a marker variable to check for common method bias. Any high correlation of the marker variable with any of the other variables is an indicator of common method bias. We used another survey variable, not used in this study, as a marker (the extent to which parties have worked together before). Table 3 shows the correlation coefficients and R2 between variables in the model and the marker. The correlations were low (maximum R2 = 0.034)—this suggests that common method bias was not a problem in our survey data.

Table 3. Correlation and R2 between variables and marker.

Data analysis: MLA

We carried out an MLA as the observations in the dataset were not fully independent (Hox, 2002). Using multilevel regression analysis makes sense only if there is a sufficient variation in the dependent variable at the project level. In our analysis of perceived performance, for instance, according to the baseline (null) model, about 42.6% of the variation in perceived project performance could be explained by project- relevant characteristics—individual-level variance = 0.1549; project-level variance = 0.1152. In the case of perceived project innovativeness, 47.8% of the variation could be explained by project-relevant characteristics. Given the high percentage, combined with significant -2ll values, it was clear that single- level regression analysis was not going to be suitable.

Contract characteristics and their impact on PPP performance

Our data consisted of projects that were officially recognized by the Dutch government as PPP projects. Most of the PPP projects in the survey were managed through a contract (92%).

To examine the relations between the variables, we first performed a simple correlation between the variables of the conceptual model. The results are shown in table 4. The first observation was that there were few significant correlations. There was a negative significant correlation between length of contract and innovativeness (contrary to H1); a negative relationship between the possibility of sanctions and performance (contrary to H3); and a negative relation between complexity of contract and both performance and innovativeness (in line with H4). Flexibility and the possibility of negotiation did not significantly affect performance or innovativeness. What is interesting is that the length of contracts was mostly negatively related to the other contract characteristics. So longer contracts were thought to be less flexible and to have fewer possibilities for negotiation possibilities.

Table 4. Descriptive statistics and correlations between variables.

Contract characteristics and (perceived) partnership performance and innovativeness

To examine the effects of contract characteristics, we performed two multilevel regression analyses: one with perceived performance as dependent variable and one with innovativeness as dependent variable. In both cases, we used the same independent variables: the five contract characteristics and the two control variables. The results are shown in table 5.

Table 5. Multilevel regression model for perceived project performance and perceived project innovativeness.

As table 5 shows, some contract characteristics had a negative effect on project performance and innovativeness. The possibility of sanctions and complexity both had a negative impact on performance and innovativeness; length of contract had a negative impact on innovativeness. Flexibility and the possibility of negotiation both had a positive effect on performance and innovativeness.

However, none of the contract characteristics were significant when we performed a multilevel analysis, with the exception of the possibility of sanctions, which had a significant negative impact on performance.

This means that no relation can be found between contract characteristics and performance and innovativeness, except that sanctions have a negative effect on performance (contrary to our expectation in H3 that sanctions would have a positive effect).

Conclusions: contract characteristics are not the key

In this paper, we looked at the effects of the complexity of PPP contracts, the possibility of negotiation, the possibility of sanctions, flexibility, and the length of the contract on performance. Only the possibility of imposing sanctions was thought to have any impact (negative) on PPP performance. Our findings therefore cast some doubt on earlier research emphasizing the importance of contract characteristics for project performance. Our findings suggest that we need to look elsewhere to find explanations for PPP performance.

Limitations of our study

Of course, our study has its limitations. In the first place, we looked at respondents’ perceptions. We also had only one item to measure the various contract characteristics (although we used a 10-point scale between two extremes). Furthermore, we selected only people who were actively involved in PPP projects and who thus were knowledgeable about those projects. As self-reported data on performance can have some drawbacks, we chose to explicitly indicate this by using the term ‘perceived project performance’.

Second, as the same respondents provided the information for both the independent and the dependent variables, inflated relationships between variables could have occurred. Fortunately, our tests indicated that common method bias as a consequence of self-reported data was not of great concern in this study, and that it is unlikely to confound the interpretation of results.

Third, as not all the respondents that we approached actually filled in the survey, for some (mostly larger) projects, more respondents reacted than for other projects. For this reason, we conducted multilevel tests, where applicable, to deal with the multi-level nature of our data. For about half of our projects, however, we had only one respondent per project who filled in the survey, whereas at least two respondents would have been better for aggregation of the results to the project level.

Despite the limitations, our findings are important because the literature on PPP—both the theoretical literature inspired by institutional economics and new public management ideas and the ‘grey’ literature from public organizations, consultancy agencies and audit organizations—emphasize the importance of contracts for PPP performance. So these findings certainly need some reflection.

Are contracts important or not?

First of all, our findings are perhaps less surprising than they seem at first glance. They are consistent with earlier surveys that did not find any significant relation between organizational characteristics of institutional PPPs (such as the form of special purpose organizations, their arm’s-length positions vis- à-vis principals and so on) and PPP performance (see Kort and Klijn, 2011; Steijn et al., 2011). These findings indicate that the influence of formal contracts and organizational arrangements on PPP performance seems to be overrated. Other factors may be important, perhaps even more important, like the managerial effort that is put into the PPP (see Steijn et al., 2011). There are also several authors who have criticized the basic assumptions behind the contractual PPP idea and questioned whether tight contracts can provide better and more efficient services (see Shaoul, 2005).

Of course contract characteristics do not have a direct influence on performance: they simply provide the possibility of renegotiation etc. Several earlier papers stress that it is managerial effort in partnerships that make the difference in performance (see Steijn et al., 2011; Kort and Klijn, 2011).

Another explanation could be that partnerships are more complex than the literature and the debate on DBFMO suggest (Klijn et al., 2008). A complex network of actors is involved in a PPP and their relationships are not all regulated by the contract. The network of actors involved in a private consortium, for example, is very complicated and diverse (builders, banks, consultants and operators) and it may well be that, despite having an integrated DBFMO contract, the reality behind the scenes of this consortium is highly fragmented, with relationships being arranged and governed in traditional ways.

So, do our findings imply that contract characteristics are of no importance in the performance of PPP projects? We think the case is more complicated than this.

Good contracts and other organizational features have relevance for PPP projects, despite our not finding any direct and strong correlation with outcomes. However, their influence may be revealed only in interaction with other factors. What is clear, though, is that they cannot guarantee good performance and innovation. Contracts are probably a necessary, or at least an important condition, but they are not an iron-clad guarantee of success. A wider range of factors are at work in the complex networks that make up PPPs. Further research is needed to improve our understanding of the relative importance of these factors and how they interact. For now, also on the basis of earlier research, we hypothesize that, in particular, agency factors like the quality of interaction, managerial activities and trust between the partners make partnerships work (see also Kort et al., forthcoming). Our findings are a warning to anyone who thinks that the performance of PPPs can be steered by contracts alone. They are probably not even the most important factor.

Acknowledgements

This paper was written as a result of the research project ‘Governance for smartening public private partnership’ implemented in collaboration with the University of Twente and financed by the NWO (Dutch research Council: No. 409-14-014) and co-financed by NSOB, Deltares, Rebel group, Resetmanagement, Twijnstra Gudde and Rijkswaterstaat.

Additional information

Notes on contributors

Erik Hans Klijn

Erik Hans Klijn is a professor of public administration, Erasmus University Rotterdam, The Netherlands.

Joop Koppenjan

Joop F.M. Koppenjan is a professor of public administration, Erasmus University Rotterdam, The Netherlands.