Performance measurement in university–industry innovation networks: implementation practices and challenges of industrial organisations

Abstract

From the perspective of industrial small and medium size enterprises (SMEs), this study explores the implementation of performance measurement practices and challenges in university–industry innovation networks. In this research, two single-case studies were conducted to explore the implementation practices and challenges of performance measurement in university–industry collaborations. Thirty Finnish SMEs in the first innovation network and 10 Finnish SMEs in the second innovation network participated in university–industry collaboration that were established to facilitate their involvement in a long-term innovation process. The results of the study revealed that industrial SMEs are interested in the performance measurement of societal-level outputs by university–industry innovation networks, even though they face challenges in understanding the aims and goals of the funding programmes. Furthermore, the results showed that the industrial SMEs understood the intellectual nature of the university–industry innovation networks, but their performance measurement activities were business related. Also the lack of understanding of the context and the process of the performance measurement in university–industry collaborations seemed to shift industrial organisations’ focus to the content stage of the performance measurement and to the use of traditional performance measures and tools to estimate the advantages gained.

Keywords:

• Performance measurement
• university–industry collaboration
• innovation network

1. Introduction

The long history of university–industry partnerships and collaboration activities (cf. Bishop, D’Este, and Neely 2011) represents one means of increasing and supporting industrial organisations’ competitive advantages and innovation activities. Currently, there is increasingly growing societal pressure on universities to act as operators for innovations and provide economic growth for society (Ankrah and AL-Tabbaa 2015). This is one reason why universities are actively forming partnerships with private sector organisations while fulfilling third-mission activities that facilitate the universities’ engagement with society. Various funding programmes (e.g. Horizon 2020) often expect universities to collaborate with industry organisations to ensure that the research conducted in collaborations can be exploited throughout society. Additionally, continuously rising operation expenses (e.g. salaries and real estate costs) and decreasing basic governmental funding have increased universities’ interest in seeking relationships with other organisations to secure and maintain the universities’ subject areas and competencies. For these reasons, the last two decades have witnessed an increase in partnerships and collaboration activities between universities and industry in several nations, including European Union countries (Slotte and Tynjälä 2003; Gertner, Roberts, and Charles 2011; Ankrah and AL-Tabbaa 2015). University–industry collaborations include different types of interactions. Usually, they form two types of collaborations. The first is academic engagement, which refers to the collaboration between universities and the utilizers of academic science, such as private- and public-sector organisations. The second is commercialization, which is the exploitation of university-generated intellectual property (Perkmann 2015). More often, industrial organisations also perceive universities as attractive partners in supporting their innovation activities (e.g. Mäkimattila, Junell, and Rantala 2015). Additionally, large organisations are no longer the only ones seeking a competitive advantage from partnerships with universities; small and medium-sized enterprises (SMEs) are more frequently involved in these collaboration activities (Perkmann and Walsh 2007; Bishop, D’Este, and Neely 2011).

The increase in collaboration and innovation activities poses numerous challenges for the management of university–industry collaborations. One main issue that these challenges have raised is how the collaboration and development activities should be measured and evaluated (Perkmann et al. 2013; Albats, Fiegenbaum, and Cunningham 2017). Because of the increased interest in these collaborative activities, some frameworks were proposed to improve their management and evaluation (e.g. Al-Ashaab et al. 2011; Perkmann, Neely, and Walsh 2011). These frameworks and tools, however, are mainly theoretical and there is a need for further empirical evidence of the implementation and actual use of the presented models and measures. Perkmann and Walsh (2007) suggested that empirical research should address the question of the evaluation measures that implemented in collaborative university–industry networks. Thus, the present study aims to provide a greater empirical understanding of the performance measurement used in university–industry innovation networks. To accomplish this goal, the current study explores the currently implemented performance measurement practices and the challenges to them from the perspective of industrial SMEs. This paper presents two single case studies in which the phenomenon is explored in practice. Empirical data for this study were gathered from two university–industry innovation networks that were established to support the long-term innovation activities and processes of industrial organisations. This study focuses on the issue of performance measurement as part of the management of these innovation networks. The following research questions guide the study:

•	RQ1: How is the implementation of performance measurement practices perceived by industrial SMEs in university–industry innovation networks?
•	RQ2: How are challenges to the implementation of performance measurement perceived by industrial SMEs in university–industry innovation-networks?

There is limited holistic understanding of these phenomena, such as the role of performance management and measurement in SMEs that work and collaborate in open innovation-driven networks (Bititci et al. 2012). One reason is the absence of easily accessible cases of networks that researchers could study (Bititci et al. 2012). However, in the present study, the research group is included in two collaborative university–industry innovation networks. This study’s results can be utilised by organisations that collaborate with universities, university researchers who participate and manage these collaborations, and corporate financiers and policymakers to develop better methods for managing and evaluating university–industry collaborations.

2. University–industry collaboration networks

Various collaborative networks have emerged in recent years in response to the changes in the operating environments of organisations (Camarinha-Matos and Afsarmanesh 2008). As industrial organisations continuously seek new methods to develop their businesses and promote new alliances to gain knowledge and a competitive advantage, they also increasingly view universities as potential sources of innovation and knowledge. Consequently, an increasing number of organisations are pursuing knowledge and innovation by forming partnerships and collaborations with universities (Perkmann, Neely, and Walsh 2011; Perkmann et al. 2013; Cunningham and Link 2015). In industrial organisations, the prospect of collaborating with universities is tempting and can also support learning and information transfer between academy and workplace (Konkola et al. 2007; Reeve and Gallacher 2007). Because of the increased engagement, the effects of university research on the innovative activities of organisations have become the focus of academics and policy makers (Bishop, D’Este, and Neely 2011).

According to Ankrah and AL-Tabbaa (2015), the university–industry collaborations most common in practice and in the literature are alliances, networks, joint ventures, and consortia. These forms vary in the degree to which the participating organisations are connected. Through these different forms of collaborations, organisations can typically participate in projects focused on specific scientific or technical areas. However, in some cases, partnerships are formed based on long-term development and collaboration, instead of the need to solve a technical problem or to quickly create commercial products. From these long-term collaborations, organisations are interested in gaining more social capital and innovation capabilities, for example. Perkmann and Walsh (2007) suggest that university–industry partnerships and collaborations are commonly practiced although some differences might exist among various industries. These cited authors also demonstrate that open and networked innovation activities suggest that actual partnerships and collaborations between universities and industrial organisations – rather than generic links – play a stronger role in supporting the innovation activities and capabilities of the participating organisations. They highlight that organisations in these partnerships expect to see increased capacity for innovation rather than immediate commercialised tangible outcomes.

Collaborations with universities are no longer strictly the pursuit of large organisations. Both small and large organisations are forming collaborative networks to develop and support their innovation activities in order to generate value to markets and customers (cf. Bititci et al. 2012). In the future, SMEs will likely play even more important roles in economic growth, job creation, and innovation development. As such, the industrial SMEs are expected to form and work in collaborative networks, contributing to and benefiting from the emerging innovation environments (Bititci et al. 2012). Despite the importance of the management and the control of these networks, the current state of knowledge regarding performance measurement seems limited to studies from more traditional performance measurement perspectives. Moreover, university–industry collaborations provide open-innovation surroundings to complement traditional internal innovation and development activities (Coombs, Harvey, and Tether 2003; Ankrah and AL-Tabbaa 2015).

4. Performance measurement in university–industry collaborations

Since collaboration among organisations has increased, academic circles have begun to focus on the management and the role of performance measurement in collaborative activities and networks (e.g. Tsai 2009). When collaboration activities among organisations become more structured, they must be managed and evaluated properly, or the risk of failure in given tasks increases. In other words, if organisations aim to develop and sustain their competitive advantages through collaboration, the structure of the latter must be understood and managed, otherwise, the objectives will not be achieved and the aims of the partnership or collaboration will not be attained (Ukko et al. 2015).

According to Kaplan, Norton, and Rugelsjoen (2010), understanding how to measure and evaluate network-level performance supports collaboration activities that enhance management, strategy, and commitment at the network level. This performance measurement should be implemented, properly adapted, and visible to all participants to enhance decision-making that promotes the management of the collaboration and network activities. The literature on performance management has recognised the trend toward inter-organisational work, in addition to the changing contexts in which performance measurement is used (Bititci et al. 2012). According to Bititci et al. (2012), the thinking has already advanced from simple collaborative organisations involving a few partners to complex networks of organisations that work together to innovate and gain a competitive advantage. However, there seems to be a lack of holistic understanding of the performance measurement and the challenges associated with such collaborative networks of organisations. According to Bititci et al. (2012), the problem is how the performance of the collaborative organisation should be managed while also managing the performance of the participating organisations as a complete system.

Another challenge to measuring and evaluating performance in these collaborative innovation networks is the involvement of a university. Organisations that collaborate in research and development projects with universities have recognised the need for systematic evaluation and measurement of the projects. Outcomes must be assessed and ongoing activities monitored in order to implement improvements during collaborations (Perkmann, Neely, and Walsh 2011). Consequently, the development of systems to measure the performance of organisations’ innovation collaborations and to assess the results of these collaborations is of paramount importance for both industrial organisations and universities (Piva and Rossi-Lamastra 2013). Traditionally, university–industry collaborations have emphasised easily quantifiable output measures, such as patents or academic publications (e.g. Grimaldi and von Tunzelmann 2002). The participating organisations, however, may find it difficult to evaluate organisational operations and actions, the processes generating the outputs, and their actual effects on innovation capabilities. Additionally, the evaluation is often subjective, based on a participant’s satisfaction with the process and the outcomes (Perkmann, Neely, and Walsh 2011). A fully unified set of performance measures for university–industry collaboration evaluation could not exist because each collaboration and alliance is unique and each case differs by characteristics such as the form of collaboration, its goals, resources, partners relationships, and goals (Rossi and Rosli 2015; Albats, Fiegenbaum, and Cunningham 2017). However, there also exist some similarities among university–industry collaborations. Thus, it would be important to define and implement performance measurement practices and key performance indicators (KPIs) that include elements common to all university–industry collaborations. In this way, the implemented measurement practices will support the evaluation by providing directions for improvement in current and future collaborative initiatives (Flores, Al-Ashaab, and Magyar 2009; Albats, Fiegenbaum, and Cunningham 2017).

Several studies on performance measurement have suggested that to understand how a performance measurement system (PMS) can be designed, built, and used, its context, process, and content must be captured (Pettigrew 1985; Cuthbertson and Piotrowicz 2011). In this approach, the participating organisations’ roles and responsibilities, together with their joint vision, provide grounds for measurement and evaluation. Cuthbertson and Piotrowicz’s (2011) framework, originally presented by Pettigrew (1985) and constructed for the supply chain context, incorporates the following elements:

•	Context. Under what conditions does the measurement take place? These comprise factors that have impacts on the collaborative organisations’ performance measurement, the organisational context (internal organisational factors), and the collaborative network context (factors specific to the collaborative network environment).
•	Process. How is the performance measurement carried out? It involves the tools, methods, and frameworks used to measure collaborative network performance; the ways that data are captured, presented, and used, as well as the development of the measurement system.
•	Content. What is measured? It includes metrics, levels, categories, and dimensions.

Several authors have also discussed the evaluation process for collaboration in general and university–industry collaboration in particular (Perkmann, Neely, and Walsh 2011; Albats, Fiegenbaum, and Cunningham 2017). Perkmann, Neely, and Walsh (2011) identified four stages of university–industry collaboration: inputs, in-process activities, outputs, and outcomes. Some metrics were presented in other studies (Perkmann, Neely, and Walsh 2011; Tijssen 2012; Albats, Fiegenbaum, and Cunningham 2017), which could be utilised in university–industry collaboration:

•	Input: e.g. both parties’ resources (time, money, and staff allocated to collaboration), and the capabilities and motivation of both parties.
•	In process: e.g. relevant research, high-quality research, and training and learning opportunities.
•	Output: e.g. new technologies, new scientific knowledge, and skilled and trained staff.
•	Impact/outcome: e.g. new ideas, solution concepts, innovation, and human capital.

Although the literature shows increasing interest in the performance measurement of university collaborations and its benefits, many current collaborations fail in the implementation of the measurement practices. For that reason, comprehensive PMSs are not actively used in university–industry collaborations. Even though performance measurement practices are carefully designed and built, implementing and using them as part of management can be a challenging task. Hence, after constructing the PMS to university-industry collaboration, the focus should be on implementing the system. It has been suggested (e.g. Bourne et al. 2000) that the task of implementing and utilising PMSs is far from complete at the conclusions of the design and building phases.

5. Research design

This research involved two single-case studies that explored the implementation practices and the challenges of performance measurement in university–industry collaboration. The phenomenon was explored in two university–industry SME innovation networks in Finland. The two different cases were not used to perform a comparative study of the phenomenon but to try to gain a deeper understanding of the phenomenon through the collection of a large amount of data. A case-study strategy was used to answer ‘how’ and ‘why’ questions and gain an in-depth understanding of the phenomenon under study (Yin 2009).

Case studies focus on understanding a certain phenomenon (Eisenhardt 1989). For that reason, the case study approach was chosen and utilised in this study to explore the phenomenon of performance measurement in university–industry innovation networks. As university–industry innovation networks are a growing and under-studied research area, performance measurement as part of their management is not fully understood. In the present study, the researchers were able to study university–industry innovation networks in natural settings and real life contexts.

Although a case study is sometimes considered a single research method, it should be viewed as allowing the employment of various qualitative and quantitative approaches, such as analysing archives, conducting interviews, and using questionnaires (Gummesson 2000; Yin 2009). The possibility of employing different qualitative and quantitative approaches to gain in-depth empirical level understanding of the phenomenon was another motivating factor in our selection of the case study approach. This paper, thus, presents two single-case studies focusing on performance measurement in university–industry collaboration. The empirical data were gathered using a variety of methods, as presented in detail in Table 1.

Table 1. Summary of the data collection.

Data collection method	Respondents in each phase of data collection	Number of participants	Data analysis methods
Individual interviews	1–2 members of each participating organisation (CEOs, sales managers, development managers)	50 persons	Qualitative content analysis, cross-analysis
Workshop observation	Participating organisations:	Workshops involving 15–45 participants	Qualitative content analysis, cross-analysis
	10 workshops in Case A
	3 workshops in Case B	Workshops involving 8–15 participants
	Data based on observations, discussions, and feedback during the workshops
Individual interviews	Interviews arranged after the project to explore the evaluation and effects of the collaborative actions	10 from Case A	Qualitative content analysis, cross-analysis
		5 from Case B
	1–2 members of each participating organisation (CEOs, sales managers, development managers)
Survey	Survey arranged after the projects to explore the evaluation of the networks	10 from Case A	Quantitative analysis
		5 from Case B

With regard to the present case study, the review of the extant literature revealed minimal practical knowledge about the practices and the challenges of performance measurement implementation in university–industry innovation networks. Thus, the present case study provides new, practical information regarding the performance measurement in university–industry collaboration. In this study, 30 Finnish SMEs in Case A and 10 Finnish SMEs in Case B participated in university–industry innovation networks that were established to facilitate their involvement in a long-term innovation process. The data were collected in formal individual interviews with participating organisations in the building phases of both networks, during 10 group workshops at the working phase in Case A and during three group workshops in Case B. The data included the feedback provided after the workshops and in formal interviews with the participants during the evaluation phase of the collaboration. The participants in the group workshops were organisation representatives, university researchers, and project workers. Three to five university observers were present at each group meeting, which was documented. The researchers also participated in individual discussions with the participants before, during, and in intervals between the meetings. The interviews were structured beforehand and recorded for use in the data analysis. The individual discussions were informal, and they were documented in field notes. Researcher triangulation was used to validate the interpretations of the data, which were gathered with various methods. During the coding process of the data, investigator triangulation (involving three performance measurement researchers) and data triangulation were used to overcome the potential biases derived from single-observer and single-data-set studies.

5.1. Case descriptions

Universities have traditionally formed alliances and collaborative research and development projects with SMEs that operate in different manufacturing and production industries. However, in both innovation networks in Cases A and B, the study aimed to establish university-innovation networks with organisations operating mainly in service businesses. In both cases, the idea of the innovation and development activities during the processes was to support and develop long-term innovation capabilities of the participating organisations. In both cases, the innovation networks were constructed and established such that the participating industrial organisations could focus on and develop only the topics that they found necessary, which meant that they had autonomy in deciding the topics on which they wanted focus in collaboration with other organisations and university researchers.

Case A is a university-facilitated innovation network that was established to facilitate the participation of industrial SMEs in a long-term innovation process. Thirty Finnish SMEs were brought together to collaborate and develop innovation activities and capabilities and to generate new knowledge to support them. Traditionally, such collaborations or networks are built around defined fields of business or clusters. In this case, the network was built around three focal themes that were crucial for local regional business activities. The focus was on SMEs in the service sector. The idea was to increase the innovation capability, not only of individual organisations but also of the entire network, and decrease the SMEs’ barriers to participating in partnerships and collaboration activities with the university. The established university-industry innovation network was a horizontal alliance in which open innovation tools were used to seek information and to build cooperation between industrial organisations and local university units. The working methods used during the networking were highly participatory, group-based activating methods that were developed and designed based on the themes of the group meetings.

In Case B, the basic structure of the university–industry innovation network followed the model presented in Case A. The established university–industry innovation network was a horizontal alliance in which open innovation tools were used to seek information and to build cooperation between industrial organisations and local university units. Ten Finnish industrial SMEs collaborated in a long-term innovation network to develop and innovate how future technology solutions and tools could be utilised better as part of preventive healthcare services. The collaborative R&D project in this case was a follow-up to a research project that focused on the role of user-oriented gerontechnology in elder care services. The memory and reminiscence stick (mStick) and the health stick (hStick) for increasing user involvement in the services was developed during the project (Pekkarinen et al. 2013) that was conducted to follow up the innovation network. Thus, the SMEs participating in the network were in business areas related to IT services, healthcare services, and sports and recreation services.

5.2. Data analysis

Cross-case analysis was selected as a main method for this study because it can facilitate the comparison of the processes that are the units of analyses in selected cases. Cross-case analysis also enables researchers to explore concepts, theories, and hypotheses among different contexts and surroundings. In this study, the cross-case analysis aimed to explore the commonalities of performance measurement implementation in two different innovation networks. Data were collected in individual interviews with participating organisations in university–industry partnerships during the building phases of Cases A and B. These were analysed through a multi-coding process to generate patterns related to performance measurement implementation practices and challenges. During the coding process, these were structured into more generic factors. In the first round of the coding process, the data were analysed and arranged utilising Cuthbertson and Piotrowicz’s (2011) framework, which incorporates context, process, and content factors. The aim of the first coding round was to find the patterns related to each factor. The second round of the coding process was undertaken when the cases were ongoing, utilising the data gathered from the interviews, the workshops, and the individual discussions with the participants. During this second round, generic patterns related to current challenges and practices of performance measurement in university–industry innovation networks were arranged by utilising the four-stage evaluation process model (input, in-process, output, outcome/impact) (Perkmann, Neely, and Walsh 2011). The aim of the second coding round was to find patterns related to the practices in the implementation of performance measurement in each stage as well as the challenges to them. In the third round of the coding process, the data gathered from the workshops and the individual interviews after the cases ended were used to identify the performance measurement implementation practices and challenges arranged to evaluate the success of the university–industry innovation networks. At each stage of the coding process, research triangulation was used to validate the structured patterns. Table 2 summarises the overall coding process of the data.

Table 2. Summary of the coding process.

Coding	Data utilised	Aim of the coding	Main results
First round	Individual interviews/discussions:	To arrange patterns related to implementation challenges and practices of performance measurement in university–industry innovation networks from the perspective of industrial organisations	• Implementation practices and challenges related to: • the context of performance measurement • the process of performance measurement • the content of performance measurement
	1–2 members of the each participating organisation (CEOs, sales managers, development managers)
Second round	Individual interviews/discussions:	To arrange generic patterns related to the current implementation of input, in-process, and output measures in university–industry innovation networks	• Practices and challenges related to the implementation of: • input measures • in-process measures • output measures
	1–2 members of each participating organisation (CEOs, sales managers, development managers)
	Workshop observations:
	10 workshops in Case A
	3 workshops in Case B
Third round	Workshop observations:	To arrange patterns related to performance measurement practices and challenges to the success of university–industry innovation networks	• Practices and challenges related to the implementation of: • outcome/impact measures • collaboration success measures
	10 workshops in Case A	To arrange patterns related to performance measurement practices and challenges to outcome and impact measures
	3 workshops in Case B
	Interviews arranged after the project to explore the
	evaluation and effects of the collaborative actions
	1–2 members of each participating organisation (CEOs, sales managers, development managers)
Additional data analyses	Survey conducted after the projects to explore the evaluation of the networks	To confirm the findings and patterns arranged in the previous rounds of coding	• Implementation practices and challenges related to the context, process, and content of performance measurement • Practices and challenges related to the implementation of input, in-process, output, and outcome/impact measures

In the final round of the coding process, additional data collected in the survey were used to confirm the findings and patterns generated during the previous coding rounds. The patterns generated during the first and second rounds of coding were used to formulate the survey questions. The survey data were coded by calculating the mean and standard deviations for the responses to each question.

6. Results of the study

University–industry collaborations by universities and industrial organisations have larger societal effects and benefits. Their funding agencies (e.g. the European Commission’s Horizon 2020, European Regional Development Funds, and National Research Councils and Foundations) also provide societal-level instructions and evaluation criteria for collaboration. Nevertheless, the SMEs participating these collaborations in the form of innovation networks were strongly oriented to the operational level. The study’s results revealed that the primary focus of the participating organisations’ collaboration and innovation networking activities seemed to be gaining more innovation capabilities and intellectual capital, for example, new organisational learning and development practices and familiarity with other organisations’ businesses. These capabilities were perceived as enablers for business gains from the collaboration operations. The organisations’ representatives offered the following reasons for participating in the collaboration activities:

•	They wanted to become acquainted with other entrepreneurs. Personal relationships and trust have significant roles in formal contracts, so one main reason for participating was to accumulate social capital.
•	They hoped to gain a better understanding of other organisations. Many of the participants had basic knowledge of the others’ business ideas, but they lacked an understanding of how these businesses operated on a practical level.
•	They were interested in promoting and marketing their own products and services. When the participants had the chance to meet the other organisations’ representatives at the same time, they regarded it as a good opportunity for marketing their respective organisations.
•	Some of the organisations’ representatives also wanted to become familiar with the university (e.g. to obtain recent research results), but the university’s most important role seemed to enable the organisations to collaborate with one another.

Reflecting on research question one, in contrast to the literature on the performance measurement of university–industry collaborations, in both Case A and Case B, the industrial organisations recognised the importance of the performance measurement of these activities. The participating SMEs mainly shared the opinion that a PMS should be created to evaluate the performance of individual organisations and the entire network. The results also showed that the actual purpose of the network-level PMS in university–industry collaboration activities was not as obvious to the industrial organisations. In normal business environments and as part of daily operations, the participants were familiar with evaluating and measuring the performance of individual organisations. However, they were unsure of the purpose of evaluating and measuring the performance of collaborative activities. The lack of understanding of the actual using purpose of the PMS was found to be one of the main challenges to the implementation of measurement systems and individual measures.

With regard to the first research question, the content of the performance measurement of the innovation network (Cuthbertson and Piotrowicz 2011) seemed to be clearer for the participating organisations than its context and process. Based on the reasons given for the organisations to participate in these collaborative networks, it seemed to be easier for them to understand what should be measured in these alliances (e.g. increased learning, increased social capital, and new innovations). These issues have received attention in academic circles over the last few years. Although several studies have confirmed the positive relationship between organisations’ innovativeness and business performance, the literature lacks frameworks or models for measuring and evaluating innovation capabilities. Saunila and Ukko (2012) present a conceptual framework for measuring innovation capability and its effects. They emphasise that it is not enough to know how many new innovative processes, actions, or products have been initiated if there is no understanding of their connection to business performance. However, due to this phenomenon’s novelty, even in academic circles, it was found to be unknown among industrial organisations, which caused the failure of the implementation. The industrial organisations did not seem interested in evaluating the process inputs in university–industry innovation networks. Their current focus and interest in performance measurement was the evaluation of process activities and process outputs and outcomes (e.g. new customer relationships, innovations, and strategic partnerships) with other industrial organisations. The role of the university in these innovation networks was mainly perceived as a facilitator for the research and development of industrial organisations and their innovation activities. For that reason, the participants expressed that they were not interested in the performance measurement of the activities and outcomes of their collaboration with universities. However, some participants said that the business-related effects and outcomes of the collaboration with the university could be measured. For example, new companies that were spinning off from the university to fulfil the university’s third mission should be evaluated.

Regarding research question two, the SMEs participated in these collaborations to gain innovation capabilities and human capital, indicating that the participants were not capable of evaluating the societal level activities or effects of the collaborations. Many participants mentioned that although they would be interested in evaluating the societal effects of university–industry collaboration, they lacked a connection between their operational networking activities and the evaluation criteria provided by funding agencies. It was also commonly mentioned that the participating SMEs were not aware of the aims and goals of the funding programmes. From their point of view, they participated in individual research and development projects.

In further response to research question two, the findings reflected the challenges to understanding the purposes of the performance measurement. Specifically, the findings showed that the context (Cuthbertson and Piotrowicz 2011) of the performance measurement of the university–industry innovation networks was unclear to the participating SMEs. Moreover, the findings indicated that they were confused about the conditions under which the measurement should take place. This observation was supported by the results gathered during the workshops, which revealed that the participating SMEs found it hard to form a joint vision for the innovation network; thus, it was difficult for them to understand their roles and responsibilities as part of the network. With regard to the first research question, the results also revealed that the SMEs were mainly interested in measuring the innovation activities and the advances gained in the activities with other industrial organisations. The results indicated that although the motivations for participation in innovation networks were apparently related to intellectual capital, the performance measurement activities seemed to be strongly business related. Nevertheless, the participating organisations’ representatives seemed to understand that it might take time for industrial organisations to realise that intellectual advantages could be gained from collaborating in innovation activities with other SMEs, including increased revenues or lower costs. For this reason, they acknowledged that during the process, innovation capabilities and intellectual capital should be evaluated even though they found it challenging.

Reflecting on both research questions, we see that in addition to the uncertainty about the context of the performance measurement in the university–industry innovation network, the participating SMEs faced challenges in the process state (Cuthbertson and Piotrowicz 2011) of the performance measurement. Instead of implementing performance measurement tools and practices that were designed for the university–industry context, they had tried to use existing ones to capture the performance of the collaboration from the perspective of their own performance. However, these tools and methods, normally used to evaluate operations during daily business activities, were not found suitable for evaluating the performance of the university–industry innovation network. These traditional tools and measurements were the so-called ‘hard measures’ that focused on tangible outputs and were mainly mentioned as related to business performance, not concentrating on intangible aspects and innovation capabilities that were set as parts of the main target of the collaboration. Table 3 summarises the current challenges and practices related to the implementation of performance measurement in university–industry innovation networks.

Table 3. Summary of performance measurement practices and challenges of industrial SMEs.

	Industrial SME’s views on implementation of performance measurement practices	Challenges related to the implementation of performance measurement
Societal-level performance measurement of university–industry innovation networks	• Interest in the performance measurement of societal-level outputs of university–industry innovation networks.	• Industrial SMEs are not aware of the aims and goals of the funding programmes. From their point of view, they participate in individual research and development projects.
Network-level performance measurement of university–industry innovation networks	• Industrial SMEs recognise the importance of the performance measurement of network-level activities. The participating SMEs mainly shared the opinion that a PMS should be created to evaluate the performance of individual organisations and the entire network. • SMEs are mainly interested in measuring the innovation activities and advances gained in activities with other industrial organisations. Although the motivation for participation in innovation networks seem related to intellectual capital, the performance measurement activities seem to be strongly related to business.	• The purpose of the network level of PMS in university–industry collaboration activities did not seem obvious to the industrial SMEs. • The context of the performance measurement of the university–industry innovation networks seemed unclear for the participating SMEs, and they seemed to be confused about the conditions under which the measurement should take place. • Although the literature on the performance measurement of university-industry collaborations recognises the frameworks developed for such contexts, the industrial SMEs seemed to be unfamiliar with them and to lack the proper understanding and means to implement them.
Individual-level performance measurement of university–industry innovation networks	• Performance measurement of goals and tasks that were originally defined for the aims of collaboration. • The current focus and interest in performance measurement seemed to be evaluating process activities and process outputs and outcomes (e.g. new customer relationships, innovations, or strategic partnerships) with other industrial organisations. • Interest in the performance measurement of increased learning, increased social capital, and new innovations.	• Challenges in developing and implementing new tools and methods to evaluate or measure the results or effects that were secondary outcomes of the collaboration. • Industrial SMEs currently seemed to be unfamiliar with frameworks and tools for measuring and evaluating innovation capabilities, which caused the failure of implementation.

7. Discussion

An important aspect of university–industry collaborations concerns their effects on society (Rossi and Rosli 2015), which should be measured in order to show the utility of university–industry collaborations. The results of the study showed that instead of using performance measurement tools and practices to track the societal-level effects of university–industry innovation networks, industrial SMEs are interested in measuring societal-level outputs that are business related. The results also showed that the industrial SMEs were interested in implementing such measurements in the university-industry innovation networks while the projects were running. Nevertheless, they faced challenges in implementing the measurement practices and tools. The biggest challenges were related to challenges in understanding the connection of operational-level activities to the ‘bigger picture’ as well as the societal-level aims and goals of the funding programmes. Recent studies (Perkmann, Neely, and Walsh 2011; Rossi and Rosli 2015; Albats, Fiegenbaum, and Cunningham 2017) found that to achieve societal-level effects as a result of collaboration, the participants should organise joint public lectures or write press releases. However, the industrial SMEs viewed such activities and their measurement as the responsibility of the university. Hence, they should be implemented by university operators. One reason for the industrial SMEs interest in the measurement of societal-level outputs instead of effects, which supports the findings of Cunningham and Link (2015) and Albats, Fiegenbaum, and Cunningham (2017), was related to the different timeframes that the parties used. Although industrial SMEs participated in networking activities aimed at supporting long-term innovation, short-term performance measurement practices were implemented so that the business value could be tracked while collaboration activities were performed.

Regarding the context (Cuthbertson and Piotrowicz 2011) of the performance measurement of the university-industry collaboration, the industrial SMEs considered difficulties in forming a joint vision for the innovation network and understanding the purpose of the network-level PMS. Instead of developing and implementing network-level KPIs, the industrial SMEs implemented KPIs that measured the performance of their own operations as a part of the network. For example, the participating SMEs used impact measures such as strategic partnerships and change/renewal of business revenue structure, which Albats, Fiegenbaum, and Cunningham (2017) showed were among the most important KPIs in university-industry collaboration. According to these authors, as an indicator, the new strategic partner meant the possibility of future collaborations based on the experience gained in the current or recently finished joint projects (Perkmann, Neely, and Walsh 2011; Albats, Fiegenbaum, and Cunningham 2017). However, instead of evaluating the future collaborations with the university, the industrial SMEs seemed interested only in evaluating the future collaboration possibilities with other SMEs, which indicated that their interest in evaluating partnerships was business related. In addition to the challenges in the context of the performance measurement in university–industry innovation networks, the process (Cuthbertson and Piotrowicz 2011) of the performance measurement was considered difficult to implement. Although literature on the performance measurement of university–industry collaborations recognises frameworks that were proposed to improve their management and evaluation (e.g. Al-Ashaab et al. 2011; Perkmann, Neely, and Walsh 2011; Albats, Fiegenbaum, and Cunningham 2017), the industrial SMEs seemed unfamiliar with them. Therefore, these organisations lacked the means and understanding required to implement the proposed frameworks and measures. Therefore, the industrial SMEs mainly implemented the traditional performance measurement tools and methods that were originally developed to evaluate their daily business operations.

While the industrial SMEs faced challenges in understanding the context of performance measurement and implementing it in university-industry innovation networks, the results indicated that they were familiar with the content (Cuthbertson and Piotrowicz 2011) of performance measurement. It was perceived as deriving directly from the reasons and the motivations for participating. Some reasons included becoming acquainted with other entrepreneurs, accessing learning opportunities, creating innovation capability, and sharing knowledge. The challenges to understanding the context and process of university-industry innovation networks also caused the industrial SMEs to be interested in the measurement of the outputs (Perkmann, Neely, and Walsh 2011; Albats, Fiegenbaum, and Cunningham 2017) of university-industry innovation networks.

8. Conclusions

This study explored performance measurement implementation practices and challenges in university–industry innovation networks from the perspective of industrial SMEs. The implementation of performance measurement practices and the emerging challenges to it were studied in two university–industry innovation networks. The results showed that industrial SMEs seemed to collaborate in innovation networks with the university to gain more innovation capabilities and intellectual capital, such as new learning and development practices. However, their performance measurement activities seemed strongly related to business. First, regarding the content of the performance measurement, and based on the reasons why the organisations participated in these collaborative innovation networks, it seemed easier for them to understand what should be measured in these collaborations, for example, increased learning, increased social capital, and new innovations. Second, regarding the context of the performance measurement, the participating organisations found it hard to formulate a joint vision of the innovation network; therefore, it was difficult for them to understand their roles and responsibilities in the network. The industrial SMEs seemed interested in societal- and operational-level performance measurement activities during the collaborations, and they expressed that the evaluation of outcomes and impacts should be done by the universities. Third, regarding the process of performance measurement in the participating organisations, they found it difficult to understand the purpose of the network-level PMS, which caused challenges in the implementation of network level performance measurement.

The reasons for industrial organisations’ participation in collaborations and innovation networks with universities seem linked to intellectual capital and knowledge acquisition although they miss the clear connection between the advantages gained from the development activities and the actual business performance. Despite the apparent importance of measuring the performance of these university–industry innovation networks, the industrial organisations face challenges in recognising the performance measurement practices developed for university-industry collaborations, which is why they face challenges in implementing them. The lack of understanding of the context and the process of the performance measurement also seems to shift the industrial organisations’ focus to the content phase of the performance measurement and to the use of traditional measurements and tools to estimate the advantages gained. However, these traditional tools and measures focused on tangible outputs and were mainly mentioned as related to business performance instead of emphasising intangible aspects and innovation capabilities that were set as parts of the main target of the collaboration. Furthermore, as industrial organisations are unfamiliar with evaluating issues such as innovation capabilities and intellectual capital, they do not know how to measure them more precisely in university-industry collaboration contexts. For this reason, they need more support in implementing performance measurement tools and frameworks to evaluate and manage their innovation and development activities with universities and to develop measurements that will assess intellectual aspects of these activities.

This study’s limitation is that it is based on two different cases of university–industry innovation networks. However, because the study was aimed mainly to increase the empirical understanding of the implementation of performance measurement practices and the challenges in collaboration activities in university–industry innovation networks, the research findings can be utilised by different stakeholder groups interested in the evaluation of university–industry partnerships and innovation activities in general. Further research is suggested to develop both theoretical and empirical knowledge of the implementation practices and challenges of PMSs for innovation activities between universities and industrial organisations. Thus, the results of this study provide both academics and practitioners with valuable information about the implementation of PMSs in practice and the related challenges. Based on this information, academics and practitioners will be able to develop and implement better performance measurement frameworks and tools.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Tero Rantala is a researcher and doctoral student at Lappeenranta University of Technology. His current research focuses on performance management and measurement of university–industry collaborations, including the measurement of innovation activities in private and public sector organizations. In addition, his current research interests involve different areas of performance management in digital business environments and sustainable business contexts.

Juhani Ukko is a senior researcher at Lappeenranta University of Technology, School of Engineering Science. His current research interests involve different areas of performance management and measurement, related to operations management, digital services, innovation and sustainable business.