User-driven innovation: scale development and validation

Abstract

In dynamic business environments entrepreneurs increasingly strive to customise new products/services to displayed and latent user needs. User-driven innovation (U.D.I.) aims to incorporate user needs by giving users an active role in the innovation process. Despite the growing interest of researchers in U.D.I., empirical evidence remains scarce, because of a lack of a psychometrically sound instrument to enhance insight into U.D.I. This paper derives an integrative definition of U.D.I. from different U.D.I. research streams and proposes a model with three distinctive dimensions of U.D.I.: user involvement, searching feedback and design orientation. Three consecutive studies result in a 13-item U.D.I. scale with appropriate reliability, dimensionality, convergent and discriminant validity. Pilot studies include researchers, entrepreneurs and practitioners. The main study comprises data of 357 S.M.E.s. The analyses confirm the multidimensionality of the proposed construct. This study contributes to existing research of U.D.I. in entrepreneurship by addressing the multidimensional nature of U.D.I. with a new research instrument. The proposed U.D.I. scale can be used in future investigations of U.D.I. The construct is informative also for practitioners in introducing U.D.I. to their companies.

Keywords:

• User-driven innovation
• scale validation
• user involvement
• SMEs research
• R&D Management
• small business management

JEL classifications:

• O31
• L26
• C18

1. Introduction

Entrepreneurs in dynamic business environments discover new business opportunities in customising products/services to user needs (Priem, Li, & Carr, 2012) and exploit the benefits of involving users into the innovation process (Smith & Shah, 2013). Meeting user needs is a business imperative, and user-driven innovation (U.D.I.) is an umbrella term for innovation methods which aim to customise new products/services to users’ needs by involving users in the innovation process. U.D.I. improves product quality (Feng, Sun, & Zhang, 2010) and performance (Lau, Tang, & Yam, 2010), contributes to micro innovations (Hyysalo, 2009), reduces the market risk of disruptive innovations (Enkel, Kausch, & Gassmann, 2005), impacts user satisfaction and facilitates a company’s innovation capabilities (Ngo & O’Cass, 2013). The U.D.I. brings to the surface a need for changes in thinking about approaches to value creation from a company-oriented view to a user-oriented view (Lockwood, 2009; Ottosson, 2004). U.D.I. is grounded in the demand side of value creation in recent management and entrepreneurship research, some of which concerns new product/service development (Alam, 2002; Kaulio, 1998; Ottosson, 2004). Another approach studies user entrepreneurship and product/service commercialisation (Shah & Tripsas, 2007). This paper focuses on U.D.I. in new product/service development.

Despite the recognition of the need for empirical investigation (De Moor et al., 2010), U.D.I. discussions are predominantly focused on the different strategies (Buchanan, Abbott, Bentley, Lanceley, & Meyer, 2005; Hjalager & Nordin, 2011; Sandmeier, 2009), case studies (Lettl, Herstatt, & Gemuenden, 2006), consequences and challenges of integrating users into the innovation process (da Mota Pedrosa, 2012; Sandmeier, Morrison, & Gassmann, 2010). The literature offers two streams of thoughts connected with U.D.I. One stream lies in entrepreneurship and innovation literature, with von Hippel’s research of lead users as a source of innovation (Lüthje & Herstatt, 2004; von Hippel, 1986). Another stream lies in marketing literature and predominantly encompasses the service development topic with the concept of customer integration/involvement (Alam, 2002; Chien & Chen, 2010). Multiple perspectives on U.D.I., including research of user involvement, investigating user needs, methods of application of U.D.I., and the role of design in U.D.I. result in lack of clarity of this research area. For instance, entrepreneurship emphasises lead users involvement and user feedback (Blank, 2013; von Hippel, 1986), marketing focuses on customer integration with co-creation (Alam, 2002), innovation literature investigates U.D.I. as a process (da Mota Pedrosa, 2012), and design studies present different methods for user participation in designing new products/services (Dell’Era & Landoni, 2014). Empirical research of U.D.I. remains scarce also because of a lack of reliable and valid tools to thoroughly examine it. Previous work in this field focuses on measuring user involvement (Carbonell, Rodriguez-Escudero, & Pujari, 2009; Chien & Chen, 2010; Feng et al., 2010). While user involvement is an important aspect of U.D.I., it does not include the current trends of design orientation and user feedback in an entrepreneurship stream of literature about U.D.I. and reveals only a partial picture of U.D.I. Moreover, not all of the scales in the literature have been validated. Since U.D.I. has been found to have an important effect on the quality of new products/services (Feng et al., 2010; Lau, 2011), it is of benefit to develop a validated measure for organisations.

To address the identified research gap, this study first aims to deliver a theoretically justified, reliable and valid measure of U.D.I. by reviewing the growing research in this interdisciplinary field, and considering the latest theoretical improvements of the concept. On this basis we propose an integrative definition of U.D.I., and posit U.D.I. as consisting of three distinctive components: user involvement in the innovation process, searching for feedback from users and a design orientation toward developing desirable user experiences. Second, a scale is developed that will serve researchers in investigating diverse aspects of U.D.I., including antecedents, consequences, mediating and moderating effects.

By doing so, this paper will make the following main contributions to the literature. First, we clarify the U.D.I. construct and deliver a valid scale to measure it. Second, by proposing two additional dimensions, we extent current research on user involvement. Third, by proposing an integrative definition of U.D.I., we attempt to integrate the parallel research stream of entrepreneurship and innovation to form a solid ground for further investigations.

This article reports on three studies conducted to develop and validate the U.D.I. scale. Study 1 focuses on generating an item pool and assessment of their face validity with experts’ help. Study 2 reports on item purification, along with initial reliability data and factor structures. Study 3 uses a sample of entrepreneurs to examine convergent and discriminant validity of the U.D.I. scale.

2. User-driven innovation

Research on U.D.I. was fostered by von Hippel (1986), who considered lead users to be an important source of innovation. Later, the U.D.I. field diversified in different directions. One stream of research is predominantly focused on U.D.I. as researching and understanding user needs, both displayed and latent (Christiansson et al., 2008; Hjalager & Nordin, 2011; Rosted, 2005). Another stream concentrated on U.D.I. as involving users as active participants in all phases of the innovation process, for instance in idea generation, prototyping, product/service conceptualisation and commercialisation (Alam, 2002; Grunert et al., 2008; Kaulio, 1998). Both streams of literature are reflected through different definitions of U.D.I. (Table 1). For instance, Grunert et al.’s (2010) and Christiansson et al.’s (2008) definitions highlight researching user needs as a central characteristic of U.D.I. Other definitions, in addition to researching user needs, reflect the active role of users in the innovation process (e.g. Hjalager & Nordin, 2011; Wise & Hogenhaven, 2008). More recently, researchers have also studied U.D.I. with regards to developing intuitive and human-centred design, demonstrated in different approaches, such as design thinking, participatory design and human/user-oriented/centred-design (Beckman & Barry, 2007; Venkatesh, Digerfeldt-Månsson, Brunel, & Chen, 2012; Veryzer & Borja de Mozota, 2005). With the latest trends on lean start-up and customer development (Blank, 2013; Ries, 2011), continuous user feedback has also become central to the process of entrepreneurial innovation.

Table 1. Definitions of U.D.I.

Definition	Focus	Source
U.D.I. is the process of tapping users’ knowledge in order to develop new products, services and concepts. A U.D.I. process is based on an understanding of true user needs and a more systematic involvement of users.	User needs and users as contributors	Wise and Hogenhaven (2008, p. 21)
U.D.I. is a systematic approach to develop new products and services, building on investigation or adoption of users’ life, identity, praxis and needs, including unrevealed needs.	User needs	Christiansson et al. (2008, p. 250)
U.D.I. is characterised in three dimensions: customer focus, skills for analysing and assessing customer needs, methods applied in conducting user surveys.	U.D.I. as a process	Rosted (2005, p. 56)
U.D.I. is the phenomenon by which new products, services, concepts, processes, distribution systems, marketing methods, etc. are inspired by or are the results of needs, ideas and opinions derived from external purchasers or users. U.D.I. involves existing and/or potential users and the processes rely on systematic activities that search for, acknowledge, tap and understand the users’ explicit, as well as implicit, knowledge and ideas. Methods in U.D.I. span from superficial observations, to consultations and intensive involvement of the users in co-creation processes.	Users as contributors	Hjalager and Nordin (2011, p. 290)
Consumer-driven innovation is a process towards the development of a new product or service in which an integrated analysis and understanding of the consumers’ wants, needs and preference formation play a key role.	User needs	Grunert et al. (2010, p. 4)

Considering these developments in the literature, we conceptualise U.D.I. with three complimentary dimensions: (1) user involvement in the innovation process and in researching and understanding their needs, (2) searching for feedback from users in a continuous manner through prototype testing, pilot sales, testing product ideas among users, etc. and (3) design orientation as adapting the user interface of the product/service to the users’ abilities, needs and desires. Those three dimensions of U.D.I. were evident also from the interviews with nine entrepreneurs and researchers. However, the interviews’ analysis exceeds the aim of the present article; thus, we will describe the three dimensions only with the findings from the literature. User involvement dimension reflects both streams of literature mentioned above, along with the definitions provided in Table 1. The dimension refers to the breadth and depth of the users’ contribution in the innovation process (Fang, Palmatier, & Evans, 2008). The second dimension, searching for feedback, is derived from the latest advancements in the entrepreneurship literature where user feedback plays a crucial role in entrepreneurial innovation (Blank, 2013). Entrepreneurial innovation is less systematic comparing to innovation process in big companies (Garud, Gehman, & Giuliani, 2014; Rosenbusch, Brinckmann, & Bausch, 2011). Continuous feedback by users may represent a common characteristic of a less systematic entrepreneurial innovation process. Searching for feedback denotes the deliberate and continuous user testing of product/service concepts, prototypes and business models. Companies differ with regards to the number of testing activities and in the starting stage of searching for feedback. Similarly, also the third dimension, design orientation, is derived from entrepreneurship literature in which a design thinking approach (Brown, 2008) uses designer’s tools in new product/service/business development. Design orientation signifies a firm’s orientation to achieve user friendliness by capturing users’ functional and psychological needs; it also encompasses intuitive functionality, aesthetic appearance and uniqueness on the market with a distinctive brand. In the following section, each of the proposed dimensions of U.D.I. is developed.

2.1. User involvement

User involvement impacts product quality, delivery reliability, process flexibility, customer service and user satisfaction (Feng et al., 2010; Kujala, 2003). As a source of knowledge, users can also improve the effectiveness of the innovation process (Fang et al., 2008). Users can have three roles in new product development (N.P.D.): the user as a resource of ideas, the user as co-creator in a design and development task and the user as a participant in product/service testing and support (Nambisan, 2002). The most frequent form of user involvement is in researching users in order to gain an in-depth understanding of their needs, both obvious and latent. Exploration of users’ needs includes a user’s broader life, identity, value system and desired holistic experience with the product or service (Hjalager & Nordin, 2011). In addition to researching users’ needs, some companies also involve users in idea generation, prototyping, product/service conceptualisation, product/service support and continuous improvement (Nambisan, 2002). In U.D.I., user involvement is active rather than passive. Users participate in some or all phases of product/service development; innovation principles are open (Lichtenthaler, 2011); and companies motivate users to participate in the process (Nambisan, 2002). Companies with more intensive user involvement will easily absorb their knowledge, which can be beneficial in enhanced customer value creation (Feng et al., 2010). User involvement enables companies to integrate outward and inward knowledge transfers (Lichtenthaler, 2011). Bisgaard and Hogenhaven (2010) provide a framework for clarification of U.D.I. from a company perspective. The framework is based on two dimensions. The first dimension divides U.D.I. methods into two groups according to direct or indirect involvement of users in the innovation process. The second dimension refers to the nature of user needs; they can either be acknowledged or unacknowledged. The framework results in four generic categories of U.D.I. methods: user exploration, user participation, user innovation and user tests.

In summary, user involvement encompasses two aspects: researching users’ needs and users as active actors in the innovation process. User involvement is related to the second dimension, i.e., searching feedback. This is evident from the nature of user involvement methods. Despite the methods being focused on users’ contribution to new product/service development, the process also leads to feedback, because users can hardly contribute without expressing their own attitudes and perceptions. However, the distinction between user involvement dimension and searching feedback lies in the type of user contribution. User involvement dimension includes users’ contributions in terms of new ideas, functions and features of new product/service. Searching for feedback dimension on the other hand refers to providing judgement about an input given by the company. Searching for feedback thus refers to testing prototypes of new product/service, business model, brand, etc.

2.2. Searching feedback

Learning theories suggest that frequent, immediate and regular feedback enhance the rapidity and quality of the learning process (Kolb, 1984). Searching for feedback focuses on the deliberate searching for new information about various aspects of product/service development or the business model. Users’ feedback helps companies to fulfil users’ expectations and latent needs. Feedback reveals users’ functional, symbolic and experiential needs. These three aspects of users’ needs might guide companies in searching for feedback in a holistic way. Feedback about product’s/service’s characteristics are not enough, however, because functionalities refer only to user’s functional needs. Psychological needs refer to users’ emotional, symbolic and experiential needs (Verganti, 2008). Feedback on how a product/service meets user’s psychological needs is a crucial part of searching for the feedback dimension in U.D.I. Testing in technology-driven innovation is predominantly focused on usability tests (De Moor et al., 2010). Searching for feedback in U.D.I. also comprises testing the soft aspects of a new product/service. Via the deliberate gathering of feedback, companies can test their business model, adapt functionality of a new product/service, change user interfaces and customise the product/service to meet different users’ preferences.

Feedback assists companies in obtaining an informed picture before and after the product/service launch (De Moor et al., 2010). User feedback allows companies to make judgements about the innovation process. Early feedback reveals whether a new product/service meets only articulated or also unarticulated user needs. Feedback is related to both functional and psychological needs. The distinction between these two needs is important, because it helps a company to react accordingly to user feedback. If feedback is related to the usability of a new product/service, further innovation effort will be focused on functionality. If feedback is related to psychological needs, supplementary innovation effort will be focused on the brand development or/and aesthetic appearance of a new product/service. Via pilot sales, which are also a form of feedback, companies obtain information about the market potential of new products/services and about the viability of their business models.

Companies can gather feedback in all phases of product/service development. Triggering feedback includes, but is not limited to, testing the initial product/service idea among potential users, testing functionality and design of mock-ups, small-scale pilot sales, rapid prototyping, testing of brand identity, feedback about the business model, and role playing. User feedback can change not only product’s/service’s functionality and appearance, but can also effect business model development and the innovation process itself. As searching for feedback includes users and creates dialogues with users, this dimension is connected with user involvement. By providing feedback, users automatically become a part of the innovation process. In this aspect searching for feedback dimension is related to user involvement dimension. Users provide information, ideas, insights, reflection, perception and experiential knowledge about the product or business model. Users’ insights are subjective and usually not burdened with technical knowledge about the product (Nambisan, 2002). Users’ feedback thus originates from their experience with products or services and from the aesthetic appearance of a product/service.

2.3. Design orientation

Design orientation in U.D.I. concentrates on developing user-friendly products/services with intuitive user interfaces. Design orientation encompasses the entire product/service appearance, including the brand. Lack of attention to the design aspects of the U.D.I. process may lead to implementation constraints (Kujala, 2003) and result in unrealistic imaginary products/services, which is a common critique of the U.D.I. approach (Enkel et al., 2005). As design also adds value to a new product, design management literature considers design to be a source of competitive advantage (D’Ippolito, 2014; Johansson-Sköldberg, Woodilla, & Çetinkaya, 2013; Verganti, 2008). Different approaches to the design in terms of U.D.I. have arisen lately, for instance participatory design, contextual design, emphatic design, design thinking and human/user-oriented/centred design (Kujala, 2003; Veryzer & Borja de Mozota, 2005). Researchers agree on the challenges of involving users in the design process, for instance, decreased cost and time efficiency, the need to educate users about design, a need for intensive communication with users and a lack of suitable tools for involving users (Enkel et al., 2005; Kujala, 2003). Despite the challenges, researchers conclude that design orientation results in a better fit of a new product/service to user needs and interaction preferences (Venkatesh et al., 2012; Veryzer & Borja de Mozota, 2005).

We argue design orientation to be a separate dimension, since some companies understand design only as the aesthetic appearance of a product, while others comprehend the crucial role of design in assuring desirable user experiences (D’Ippolito, 2014). Design in U.D.I. focuses on the user experience in terms of both functional and symbolic features. If a company understands design to be an antecedent of user experience, it then concentrates its innovation effort on ensuring a valuable user experience, which in turn impacts user satisfaction (Yoon, 2010), future use (Castañeda, Muñoz-Leiva, & Luque, 2007) and recommendations to other potential users (Santos, Mazzone, Aguilar, & Boticario, 2012).

Deriving from this development of three dimensions, we define U.D.I. as an approach to new product/service development, which aims to provide desirable user experience by involving users in the innovation process, continuous searching of feedback and creating an intuitive design. Such a conceptualisation comprises all three aspects of U.D.I. and complements the existing definitions by including the role of the design and user feedback as an integral part of U.D.I. The proposed definition covers the message of existing definitions of U.D.I. regarding the aspects of researching user needs and the active role of users in the innovation process (Table 1). Involving users in the innovation process in the proposed definition relates to both researching user needs and their active role in the process. The active role of users from existing definitions is partially covered also in the second aspect of the proposed definition, i.e. continuous searching of feedback. At the same time this part of the definition incorporates contemporary entrepreneurship literature, with the focus on user feedback in entrepreneurial innovation (Blank, 2013). In addition, the proposed definition upgrades the existing definitions of U.D.I. by adding the third aspect, i.e., creating an intuitive design. The third aspect is also justified in the latest entrepreneurship and design literature. Design thinking and participatory design in new product/service development aim to develop intuitive product/service appearance, which is driven by user needs. All three proposed dimensions derive from a demand-side of value creation and consider the user as a central part of this innovation strategy.

3. Scale development

The U.D.I. scale was developed across three studies. The first study concerns item generation and assesses their face validity with a sample of experts. The second study presents an item purification procedure with initial reliability analysis and factor structure. The third study offers evidence of convergent and discriminant validities of the U.D.I. scale with a sample of entrepreneurs.

3.1. Study 1: Item generation

A process of measuring development recommended by Churchill (1979) was employed. In conducting the analysis, we also considered procedures in other validation studies: Rossiter (2008), Reid and Roberts (2011), Tang, Kacmar, and Busenitz (2012), Cardon, Gregoire, Stevens, and Patel (2013) and Yi and Gong (2013). This research generated a broader pool of 115 items from previous exploratory pilot studies, literature reviews and exploratory interviews with nine entrepreneurs and researchers. The items from the literature were based on the earlier work by Deshpandé, Farley, and Webster (1993), Ittner and Larcker (1997), Song and Parry (1997), Narver, Slater, and MacLachlan (2004), Rosted (2005), Ramani and Kumar (2008), Brown and Katz (2009), Feng et al. (2010) and Karpen, Bove, and Lukas (2012).

A screening of the initial pool of items followed. Doubled, double-barrelled, ambiguous and overly similar items were eliminated, which resulted in 64 items. Via a translation-back translation procedure (Brislin, 1970) those items were adapted from English to Slovenian. In order to consider the object element, attribute element and rater entity element in assessing the content validity (Rossiter, 2008), 16 researchers, practitioners and Ph.D. students then evaluated these 64 items for representativeness. Researchers and Ph.D. students had experience in psychometrics (four of them had a background in statistics, four in business administration and two in psychology), while practitioners had experience in involving users in new product/service development. The evaluators were presented with a description of the construct and each dimension, respectively. Then they rated the representativeness of each item on a 5-point agreement scale (1 = ‘strongly disagree’, 5 = ‘strongly agree’). They also had an opportunity to suggest an alternative item. Only items evaluated as clearly representative by at least 10 of the 16 evaluators were retained (Tian, Bearden, & Hunter, 2001). A total of 33 items were eliminated for failing to provide face validity, leaving 31 items.

3.2. Study 2: Item purification

The purpose of this research was to determine the factor structure of U.D.I. and to purify the scale based on the psychometric properties. We collected survey data from 129 practitioners who participate in part-time study for a business degree. Other studies also use convenient samples for preliminary research in the development of scales (e.g. Tang et al., 2012; Yi & Gong, 2013); 31% were male and, on average, they had 8.1 years of working experience. A 5-point Likert scale was used. The respondents were asked to have in mind the practices in their companies when reading the items. They also had the possibility to write comments about the items.

First, corrected item-to-total correlations and item correlations for each set of items representing components of U.D.I. were examined. Seven items with corrected item-to-total correlations below 0.50 and item correlations below 0.30 were eliminated from further analysis (Hair, Black, Babin, & Anderson, 2009). The remaining 24 items were evaluated via exploratory factor analysis, i.e., principal components with varimax rotation. With 5.4:1, our observations to respondent ratio exceeds the 5:1 rule-of-thumb ratio (Hair et al., 2009). The K.M.O. measure of sampling adequacy was 0.84, and a significant χ² value for the Bartlett’s test of sphericity (χ² = 596.58, p < 0.001) indicated the appropriateness of data for factor analysis. Next, nine items with factor loadings below 0.50, cross-loadings above 0.40 and commonalities below 0.30 were eliminated (Hair et al., 2009). With use of exploratory factor analysis, the scale is purified to 15 items that represent three U.D.I. dimensions with eigenvalues above 1. In total, they explained 53.91% of the variance. Cronbach’s alpha reliability coefficients exceeded the 0.70 cut-off value (Nunnally & Bernstein, 1994). The alpha values are 0.79, 0.78 and 0.70 for user involvement, searching feedback and design orientation dimensions, respectively. The final items retained for the main study and confirmatory factor analysis are enumerated in Table 2.

Table 2. Results of exploratory factor analysis (study 3, N = 357).

Items	Factor 1	Factor 2	Factor 3
User involvement
U.I.1. We actively encourage users to present to us their ideas on improving our products or services, as well as their ideas on new ones.	0.70	0.27	0.05
U.I.2. We are including the users in all phases of the innovation process.	0.80	0.12	0.05
U.I.3. Users are a part of a developmental team for new products/services.	0.77	0.13	0.03
U.I.4. We conduct personal interviews with the users when developing new products or services.	0.77	0.14	0.16
U.I.5. When developing products or services, we cooperate with leading (advanced) users.	0.73	0.33	−0.02
U.I.6. We encourage users to share their experiences and stories about their habits, product usage, shopping decisions, etc.	0.69	0.26	0.19
Searching feedback
S.F.1. We regularly check our ideas for new products or services with our users.	0.38	0.68	0.13
S.F.2. We continuously monitor the development process to check how well the new product or service is adjusted to the needs of different users.	0.42	0.62	0.25
S.F.3. We test prototype among our users several times.	0.33	0.75	0.06
S.F.4. We organise pilot sales before mass sales.	0.06	0.67	0.18
Design orientation
D.O.1. In the process of developing new products or services, we aim to develop such properties that make the products easy to use, regardless of the users’ demands.	0.22	0.04	0.77
D.O.2. New products or services are designed so that their use is intuitive (i.e., the user does not need instructions but only follows the design).	0.10	0.12	0.82
DO3. The visual image is our way of ensuring the users like our products or services.	−0.09	0.34	0.71
Eigenvalue	5.23	1.78	1.01
Percentage of variance explained	29.36	17.62	15.02
Cumulative percentage of variance explained	29.36	46.98	62.00
Cronbach’s α	0.87	0.76	0.70

Source: Own calculations.

4. Study 3: Main study with SMEs

4.1. Participants and procedures

The retained 15 items were examined for construct validity. Via an on-line survey, we collected data from young companies (0–15 years old) operating in multiple industries in Slovenia. A random sample of 4267 companies that have at least one employee was obtained from a Slovenian national database of companies. The first invitations were sent in March 2014, followed by three reminders sent to increase the response rate. A total of 357 complete surveys yielded a response rate of 10.6%. In calculating the response rate, any company that was not reachable due to a lack of access to their director’s email address were dropped. Regarding demographics, 72.4% of respondents were male; 14.9% of respondents had a high school degree or less, 23.5% had a college degree, 42.0% had a bachelor’s degree and 19.6% had a graduate degree. The average working experience was 17.7 years (S.D. = 9.5). T-tests did not reveal any significant differences in dependent and independent variables between early and late respondents, nor any significant differences in a firm’s characteristics between respondents and non-respondents in terms of sales, number of employees and age. Each survey item required a response based on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree).

4.2. Reliability assessment and construct validation

4.2.1. Dimensionality and reliability

Principal components and confirmatory factor analyses were performed. One item was dropped due to item correlations below 0.30 (Hair et al., 2009). After principal components analysis, one additional item was eliminated, because it did not load on its intended factor. After the elimination, the analysis was repeated with 13 items. According to the K.M.O. measure (0.89) and Bartlett’s test of sphericity (χ² = 1858.82, p < 0.001), the data were suitable for analysis. The principal component factor analysis requested three factors with a varimax rotation was conducted; three factors explained 61.99% of the total variance. Table 2 presents the rotated factor solution. Item loadings are greater than 0.40 on a single factor with minimal cross loadings on user involvement and searching feedback factors, which are theoretically related as presented in the conceptualisation of the U.D.I. construct. The reliability estimates (Cronbach’s alpha) exceeded the 0.70 cut-off value (Nunnally & Bernstein, 1994) and the average variance extracted is greater than 0.50 (Bagozzi & Yi, 1988). The proposed three-dimensional conceptualisation of U.D.I. is reflected in the factor structure; therefore, we proceeded with confirmatory factor analysis to confirm the dimensionality.

Confirmatory factor analysis using Lisrel 8.5 estimated two alternative models, i.e., the proposed three-factor model and one-factor model. The results confirmed the dimensionality of the 13-item, three-factor scale (χ²₍₆₂₎ = 135.10, p < 0.001, G.F.I. = 0.98, C.F.I. = 0.96, N.N.F.I. = 0.95, R.M.S.E.A. = 0.06). The fit indices for the one-factor model are: χ²₍₆₅₎ = 244.43, p < 0.001, G.F.I. = 0.96, C.F.I. = 0.91, N.N.F.I. = 0.89, R.M.S.E.A. = 0.08. The fit of the three-factor model is significantly better than the one-factor model (χ²_(difference) = 109.33, df = 3, p < 0.001). Table 3 displays outputs of the confirmatory factor analysis.

Table 3. Outputs of confirmatory factor analysis (study 3, N = 357).

Items	Standardised loading	T-value	Reliability (C.R., A.V.E.)
User involvement			C.R. = 0.95
UI1	0.82	31.43**	A.V.E. = 0.76
UI2	0.85	40.60**
UI3	0.87	36.41**
UI4	0.91	47.98**
UI5	0.90	42.54**
UI6	0.88	43.53**
Searching feedback			C.R. = 0.89
SF1	0.86	35.06**	A.V.E. = 0.67
SF2	0.90	37.39**
SF3	0.86	38.80**
SF4	0.62	17.18**
Design orientation			C.R. = 0.80
DO1	.073	18.01**	A.V.E. = 0.57
DO2	0.84	28.18**
DO3	0.69	15.77**

Notes: C.R., composite reliability; A.V.E., average variance extracted.

^** p < 0.01

Source: Own calculations.

4.2.2. Convergent and discriminant validity

A procedure suggested by Fornell and Larcker (1981) was employed in order to calculate composite reliabilities and the average variance extracted. Composite reliabilities exceeded the 0.70 threshold (user involvement: 0.95, searching feedback: 0.89, design orientation: 0.80) and the average variance extracted for each dimension was greater than 0.50 (user involvement: 0.76, searching feedback: 0.67, design orientation: 0.57), thereby indicating convergent validity (Bagozzi & Yi, 1988). Moreover, composite reliabilities greater than average variance extracted showed convergent validity (Hair et al., 2009).

To assess discriminant validity, a Chi-square difference test proposed by Bagozzi and Yi (1988) was performed. For each pair of factors, the constrained model with a free model was compared; specifically, the correlations between two factors in constrained model were set to equal one for each pair. Chi-square differences were significant in all cases (χ²_(U.I.-S.F.)(1) = 62.32, χ²_(U.I.-D.O.)(1) = 96.97, χ²_(D.O.-S.F.)(1) = 29.35), which indicated discriminant validity. The three dimensions of U.D.I. are distinctive and measure different aspects of U.D.I.

Because of the high correlation among the three dimensions we performed a second-order confirmatory analysis if there might be a higher order factor construct—U.D.I. A second order confirmatory factory analysis reveals that the three dimensions reflect a higher-order construct of U.D.I. with the following factor loadings: user involvement: 0.89 (t = 15.77), searching feedback: 1.03 (t = 26.22), design orientation: 0.74 (t = 15.77). The second-order measurement model fits the data relatively well (χ²₍₆₂₎ = 137.14, p < 0.001, G.F.I. = 0.98, R.M.S.E.A. = 0.06, S.R.M.R. = 0.20, T.L.I. = 0.96, C.F.I. = 0.97). The S.R.M.R. fit index is above the recommended value of 0.08 (Hair et al., 2009). However, in their simulation study, Hu and Bentler (1999) showed that a cut-off value of 0.96 for T.L.I., BL89, R.N.I., Gamma Hat and C.F.I. in combination with S.R.M.R. above 0.06 resulted in the lowest number of Type I and Type II errors. As both T.L.I. and C.F.I. meet the 0.96 criteria, we can accept the proposed second-order measurement model.

5. Discussion

This study contributes to the theory and practice in several ways. At a theoretical level, we contribute an integrative conceptualisation of U.D.I. with three relevant dimensions. We have integrated different streams in the interdisciplinary U.D.I. literature and conceptualise U.D.I. as an approach to new product/service development that aims to provide desirable user experience by involving users in the innovation process, continuously searching for feedback and creating intuitive design. The proposed definition complements the existing definitions which are focused on researching user needs (Christiansson et al., 2008; Grunert et al., 2010) and the active role of users in the innovation process (Hjalager & Nordin, 2011; Wise & Hogenhaven, 2008). Besides those two aspects, the proposed definition reflects two additional aspects from the entrepreneurship literature, i.e. user feedback (Blank, 2013) and design orientation (Brown, 2008). By proposing an integral definition of U.D.I. we contributed to the theory of users as sources of innovation in the innovation process.

At a methodological and empirical level, we contribute a valid measure for studying antecedents, consequences, mediators and moderators of U.D.I. Through qualitative and quantitative empirical research, this study has developed and validated the U.D.I. scale. U.D.I. comprises three dimensions: user involvement, searching feedback and design orientation. All three dimensions of U.D.I. are derived from various literature streams and our own interviews. These dimensions give entrepreneurs a foundation for a holistic approach to U.D.I. in order to benefit from users’ potential. Three dimensions are theoretically significant: without searching for feedback and design orientation, the concept of U.D.I. is limited to researching user needs and obtaining ideas for new products/services from users. Such an approach does not sufficiently address the challenges connected with engaging users in the innovation process. Searching for feedback focuses on testing market potential, prototypes and business models. Design orientation integrates product appearance, along with brand development. Empirical evidence suggests that the U.D.I. scale demonstrates internal consistency reliability, convergent and discriminant validity. The scale seems to appear as a comprehensive, psychometrically sound and operationally valid measure of U.D.I. The U.D.I. scale can be part of a different overall research design, because 13 items is a manageable number to include in a longer questionnaire.

5.1. Limitations and future research

The limitations of this study include its cultural generalisability, as the Slovenian national culture is individualistic (Svetlik, 2000). Future research should investigate the psychometric characteristics of the U.D.I. scale in collectivistic countries such as China or India. Cross-cultural stability of the instrument need to examined by assessing if the factor loadings are equal across different samples. The initial model, whereby the factor pattern is set to be invariant across the samples, needs to fit to covariance matrices of the respective samples. Having a cross-cultural validated instrument, we can assess similarities and differences of the U.D.I. in different countries. The findings based on a Slovenian sample are not generalizable to firms in other countries. However, the proposed research instrument may serve as a starting point for validation in other countries. Additional items may be added and/or current items may be adapted in order to comprise different contingencies that affect U.D.I. in other countries. Furthermore, this study is limited to entrepreneurs’ views. Future studies should also include the view of employees in companies and employ a multilevel approach. Additional methods with independent observers would reveal greater insight into the actual degree of U.D.I. in entrepreneurial firms.

6. Conclusions

By providing a scale of U.D.I., we considered an appeal for stronger interdisciplinary research between the fields of entrepreneurship and other disciplines such as marketing or innovation (Ireland & Webb, 2007). Due to the lack of a valid U.D.I. scale, researchers had to depend on broader measures of user involvement (Carbonell et al., 2009; Chien & Chen, 2010; Feng et al., 2010; Ngo & O’Cass, 2013), which do not comprise distinctive aspects of U.D.I. Having a reliable measure of underlying U.D.I. dimensions, researchers will be able to investigate the nature of U.D.I. and explain the relation of U.D.I. to major concepts in entrepreneurship. The proposed dimensions (user involvement, searching feedback and design orientation) have proved to be distinctive aspects of U.D.I. We believe the three dimensions are relevant for the theory of U.D.I., because they are both theory-driven and derived from several consecutive empirical procedures (i.e., pilot studies, interviews and evaluators’ ratings of the wider set of items). Nevertheless, the validation study needs to be replicated in other countries in order to use the U.D.I. scale within cross-cultural settings.

As innovation is context-dependent (Rosenbusch et al., 2011), research in the future has to focus on exploring the settings that can most benefit from U.D.I. For instance, are firms that develop their brands along with their U.D.I. efforts on average more successful? Do marketing capabilities as more general marketing concepts influence the relationship between U.D.I. and firm performance? U.D.I. itself may be a mediator (Ngo & O’Cass, 2013). However, researchers have to explain which cognitive processes drive U.D.I. Does entrepreneurial alertness enable entrepreneurs to see business opportunities in U.D.I.? Future research needs to clarify the process of how U.D.I. contributes to competitive advantage and dynamic capabilities. The proposed U.D.I. scale will enable researchers to explore this process with quantitative research designs.

This study also has practical implications. Entrepreneurs can use this scale as a diagnostic tool for assessing the current state of U.D.I. in the firm. Based on the results, entrepreneurs can develop training programmes to adopt different methods for user involvement (e.g. ethnographic research, living labs), searching feedback (e.g. rapid prototyping, pilot sales) or design orientation (e.g. design thinking, emphatic design).

Disclosure statement

No potential conflict of interest was reported by the authors.