What an original app! : a study on the novelty of software products

ABSTRACT

Innovation in software is important to both members of the public and the software industry, and involves developing software that is both useful and novel. While novelty has attracted significant interest in relation to physical products, it is not so well studied in the context of software products. In this paper, we aim to help develop our understanding of novelty in software by investigating which kinds of software products are seen as more or less novel; what factors may contribute most to perceptions of novelty in software, and whether members of the public and those involved in software development think in the same ways about these things. Through a study consisting of 3 stages, we identify examples of software products that are seen as original, at time of writing, by both software specialists and members of the public, and also two key factors – use of software in a new context, and use of a new underlying technology, which appear to contribute strongly to perceptions of novelty in software. We briefly discuss the implications of our findings and directions for further work.

KEYWORDS:

• Software novelty
• software products
• creativity

1. Introduction

There is no doubt that the software industry is shaping our day-to-day lives. From entertainment to health applications, we rely on software products to make our lives easier. As we write this article – in the middle of the Covid-19 health emergency – the impact of software on our leisure and productive activities is even more pronounced: worldwide, education institutions have been forced to move the learning experiences of millions of students to the online space, home working has been the norm for a significant percentage of the population, online shopping has exploded and many face-to-face social activities are now carried out using video conference tools as friends and families refrain from physical meetings to break the virus transmission chain. This emergency has accelerated a tendency that was already in place: we increasingly depend on software products to carry out our daily activities.

Even before the drastic changes imposed on society by the pandemic, the software industry had become very competitive and was in constant search of creative ideas that pushed the boundaries of innovation. Software innovation is beneficial to the public as high value software products have the potential of improving their daily lives. It is also important to the software industry that aims to attract the attention of the public to stay ahead of the competition and develop their market share. One way of attracting the public’s attention is through the novelty of products brought to market.

Novelty – along with usefulness – is one of the main traits of creative products (Runco & Charles, 1993). It is a complex concept (Brown, 2021; Fiorineschi & Rotini, 2021) related to aspects including being new, or recent in time (MacCrimmon & Wagner, 1994); unusual, or statistically infrequent, (Dierich et al., 2015); or original, meaning different from others, either at the historical, societal or psychological level (Boden, 2004; Shah & Vargas-Hernandez, 2003).

In this paper, we build on knowledge already available about what factors impact the novelty of physical products, to investigate what constitutes novelty in software, and how increasing our understanding in this area might inform those in the software industry seeking to innovate with software. In particular, we investigate 2 research questions related to the concept of novelty, as it applies to software:

- What factors may contribute to the perception of novelty in software?

- Do members of the public and those with a specialist interest in software agree on which software products they perceive as novel, and the factors that might contribute to this novelty?

By answering these two research questions we expect to help improve software design practice and innovation. Knowing what factors make a software product novel will allow software designers to evaluate the novelty of their ideas in a standard and reliable way. As a result, the search for novel software solutions will be more efficient and effective.

The paper is structured as follows: Section 2 reviews previous work on novelty of physical products and software products. Section 3 presents the methods used to answer the two research questions presented above. Section 4 discusses our results. Finally, Section 5 discusses the main conclusions of this work and potential lines of further research.

2. Previous work

2.1. Factors impacting the novelty of physical products

Novelty can be evaluated based on the subjective opinion of domain experts – as proposed by Amabile’s popular consensual assessment technique (Amabile, 1982) – or based on the evaluation of specific factors that contribute to the novelty of a product. Since one of the research questions guiding our work relates to the identification of specific factors that affect the perception of novelty of software products, we focus here on the latter approach. We note that for those focusing on a factor-based approach to the evaluation of novelty, there are two variants in the methods adopted. The first identifies generic factors to be used to evaluate the novelty of a broad range of design solutions, such as ‘working principle’ (Jagtap, 2019, 2016; Lopez-Mesa & Vidal, 2006; Sarkar & Chakrabarti, 2011; Toh & Miller, 2014). The second variant identifies factors that only apply to the solutions for a specific design problem (Oman et al., 2013; Shah et al., 2003). For example, the order used to sort material when designing an automatic recycler (Oman et al., 2013). Given our aim of identifying factors that contribute to the perception of novelty of different software products, the research presented in this paper builds on the first variant.

In his seminal paper Rhodes states that ‘when an idea becomes embodied into tangible form’ we talk about a product (Rhodes, 1961). This notion of a product being a tangible object has dominated the study of product novelty in design. It is not surprising, then, that most factors identified as impacting the novelty of products are those related to their physical nature. These factors can be summarized as follows:

2.1.1. Physical/Working principle

the physical law that governs the product’s behavior, and how it is applied to the product design (Jagtap, 2016, 2019; Sarkar & Chakrabarti, 2011)

2.1.2. Functionality

the function performed by the product (Jagtap, 2016, 2019; Sarkar & Chakrabarti, 2011)

2.1.3. Physical attributes

tangible aspects such as shape, material, color, components, smell, texture, consistency, size, weight, etc. (Jagtap, 2016, 2019; Lopez-Mesa & Vidal, 2006; Sarkar & Chakrabarti, 2011; Toh & Miller, 2014)

2.1.4. Interaction

the behavior of the product, understood as how it reacts to different user inputs (Jagtap, 2016, 2019; Sarkar & Chakrabarti, 2011; Toh & Miller, 2014)

Sarkar and Chakrabarti (2011) proposed a first scale to evaluate the level of novelty of a product based on different combinations of the previous factors. Jagtap (2016, 2019) proposed a modification to that scale to incorporate more levels and different combinations of factors.

2.2. Factors impacting the novelty of software products

Research on evaluating the novelty or creativity of software products has adopted a range of different approaches, as follows. One study aimed to evaluate the level of creativity based solely on the textual description of software products (D.L. Amoroso & Couger, 1995; D. L. Amoroso & Eriksson, 2000). Here, it was found that the degree of creativity was a function of the frequency of appearance of creativity-related keywords in the descriptions and the rating given by humans to the descriptions. Zeng et al. (2009, 2012) studied the creativity of websites, and found that novelty was one of the main drivers for creativity, contributing to the intention of the user to keep browsing the web site, revisiting it in the future and purchasing from it.

A literature review on creativity and information systems published in 2013 (Müller & Ulrich, 2013) studied 88 papers, but only 5 of them referred in any way to the novelty evaluation of ideas or products, and not specifically to software products. A couple of years later, Kuzmickaja et al. (2015) evaluated the novelty of ideas for mobile services in terms of two aspects: originality and paradigm shift, as proposed by Dean et al. (2006). However, no evaluation of actual applications or specific factors affecting the novelty of the services was carried out in this study. In 2017, Kruger et al. aimed to evaluate the creativity of the final year projects of information systems students using the Creative Product Assessment Model (CPAM) proposed by Besemer and Treffinger (1981). CPAM is a general technique and as such, it does not identify specific traits that contribute to the novelty of software products.

Focusing on factor-based approaches to evaluating the novelty of software products, we see that a first important effort was presented by Couger and Dengate (1992), who identified specific factors impacting the usefulness and novelty of software, and used these to evaluate the creativity of 6 software products considered innovative at the time. The factors considered to contribute to novelty, identified by the authors based on their own experience, can be summarized as relating to: use of a new technology to computerize something for the first time; use of a new algorithm; use of a new approach/method, and effective technology transfer.

Over 20 years later, Rose and Furneaux (2016) surveyed ninety three papers and showed that new software functionality was identified as the most common factor for software innovation. Mohanani et al. (2017) also carried out an extensive literature review on, among other aspects, how creativity is measured in software engineering. Eleven out of eighty four papers analyzed reported on evaluating the creativity of software products. Of those, two used domain expert opinions, and two used a general test not developed specifically for software products: Williams´ creativity assessment test (Cooper, 1991). The remaining 7 used objective measures such as the number of added new features, number of multiple ideas generated, the ratio of number of enhancements to the number of bugs resolved and the fluctuations in quality as code evolves. All these aspects, with the exception of the number of new features, refer to the process of idea generation or code development, rather than the resulting software products.

Finally, in 2020, we presented preliminary results on identifying software products that were perceived as novel by workers from the software industry, as well as an initial list of factors that contributed to this perception (Beghelli & Jones, 2020). The factors identified were: Technology Infrastructure, New Functionality, Extra Functionality, User Interface, User Interaction and User Experience.

In this paper, we extend the research reported in (Beghelli & Jones, 2020) by refining our understanding of the factors contributing to novelty of software products and studying whether members of the public and those with a specialist interest in software agree on what software products are perceived as novel and the factors that might contribute to this novelty.

3. The perception of novelty in software products: methods

Taken together, our approach to the work reported in (Beghelli & Jones, 2020) and that reported here was as follows. We first investigated what current software products were considered novel, to then check whether the same factors identified by authors such as Couger and Dengate (1992) were still in place or if new factors had emerged, and study whether the general public had the same perception about software novelty as software specialists. Figure 1 presents an overview of our approach.

Figure 1. Figure 1. Schematic of the process followed to study the perception of novelty in software products.

Given our ultimate objective of helping the software industry to develop competitive products that attract the public’s attention through novelty, we based all stages of our study on a score-assigned perceived novelty metric, as defined in (Fiorineschi & Rotini, 2021). That is, we ask the general public (customers who buy software products) and software specialists (who guide the development of new software products) to assign a score from a predefined scale of values to the perceived novelty of selected software products.

Perceived novelty is assessed ‘by the direct evaluator´s perception about the overall idea or part of it’ (Fiorineschi & Rotini, 2021) as opposed to assessing the originality of ideas by systematically comparing them with a reference idea, or set of ideas, or by evaluating the number of times a specific idea appears in a set. We argue that score-assigned perceived novelty is best suited to our aim as it emulates the spontaneous overall evaluation of novelty that customers make when they select an application or that specialists make when considering decisions about what features to include during the software design and development process. This approach enables us to address our research question regarding potential differences between members of the public and software specialists in their perceptions of novelty in software products.

Regarding the identification of the factors contributing to the novelty of software products, we take as a basis work that has identified generic factors contributing to the novelty of physical products (Sarkar & Chakrabarti, 2011, belonging to the first variant of factor-based approaches, discussed in Section 2.1) as opposed to methods that would identify factors specific to a given software application. In this way, we can answer our research question on what factors contribute to the perception of novelty of software products in general.

3.1. Stage 1: workshop with software specialists

The full details of stage 1 are given in (Beghelli & Jones, 2020). For the sake of completeness we summarize the main points below.

In this stage, we worked with software developers and software product managers from a medium-sized software company in the business of developing application programming interfaces (APIs). The session involved 10 groups, with 3–6 people each.

We first trained the groups on a factor-based procedure for evaluating the novelty of physical products that was based on the method proposed by Sarkar and Chakrabarti (2011). Next, we asked the participants to make a list of software products that they considered to be novel at the time they were launched and identify what factors they thought made these software products novel. To do so, we asked them to use as inspiration the factors proposed by Sarkar and Chakrabarti (2011) for physical products and attempt to find analogous factors in the software domain.

The resulting information from the exploratory study of Stage 1 was a list of applications deemed original by software specialists plus a list of factors that might play a significant role in making the mentioned software products original.

3.2. Stage 2: pilot study with general public

The aim of this stage was threefold: to gain understanding on how original different applications were perceived to be by the general public; to understand what factors the general public identified as important contributors to the originality of those applications; to pilot a questionnaire for use in stage 3 of our study.

To achieve these goals we worked with 17 participants who could be seen as members of the general public. That is, the participants were users of software products but had no training in software development and did not work in the software industry.

The pilot questionnaire (shown in Appendix B) was made of two parts:

• In Part 1, respondents were asked to evaluate the level of originality of different software products using a 7-point Likert scale (1: zero originality, 2: low originality, 3: low-medium originality, 4: medium originality, 5: medium-high originality, 6: high originality, 7: very high originality). There was an option to leave a blank if respondents were not familiar with the software. For this part 16 software applications were selected from the original list of 37 compiled during the first stage. We discarded applications aimed at a very specialized audience (e.g. Tensor Flow or Google Auth), applications that might be unknown to the general public (e.g. Anki, Docu Sign) and when several applications had the same functionality, only one of them was selected.

• In Part 2, participants had to select the factor(s) they considered relevant to making each application original. The same 16 applications from Part 1 were listed in Part 2. We also included all factors identified in stage 1, although two of these were combined (New functionality and Extra functionality), and some were renamed with the aim of providing greater clarity.

We distributed the questionnaire among the participants during one session of work. Participants completed the questionnaire voluntarily, did not receive any type of compensation for doing so and did not have a time limit to complete the questionnaire. Results from this stage are reported in section 4.2.

3.3. Stage 3: survey with software specialists and general public

In this stage we worked first on improving the questionnaire used in stage 2, by refining the factors included in the questionnaire, and decreasing the effort needed to respond by using a 5-point rather 7-point Likert scales and reducing the number of software products included to just 10. We distributed the questionnaire online, to both software specialists and the general public, and finally we analyzed the data that we collected in this way. Results are reported in section 4.3.

3.3.1. Creating the online questionnaire

3.3.1.1. Revision of factors

For our final survey, our aim was that all factors included, as potentially contributing to novelty, would be easily recognized and understood, both by software specialists and members of the general public. Our refined list of 5 factors, including the definition provided for each, was as follows.

3.3.1.1.1. Brand new functionality

Software that computerizes something that could not previously be done by computers. In the sense that Sarkar and Chakrabarti (2011) define this aspect, it would correspond to a software product that performs a task that no other product, either software or not, has performed before. An example of this case is Deep Fakes, in which software is used to create convincing fictional videos. Usually, they depict real people making controversial statements (Kietzmann et al., 2020). In the sense that Couger and Dengate (1992) define this aspect, it would correspond to a software product that makes computerizable something that was not computerizable before. An example of this case is Google Maps/Waze, where the task of finding the quickest route from one place to another, taking into account real-time traffic conditions, was computerized for the first time.

3.3.1.1.2. Extra functionality

Software that includes a new function, not usually included in this type of software product. If the product adds a new function similar to what other products already offer. An example of this is the filters of Snapchat, a messaging application that included the functionality of adding pre-designed images on top of parts of the pictures being sent. Multi-functional products are not considered in the classification of Sarkar and Chakrabarti (2011), where the focus is on single functional products, since this is the most common situation with physical products. However, software products are more prone than physical products to be multi-functional and therefore to having extra functionalities added to them. This factor builds on the work of Rose and Furneaux (2016).

3.3.1.1.3. New interface

Software products in which the components (software or hardware) provide a new way of accessing information or achieving tasks. For example, before the launch of the World Wide Web, information on the Internet was not connected (there were different repositories for different types of information) and the content of a file could only be checked by downloading it. The download of files was done using command lines. With WWW users could see the information on screen (at the beginning, only text or images) before deciding to download it and they also had hyperlinks to navigate from one piece of information to another. This factor is broadly analogous to some elements relating to the physical attributes of physical products, as identified by authors such as Sarkar and Chakrabarti (2011) and Jagtap (2016, 2019), and summarized in section 2.1. In a software product the change of interface can be as simple as changing the distribution of elements on a screen, or something more radical, like adding sensing hardware, as Leap Motion did. In that case, the change of interface comes hand in hand with a change in the interaction with the user (as below).

3.3.1.1.4. New interactive behavior

Software that interacts with the user in a new way. This factor builds on Couger and Dengate’s thinking on new approaches and methods, as well as the work of Jagtap (2016, 2019), Sarkar and Chakrabarti (2011), and Toh and Miller (2014) considering interactions with physical products. Examples of this are the virtual assistants Alexa or Siri, or the hand tracking system Leap Motion.

3.3.1.1.5. New context of use

Software that is used in a new physical or social environment or by a new type of user. An example of this is the computer game Pokemon Go, which provided a new context of use by moving videogames outdoors (new physical environment).

3.3.1.2. Shortlisting of software products

We selected 10 software products to be included in the questionnaire according to the following criteria:

• The products should exhibit different levels of originality. Since our previous lists were compiled based on what the group of software specialists considered to be original applications, there was not a single software product that could be considered not original. To help participants to calibrate their answers, by having very original software products as well as well-known but potentially less original software products, we included Instagram.

• The products should exhibit different factors of potential relevance for originality. In the original list we had Oculus Rift, to represent the factor New Interactive Behavior. However, Oculus Rift by itself is not a software product (it is the hardware that supports virtual reality applications) and it was also not very well known by the participants of stage 2. Since we could not find a very well known virtual reality application, we replaced Oculus Rift with another application mentioned in stage 1 that was strong on New Interactive Behavior. For this reason, Wii Golf was also included in the list.

The final list of products selected for inclusion in our questionnaire is shown in Table 1. Here we show the position of each software product in the preliminary ranking of originality obtained in stage 2, as well as the factors which that preliminary study suggested as relevant to the product’s originality. Note that since we did not have data for the New Context of Use factor, we had to rely on our own knowledge for this.

Table 1. The 10 software products selected for inclusion in the final online questionnaire

Software Product	Originality ranking	Brand new Functionality	Extra Functionality	New Interface	New Interactive Behavior	New Context of Use
Bitcoin	1st	x
Uber	3rd			x	x	x
Virtual assistants	4th				x
Pokemon Go	5th				x	x
Twitter	6th			x
Snapchat Filters	10th		x
Google Maps/Waze	13th	x
Google Suite	16th		x
Wii Golf	–				x
Instagram	–			x

The questionnaire created in this way is shown in Appendix C.

3.3.2. Administering the online questionnaire

The questionnaire was implemented in the online platform Qualtrics. Part 1 asked the participants to evaluate the level of originality of the 10 selected software products using a 5-point Likert scale (1: zero originality, 2: low originality; 3: medium originality, 4: high originality and 5: very high originality). Part 2 asked the participants to select the factors they considered relevant for the originality of each application. Part 3 was an open question where the participants were asked to describe the last application they found original.

The questionnaire was advertised in our social networks (LinkedIn, Facebook), and was open from 12 December 2020 to 21 December 2020.

4. The perception of novelty in software products: results

4.1. Stage 1: workshop with software specialists

The full results of stage 1 are reported in (Beghelli & Jones, 2020). In summary, we obtained a list of 37 applications deemed original by software specialists (see Appendix A) plus a list of factors that might play a significant role in making these software products original. The factors identified in this way were: Technology Infrastructure, New/Extra Functionality, User Interface, User Interaction and User Experience, as described earlier.

4.2. Stage 2: pilot study with general public

Table 2 shows – for each application included in the pilot questionnaire – the number of participants that selected a given level of originality. The applications are listed in decreased order of originality, taking as a measure of originality the mean value of the level of originality assigned by the participants.

Table 2. Pilot study results on the perception of originality of software products by the general public. The numbers on the first row represent the different levels of originality (1: zero originality; 7: very high originality)

Software Product	1	2	3	4	5	6	7	Total answers	Mean
Bitcoin	0	0	0	3	4	4	4	15	5,60
PayPal	1	0	1	2	5	3	4	16	5,19
Uber	1	2	0	4	2	6	2	17	4,76
Virtual assistants	1	3	0	2	3	3	3	15	4,60
Pokemon Go	0	0	3	3	3	2	1	12	4,58
Twitter	2	0	1	4	4	3	2	16	4,56
Leap Motion	1	0	1	2	2	2	1	9	4,56
Airbnb	1	2	3	1	3	4	2	16	4,44
Spotify	0	1	5	1	2	2	2	13	4,38
Snapchat filters	1	4	0	3	1	6	0	15	4,13
Oculus Rift	0	4	2	4	1	3	1	15	4,00
Revolut	1	2	3	3	0	3	1	13	3,92
Waze/Google Maps	3	0	3	4	4	1	1	16	3,81
Deep Fakes	0	3	2	2	1	1	1	10	3,80
Tinder	1	2	3	3	4	1	0	14	3,71
Google Suite	4	3	3	1	3	1	0	15	2,93

We could observe that even though the software specialists who participated in stage 1 perceived all these applications as original, the general public who took part in the pilot study considered some of them much more novel than others. Bitcoin, for example, was considered far more original than Google Suite. Additionally, Leap Motion and DeepFakes received the lowest number of answers (9 and 10, respectively), signaling that they were perhaps not very well known by the general public.

Table 3 shows the results obtained in the second part of the pilot study. For each application and factor, the number of respondents that selected that factor as important in driving the originality of the application is shown. Note also that although the option ‘Other’ was available; none of the participants suggested a new factor for novelty.

Table 3. Pilot study results on perceptions of what factors affect the originality of software products

Software Product	Technology Infrastructure	New/Extra function	New Interface	New interaction	New User Experience
Bitcoin	11	4	6	6	8
PayPal	10	6	8	7	9
Uber	6	6	5	6	13
Virtual assistants	8	5	9	14	13
Pokemon Go	6	2	7	9	11
Twitter	3	2	7	5	7
Leap Motion	5	3	5	4	6
Airbnb	0	7	6	4	8
Spotify	1	5	6	3	7
Snapchat filters	8	6	7	12	10
Oculus Rift	5	3	3	2	3
Revolut	4	5	2	6	11
Waze/Google Maps	0	4	2	5	6
Deep Fakes	4	3	2	1	4
Tinder	1	2	4	3	5
Google Suite	4	10	3	9	9

From these preliminary results we made the following observations, which led to the refinements in our list of factors for the final survey, as described in section 3.3.1 above.

First, it was evident that responses were affected by the meaning that participants assigned to each factor. For example, it seems that by New Interaction some people understood new ways of interacting between users of the application, rather than ways of users interacting with the application, as intended. This would explain why the factor New Interaction was the most selected one for Snapchat filters, even when this application did not actually change the way the user interacts with the application. Thus, we understood it would be necessary to more clearly define each factor in our final study.

In addition, the factors included in our initial list were not independent of each other. For example: New Interface and New Interaction were parts of the User Experience. Uber, Revolut Pokemon Go, Snapchap filters, Waze, Tinder, Twitter and Spotify were all somehow mostly associated to the factor of New User Experience. However, sometimes this factor alone was selected and sometimes the mixture of New User Experience plus one component of it (Interface or Interaction) was selected. Including factors at different levels of aggregation (User experience aggregates factors as New Interface and New Interaction) seemed to cause confusion and might distort the results.

Given the above observations, we reviewed the list of factors to be included in the final stage of our study as follows.

• New Technology is a driver for new interfaces, new ways of interacting with the user and new or extra functionality. Due to this dependence, we discarded New Technology from the list of factors and only left the factors that are a result of having a new technology: New Functionality (either brand new or additional functionality), New Interface and New Interaction.

• We separated the New/Extra Functionality factor from our pilot study into two different factors: Brand New functionality and Extra Functionality.

• Based on the description of the term User Experience given by the International Organization for Standardization (2019), which includes elements of interface and interaction, we discarded the aggregated concept of User Experience and kept only the disaggregated components: New Interface and New Interaction.

• Finally, in reviewing the factors affecting User Experience, we realized there was an important component not mentioned before: Context of Use. This was therefore added to the list of factors in our final study.

4.3. Stage 3: survey with software specialists and general public

The online questionnaire was answered by 146 people, of whom 33% (47 respondents) identified as female and 67% (99 participants) as male. The gender option ‘Other/I prefer not to say’ was not selected by anyone. There were two main groups of people: a group of 85 participants related to the software industry (either as workers, educators or students) and a group of 61 participants without any relation to the software development area, except as users. We call those groups software specialists and public, respectively. The composition of those groups in terms of age and gender distribution is listed in Table 4.

Table 4. Age and gender distribution of survey respondents

		FEMALE						MALE
		18–24	25–34	35–44	45–54	55+	Total	18–24	25–34	35–44	45–54	55+	Total
Software Specialists	Worker	1	2	0	2	0	5	3	13	7	12	0	35
	Educator	0	0	0	1	0	1	0	3	10	6	0	19
	Student	9	5	0	0	1	15	6	2	2	0	0	10
Public	Public	2	2	12	10	0	26	1	1	13	17	3	35
	Total	12	9	12	13	1	47	10	19	32	35	3	99

A high gender imbalance can be observed in the group of workers and educators (software specialists), with just 12.5% and 5% of female participants, respectively. Such extreme imbalance is not observed in the remaining groups: the students and general public groups were made of 60% and 42.6% of female respondents, respectively. The high imbalance of the groups of people developing software and teaching about software, also reported in several published works (see, for example, Diaz Canedo et al., 2019), is concerning as these groups might lack the point of view of an important part of the public when designing novel applications. However, due to the lower number of answers from female respondents there is not enough data in this study to be able to derive statistically valid conclusions regarding gender-based perception of software novelty. Hence, in the rest of the analysis we will focus only on the two main groups of software specialists and members of the public.

Table 5 shows the ranking of software novelty according to the perceptions of these two groups. Each row represents a different level of originality. In every row we list the software products in that level of originality, according to each of our main groups of respondents. Against to the name of each software product we include two numbers: the average level of originality assigned to that software product (e.g. 4.58 for Bitcoin, according to software specialists) and the percentage of respondents that evaluated the level of originality of that application (e.g. 93% of the software specialists evaluated the level of novelty of Bitcoin, whereas only 72% of respondents from the general public did so). The latter figure can be assumed to represent the level of knowledge of that software product among the participants in our two groups.

Table 5. Perception of the originality of 10 software products by software specialists and by respondents from the general public

Ranking	Software Specialists (85)	Public (61)
4–5 Very high or high originality	Bitcoin (4.58/93%) Pokemon Go (4.08/93%) Uber (4.05/100%) Google Maps (4.01/100%)	Virt. Assistants (4.04/85%) Google Maps (4.03 /97%)
3–3.99 Medium originality	Virt. Assistants (3.68/96%) Google S (3.63/99%) Snapchat Fltrs (3.62/87%) Wii Golf (3.56/69%) Twitter (3.39/98%)	Bitcoin (3.89/72%) Uber (3.88/97%) Pokemon Go (3.85/75%) Wii Golf (3.75/46%) Snapchat Fltrs (3.62/64%) Instagram (3.38/95%) Google S (3.36/92%) Twitter (3.30/93%)
2–2.99 Low originality	Instagram (2.85/94%)
1–1.99 Zero originality

The disaggregated information of Table 5 showing the frequency of the answers for the different levels of originality for all software products can be found in Figure 2. To the right of each bar, the number of respondents selecting the corresponding level of originality is shown. The left and right figures present the results obtained from asking to software specialists and general public, respectively.

Figure 2. Frequency of answers of software specialists (left) and the public (right) for the different levels of perception of originality of the studied software products.

Figure 2. Continued.

Figure 2. Continued.

A Fisher’s exact test was applied to the frequency of the answers to check whether the responses of both groups (software specialists and general public) were statistically different. The groups presented statistically significant differences (i.e. p-value < 0.05) only when evaluating the originality of Bitcoin, Virtual Assistants and Instagram. The remaining software products showed no significant differences in terms of the originality evaluation between both groups. This might be considered a good result, suggesting a general consistency between the perception of originality of users and software specialists. However, the number of software products here studied is too low to be able to draw generic conclusions regarding this.

In the case of Bitcoin, the significant difference in terms of knowledge of the product (93% of the software specialists evaluated its originality versus only 72% of the general public) might be one of the reasons behind the difference of perception between both groups, as Bitcoin is not yet a mainstream software product among the general public.

In the case of Instagram and Virtual Assistants, the data from originality perception alone does not allow us to identify a clear reason behind the detected differences.

In spite of the differences between both groups, the same set of 5 software products was recognized as the most original ones: Bitcoin, Virtual Assistants, Pokemon Go, Uber and Google Maps/Waze. Thus, we will next study what factors were deemed determining of the originality of those 5 software products.

First, we performed a Fisher’s exact test on the frequency of the answers to check whether the responses of both groups were statistically different. They were not (all p-values > 0.05). Hence, from now on we analyze the aggregated responses of both groups.

Figure 3 shows the number of times participants (from both groups, software specialists and public) selected each factor as a driver of the originality for Bitcoin, Virtual Assistants, Pokemon Go, Uber and Google Maps/Waze.

Figure 3. Distribution of response frequency for each factor driving originality for Bitcoin, Virtual Assistants, Pokemon Go, Uber and Google Maps/Waze. Responses from all participants.

A χ2 goodness of fit was performed to the frequencies of the different factors selected by the respondents to determine which were the more relevant ones. To do so, first a χ2 goodness of fit was performed considering all the 5 factors. If the frequencies were significantly different (p-values of χ2 test ≤ 0.05), then a new χ2 test was performed using the 4 factors with the highest frequencies. If the frequencies were again significantly different, we performed a new χ2 test with the next 3 highest values and so on. The results were as follows:

• Bitcoin: The factors determining the originality of this software product were Brand New Functionality and New Context of Use.

• Virtual Assistants: The factors determining the originality of this software product were Brand New Functionality, New Interactive Behavior and New Context of Use.

• Pokemon Go: The factors determining the originality of this software product were New Interactive Behavior and New Context of Use

• Uber: All factors contribute to the originality of this software product: Brand New Functionality, Extra functionality, New interface, New Interactive Behavior and New Context of Use

• Google Maps/Waze (path finder): All factors contribute to the originality of this software product: Brand New Functionality, Extra functionality, New interface, New Interactive Behavior and New Context of Use

Table 6 summarizes the factors identified for the 5 software products considered as most original by the participants.

Table 6. Factors determining the originality of the 5 software products perceived as most original by the survey participants

	Brand New Functionality	Extra Functionality	New Interface	New Interactive Behavior	New Context of Use
Bitcoin	x				x
Virtual Assistants	x			x	x
Pokemon Go				x	x
Uber	x	x	x	x	x
Google Maps/Waze	x	x	x	x	x

The above findings suggest that for a software product to be considered original its designers should perhaps consider incorporating the factors of New Context of Use and New Technology, as the driver behind new functionalities, interfaces and interaction. A new context of use can be provided by either changing the users that normally use this software or making the software usable in a new physical or social environment. The latter case was the most common in the cases here analyzed. For example, Pokemon Go moved video games outdoors. In the case of including a New Technology, this should be implemented in order to provide either a Brand New Functionality (as the case of Bitcoin) or a New Interactive Behavior (e.g. Virtual assistants). There is no evidence, based on the software products studied here and the answers obtained, that Extra Functionality alone or a New Interface alone will be enough to produce a very original product, as they only appeared in cases where Brand New Functionality and New Interactive Behavior were also selected.

5. Conclusions

In this paper we have provided some examples of software products that are seen as original, at time of writing, by both software specialists and members of the public as well as identified key factors that might affect the perception of novelty of software products, namely: a new context of use, where the software is designed to be used in a new physical or social environment, and the use of a new technology that allows a product to offer a brand new functionality or a new interactive behavior, allowing users to interact in a new way with the product or application.

Thinking specifically about the research questions we began with, we can answer as follows. First, it seems that there are strong similarities in the ways in which software specialists and members of the general public think about novelty in software – despite the gender imbalance in our group of software specialists. This may be helpful to know, for software companies seeking to innovate in order to meet the needs of customers seeking new and useful products, though it should be remembered that the public seem to have a less nuanced appreciation of levels of originality, tending to think of the majority (8 out of 10) of the software applications in our study as being of ‘medium originality’.

Perhaps most importantly, we are developing a clearer understanding of what might be the factors that contribute to perceptions of novelty in software. As described above, our understanding of these factors evolved somewhat during the course of our studies. We currently believe the key factors to be either the use of software in a new context, or the use of a new technology to enable new functionality, or ways of interacting with a software product.

The above findings are in line with the findings reported in (Norman & Verganti, 2014), where new meaning (that is directly related to new context of use) and new technology are identified as the two main drivers for disruptive products. In the context of this work, a new meaning can be given to a software product either by placing it in hands of different users or by changing the physical or social environment in which it is used. We have also identified where the efforts for a new technology might be focused: either in providing a brand new functionality or a new interactive behavior.

Finally, we might argue that our findings broadly echo the overall categories of factors identified as significant to the novelty of physical products, with functionality and interaction appearing as important to software novelty as well as physical product novelty, physical or working principle being replaced by the use of a new technology, and physical attributes being replaced by user interface, although this is perhaps not so important in perceptions of novelty in software as physical attributes are in judging physical product novelty.

Of course the study we have reported here was limited, especially in the number of respondents to our final survey. Accessing a larger sample of responses would potentially enable greater certainty in our findings. It would also be interesting to include a larger number of software products in any future survey. We hope that the data we have so far collected might nevertheless provide interesting directions for further work. In the meantime, we argue that considering factors such as those we have identified above may provide a useful basis for designers to work together with software developers in building a better understanding of what constitutes novelty, and hence creativity, in software products, to push the boundaries of software innovation. Using the criteria we have identified here, software innovation challenges can be better addressed by focusing efforts on the aspects that most impact the perception of novelty in software products.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Appendix A

Original applications and factors contributing to originality, by software specialists

List of 37 software products considered by participants to be novel at the time they were launched, in alphabetical order of the category they belong to. The categories are based on the 28 categories used in the App Store

Table

CATEGORY	APPLICATIONS
Business	Google Suite (sheets, docs, etc), Google Auth, Google Search; Swift Key, Docu Sign; Amazon Web Services (AWS)
Developer Tools	Tensor Flow
Education	Anki
Entertainment	Snapchat filters
Finance	Monzo, Revolut, Bitcoin, Transfer Wise, Pay Pal
Games	Playstation, Wii Golf, Pokemon Go
Graphics & Design	Deep Fakes
Health & Fitness	Head Space, CAT scan
Music	Shazam, Spotify
Navigation	Google Maps, Uber, Waze
Shopping	Sizer, Airbnb, Wetherspoons
Social Networking	Snapchat, Tik Tok, Tinder, Twitter
Other	Oculus Rift, Leap Motion, Alexa, Siri, WWW

Appendix B

Pilot questionnaire

PART 1. In the first part of this exercise, we will think about the level of originality of different software products. For each software product listed in the following table, please mark the column with the level of originality you think this software exhibited at the time it was launched into the market. If you do not know the software, please leave the boxes blank.

Table

Software product	Zero originality	Low originality	Low-medium originality	Medium originality	Medium-high originality	High originality	Very high originality
Google Suite (docs, sheets, etc)
Snapchat filters
Pay Pal
Leap Motion
Revolut
Pokemon Go
Deep Fakes
Virtual assistants (Siri/Alexa)
Bitcoin
Spotify
Uber
Airbnb
Tinder
Oculus Rift
Waze
Twitter

PART 2. In the second part of this exercise, we will think of the aspects that make a software product original. For each software product, please tick the boxes (one or more) of the aspects you think made this product original at the time it was launched into the market.

Table

Software product	Technology infrastructure	New/Extra functionality	New interface	New interaction	New user experience	Other
Google Suite (docs, sheets, etc)
Snapchat filters
Pay Pal
Leap Motion
Revolut
Pokemon Go
Deep Fakes
Virtual assistants (Siri/Alexa)
Bitcoin
Spotify
Uber
Airbnb
Tinder
Oculus Rift
Waze
Twitter

Appendix C

Online questionnaire

Novelty in Software

Novelty in Software (Dr. Sara Jones ETH2021-0513 20.11.20 v1). We would like to invite you to take part in a research study by completing a short questionnaire, which should take no longer than 5 minutes to complete. The purpose of this study is to develop our understanding of how we think about novelty in software. We are interested to hear the views both of those involved in studying and developing software, and of members of the general public, and we hope that completing the questionnaire may be interesting for you, in helping you to reflect on your own thinking in this important, but under-researched, area. We will also be happy to share our findings, both through academic publications and social media posts to platforms such as LinkedIn, once sufficient responses to the questionnaire have been collected and analyzed. Completion of the questionnaire is voluntary. We do not believe it will involve any risks or harm, but even so, you can stop at any stage without being penalized or disadvantaged in any way. Responses to the questionnaire will be anonymous, but note that once you have completed the questionnaire and pressed ‘Submit’, you will no longer be able to withdraw your responses. Incomplete responses will not be retained. This study has been approved by City, University of London Business School (formerly Cass) Research Ethics Committee. If you have any problems, concerns or questions about this study, please e-mail Dr Sara Jones at [email protected]. If you remain unhappy and wish to complain formally, you can do this through City’s complaints procedure. To complain about the study, you can phone +44 (0)2,070,403,040, ask to speak to the Secretary to Senate Research Ethics Committee, and inform them that the name of the project is Novelty in Software. You can also write to the Secretary at:

Anna Ramberg

Research Integrity Manager

City, University of London, Northampton Square

London, EC1V 0HB

E-Mail: [email protected]

Thank you for taking the time to read this information.

Consent I understand what taking part in this study will involve.

1. I agree to take part in this study. (1)

Please provide the following information about yourself.

What is your age?

▼ 18–24 (1) … 55+ (5)

Gender

▼ Female (1) … Other/Prefer not to say (3)

Occupation

▼ Technical worker in the software industry (software developer, software product manager, etc) (1) … General public (4)

PART 1 In the first part of this exercise, you will think about the level of novelty of different software products. For each software product listed below, please indicate how original you think this software was at the time it was launched into the market.

Table

	Zero originality (1)	Low originality (2)	Medium originality (3)	High originality (4)	Very high originality (5)	I don’t know (8)
Google Suite (Google docs, Google sheets, etc) (1)
Bitcoin (2)
Snapchat filters (3)
Pokemon Go (4)
Wii Golf (5)
Google Maps/Waze (path finder) (6)
Instagram (7)
Uber (8)
Virtual assistants (Alexa, Siri) (9)
Twitter (10)

PART 2 In the second part of this exercise, you will think about the aspects that make a software product original.

For each software product, please tell us which aspects (one or more) you think made this product original at the time it was launched into the market.

Brand new functionality: software that computarises something that could not previously be done by computers

Extra functionality: software that includes a new function, not usually included in this type of software product

New interface: software products in which the components (software or hardware) provide a new way of accessing information or achieving tasks

New interactive behavior: software that interacts with the user in a new way

New context of use: software that is used in a new physical or social environment or by a new type of user

Table

	Brand new functionality (1)	Extra functionality (2)	New interface (3)	New interactive behavior (4)	New context of use (5)	Not applicable (6)
Google Suite (Google docs, Google sheets, etc.) (1)
Bitcoin (2)
Snapchat filters (3)
Pokemon Go (4)
Wii Golf (5)
Google Maps/Waze (path finder) (6)
Instagram (7)
Uber (8)
Virtual Assistants (Alexa, Siri) (9)
Twitter (10)

Finally, we would like to ask you to share an experience of using novel software. If you would be happy for us to use extracts from your answer in any future publications, please let us know here. Remember that all responses will be anonymous.

1. I agree for my response to be directly quoted (1)

Please, remember the last time you were surprised by some novel software. What software was that? What aspect made you consider the software original?

________________________________________________________________

Thank you for helping us to understand novelty in software.