QFD embedded Agile approach on IT startups project management

Abstract

Given the significant role of startups in the wellbeing of the economy and considering that more than half of the existing startups either work or have activities related to the IT field, in the current paper we have proposed a model meant to aid IT startups in their development process and in achieving customer satisfaction while ensuring the quality of deliverables. The proposed model has been applied and tested on IT startups and the results have been disseminated in the course of the paper. The need for the proposed model arose from the observations regarding the issues that IT startups are faced with in regard to customer satisfaction, timely delivery of products, reaching a level of quality for the offered products and team management.

Keywords:

• optimization
• development process
• ISO 9001:2015
• risk management

Subjects:

• L22
• L26
• O22
• O32
• O33

PUBLIC INTEREST STATEMENT

The paper analyses models used by IT startups in their development process for achieving customer satisfaction and ensuring the quality of deliverables, and highlights the particularities of each used model. It then proposes a model for helping startups better understand their customers’ needs and translate these needs into technical specifications to be followed in order to achieve customer satisfaction. The model is based on the Quality Function Deployment methodology, but also takes into account the particularities of startup businesses. The proposed model was implemented and tested in the case of three real-life startups in order to understand how it can be used and what benefits it can bring.

1. Introduction

The current globalized economic context encourages companies to focus on providing disruptive innovation in order to remain competitive (Weiblen & Chesbrough, 2015). Startups have emerged as a response to this competitive environment, given how the costs required for marketing products have decreased (Miller & Bound, 2011) and a support system for these types of businesses has been established through the help of incubators, angel investors, venture capitalists and government funds, amongst others (Lukosiute et al., 2019), and have culminated with the dot-com bubble in 2000 (Spender et al., 2017).

Startups are represented by newly created companies which are confronted with high uncertainty and rapid evolution in technologies and markets (Giardino, Unterkalmsteiner et al., 2014). Rapid advancements in the technology and Internet fields have led to an increased number of software and IT-oriented startups (Marmer et al., 2011) as well as an increased number of options regarding the development process approaches. This in turn leads to an interesting aspect regarding the quality of businesses working in the IT sector, which is currently facing issues on this matter. There have been documented attempts regarding the implementation of quality assurance practices within IT startups using Agile methodologies, and debates concerned with the implementation of Quality Management Systems within IT startups to help them deliver quality products that meet customer expectations. Quality, in this case, is considered to be the ability to assess, anticipate and fulfil customer requirements, whether these are clearly stated or just implied. It has been observed however that an implementation of QMS within IT startups would not be as beneficial as implementing certain practices, such as, for instance, documenting decisions. If then, we are to consider implementing quality management practices, it would be helpful to take a look at quality management tools, and specifically, identify those which could help startups achieve and assure quality of their products and/or services throughout their development lifecycle.

The model presented in this paper proposes a software project management framework that is based on the QFD (Quality Function Deployment) quality management tool, adapted with software startup requirements, which make it a central element for an IT project management model based on Agile development processes.

The main advantage of implementing the model is the introduction of a quality management culture in the lifecycle of IT startups focused on digital transformation, innovation and rapid deployment. This helps startups achieve an overall image regarding not only their developmental capabilities and project management estimates but also the quality of their products and the development process itself. Given how fulfilling customer requirements is an important aspect to be taken into account in order to ensure the quality of deliverables, the presented model allows the estimation/measurement of customer satisfaction through the help of the Offset indicator.

1.1. Literature review

1.1.1. Startups

Startups have evolved as a natural response to the economic climate based on innovation, quality and customer involvement. Although no consensus has been reached regarding the definition of startups, the most widely accepted definition is the one proposed by Eric Rise, according to which “startups are a human institution designed to create new products and services under conditions of extreme uncertainty” (Rise, 2011). Software startups inherit the same general characteristics as startups from other industries, with an exception for the end product and main goal, which is to provide high-tech and innovative products (Blank, 2005).

Some of the conditions required for a company in order to be considered a startup, and to help distinguish it from a small business relate to having an innovative technology or business model at its core, aiming to achieve significant sales growth and being no older than 10 years (European Startup Monitor, n.d.).

The environment in which startups operate is often described as being one filled with risk and uncertainty (Blank & Dorf, 2012) (Morales-Trujillo & García-Mireles, 2019), and that is one of the reasons why most startups fail in the first five years (Nobel, 2011) (Crowne, 2002).

Figure 1 shows the state of startups created in 2014 over a period of five years, and as it can be observed, the general trend of startup survival is a decreasing one, with only 56% of created startups surviving in their fifth year, reflecting the observations noted by the literature.

Figure 1. Startup success rate for the last five years (source (Mansfield, 2019)).

Some of the most significant issues identified for startup failure relate to the availability of financial resources here (Livingston, 2009), the level of technological innovativeness possessed (Giardino et al., 2015) and the lack of experience (Giardino, Wang and Abrahamsson, Why Early-Stage Software Startups Fail: A Behavioral Framework 2014).

The role startups and small companies play in the wellbeing of the economy has been noted throughout time by various researchers (Sedlacek & Sterk, 2014) (Kane & Marion, 2010) (Stangler et al., 2010).

Due to their particularities such as making frequent changes in order to accommodate customer requirements, a large number of software startups have embraced agile methodologies for their development process (Duc & Abrahamsson, 2016) (Pantiuchina et al., 2017). Agile methodologies inherit similar particularities such as embracing change, ensuring a flexibility for adapting to the business strategy (Coleman & O’Connor, 2008), releasing products and functionalities in an incremental, iterative way (Giardino, Unterkalmsteiner et al., 2014), embracing self-organizing teams and putting an emphasis on working code instead of thorough documentation (Beedle et al., n.d.).

1.2. Agile methodologies

Agile methodologies have evolved in the last decade and have impacted the way software development is performed (Bosch et al., 2013). Designed originally for small single team projects (Boehm & Turner, 2005), the Agile methodologies were soon implemented by larger companies due to their added benefits and despite the implementation difficulty (Dybå & Dingsøyr, 2009) (Dikert et al., 2016).

Agile software development (ASD) is a set of software engineering methodologies designed to embrace and adapt to change, based on an iterative and incremental approach highlighted in the Agile manifesto (Dybå & Dingsøyr, 2009). The term Agile development encompasses a number of development methodologies, the most popular of which is Scrum, as it can be seen from Figure 2 (Blueprintsys, n.d.).

Figure 2. Agile methodologies most commonly used.

Although there are multiple methodologies belonging to the Agile development approach, it has been observed that none of them are strictly followed by startups (Giardino, Unterkalmsteiner et al., 2014), and a reason for this might be related to the fact that startups can implement and test the newest methodologies and technologies without being affected by legacy or previous working experience (Pantiuchina et al., 2017). Lack of experience has been considered both from the perspective of startups that have no previous experience working in a company or managing a company, as well as not having sufficient background in a particular field of activity, and/or working in a cross-functional team.

1.3. Software quality models

The concern for quality software has begun to manifest since the growth of the software industry and has extended its concept regarding how quality is viewed and measured in the context of software, to cover multiple attributes besides functionality (Bashir et al., 2016).

In order to respond to the issue regarding the quality of software products, various models for customer feedback and requirements have been developed (Thongtanunam & Hassan, 2020). Among these approaches, the literature mentions rapid throwaway prototyping (Thongtanunam & Hassan, 2020), spiral model, iterative enhancement (Dalton, 2019), capability maturity model (CMM) (Torrecilla-Salinas et al., 2016). Others put an emphasis on integrating quality management practices and concepts into the software development methodology to ensure a quality software. Amongst these, there are presented the integration of ISO 9000 into the development process (Thongtanunam & Hassan, 2020), integration of TQM elements (Iqbal & Muhammad, 2017) and the integration of quality management tools and techniques into the Agile development processes (Dovleac & Ionica, 2017). Results were also obtained from the integration of the quality management tool called QFD (Quality Function Deployment) in software engineering (Leba & Ionica, 2012) (Ionica & Leba, 2014), as a software requirements selection tool (Sen & Baraçlı, 2010).

Quality models help identify the quality of a software product by analysing a set of quality characteristics which a software product has. There are a number of different quality models available, but some of the common features include Reliability, Efficiency, Usability, Portability, Maintainability, Functionality, Testability and Reusability (Al-Qutaish, 2010). Furthermore, the ISO/IEC 25010 standard proposes a quality model comprised eight quality characteristics, as follows: Functional Suitability, Performance Efficiency, Compatibility, Usability, Reliability, Security, Maintainability and Portability (ISO 25000, n.d.).

In the context of the currently proposed model, the following characteristics have been taken into account for the quality model, and are reflected in the QFD section of the model: Flexibility, Efficiency, Usability and Functionality.

2. Research context

New software startups are launched every day due to the entrepreneurial-oriented environment, which allows entrepreneurs to have access to new markets, to affordable new technologies and financial resources in the form of venture capital, angel investors or crowdfunding. However, a large number of the newly established startups will fail because of one or more reasons, among which the failure to deliver on time what the customer required (Livingston, 2009).

Given the significant role that startups play in the economy and considering the identified issues regarding the shortcomings of project management practices implemented by startups, in the current paper, the authors propose a project management model fit for startup requirements, based on the quality management tool called QFD and Scrum methodology of the Agile approach.

Another important aspect to be considered is the “visibility” of software products, and especially the quantification of the work behind the product. As it is already known, software products contain a functionality that is considered the backend or logical structure, on which the “visible” part is built. The logical structure covers the code and programming aspects of the project, while the frontend is the graphical interface that the end user interacts with. In startups, where funding could depend on venture capital or crowdfunding money, it is important to deliver visible functionality—meaning a compromise between backend and frontend functionality, with an emphasis on the latter due to concerns regarding the efficiency of the development team and making further investments in it. Functional visibility expresses how much of a product has been developed taking into account the effort of the development team and the importance of certain functionalities for the customers.

The proposed model creates a link between project management and quality management in order to ensure a timely delivery of products designed according to customer requirements.

3. Methodology

The main goal of the study was to understand the ways in which startups might address customer satisfaction while ensuring that the scarce resources available are used in an optimal way. In order to address these concerns, the research methodology for the current paper was developed with the following research questions in focus:

RQ1: How is customer satisfaction taken into account and measured in the context of IT startups?

RQ2: What project management practices do IT startups implement and are these efficient?

RQ3: How does implementing aspects of project management and quality management benefit IT startups?

In order to provide an answer to these questions, the authors first examined the literature in the field of startups and IT startups, IT startup project management practices and quality management practices in the context of IT startups. The results of the analyzed literature helped understand the requirements of IT startups and shape the model in a matter that considers their particularities.

The literature review of the paper addresses the concerns regarding the first two research questions, while the case studies presented address the third research question. After the model has been developed, the authors decided to implement it in the practices of three IT startups, in the form of case studies. Figure 3 details the workflow of the conducted research.

Figure 3. Methodology flowchart.

3.1. Data collection

Data were collected primarily by observation of the results that the startups achieved, registered in the project management platforms used, but was supplemented by interviews and questionnaires carried with the members of the development teams and stakeholders. Data were collected over a period of one year for each startup, with monthly feedback from the stakeholder and the development team, and a weekly review of the work progress measured with the help of the proposed model and the amount of work completed as finished tasks in the project management platform used. When collecting data, the authors put an emphasis on identifying not only how much work was completed in terms of finished tasks but also to what amount did the complete work satisfy customer requirements.

3.2. Data analysis

All of the collected data was taken into account and has been divided into three distinct groups in order to match the research questions that were previously raised.

4. QFD-based model for Agile development project management

The proposed project management model is based on the quality management tool called QFD in order to help define and understand customer requirements and uses the principles behind the Agile development approach and some of the particularities of the Scrum methodology. The reason behind choosing Scrum methodology of the Agile approach is the familiarity with it since more than half of those who implement Agile uses Scrum (Blueprintsys, n.d.). Practitioners experience regarding the implementation of the Scrum methodology shows an increase in customer satisfaction, a reduction in the delivery time and a more organized workflow, in which each team member knows exactly what they have to do and the timeframe allocated for their activities.

Although the Scrum methodology allows the development team to deliver products to customers faster, a struggle still remains, regarding the identification of those customer requirements which should be fulfilled first and, more importantly, measuring the satisfaction that they bring to the customers. In this sense, the proposed model provides a way of identifying the most significant customer requirements to be completed first and a way to estimate the value of the deliverables by taking into account aspects such as tasks’ degree of difficulty, customer requirements degree of importance, task interdependencies and so on.

In order for the proposed model to be used in the development process of IT startups, some conditions need to be met. Concepts from the Scrum methodology have been implemented such as an iterative delivery of products, short development cycles organized into sprints and assigning roles for the development team’s members. Regarding the roles of the team members, there must be a person responsible for the product, who plays the part of an intermediary between the development team and the customer. In the proposed model, the person is called Product Owner (PO) like in the Scrum methodology. A person must be responsible for guiding the development team towards achieving the goals established together with the PO. This person is called the Scrum Master (SM). The team responsible for building the product and not including the PO and the SM is called the Development Team (DT).

Secondly, the team must gather customer requirements in the form of User Stories (US) which are sentences meant to help the development team understand not only what the customer wants but also what the purpose of the product is and how the customer intends on using it. This will allow the development team to clarify aspects regarding the features that the product must have.

Thirdly, the development team must implement an iterative development process, in which the product is delivered in small chunks of functionality over the course of development periods of time called sprints, rather than delivering a finished product after a long time.

The proposed model uses the US as entry points, based on the QFD methodology in which customer requirements are considered inputs. After this, the development team must establish a set of things amongst which: what are the most important US that the customer expects to receive in the product; what are the required technical characteristics in order to fulfil the customer demands, the degree of difficulty for tasks and which technical characteristics must be fulfilled first.

A representation of the adapted QFD model and the inputs for it can be seen in Figure 4, where the customer requirements, technical characteristics and interdependencies are inputs and the relationship matrix is the output. The values obtained from the Relationship matrix will be used for the calculation of an overall fulfilment index, called Offset.

The selection of the QFD quality tool as a foundational basis for the proposed model, instead of another quality framework is related to the particularity of the tool, which takes into account the voice of the customer in the development process. This, the authors argue, is in accordance with Agile principles related to the integration of the customers and stakeholders in the development process.

Figure 4. Adapted QFD model.

The Offset is the core of the proposed model and it helps the development team and the customers understand the deliverables both in terms of visible functionality and amount of work done. The Offset proposed by the authors is based on the developed mathematical model and uses inputs from the adapted QFD model previously described. The Offset is determined with the help of the following formula:

(1) $Offset=\sum _{j=1}^m\sum _{i=1}^n\frac{\left\{\begin{array}{c}0,if{\prod }_{k=1}^m\left(AT\left(k\right)+1-TT\left(k,j\right)\right)=0\\ \frac{IUT\left(i,j\right)\ast T\left(j\right)\ast AT\left(j\right)}{{\sum }_{r=1}^{m}IUT\left(1,r\right)\ast T\left(r\right)},else\end{array}\right.\ast US\left(i\right)}{{\sum }_{l=1}^{n}US\left(l\right)}$(1)

As it can be seen from Equation (1)(1) $Offset=\sum _{j=1}^m\sum _{i=1}^n\frac{\left\{\begin{array}{c}0,if{\prod }_{k=1}^m\left(AT\left(k\right)+1-TT\left(k,j\right)\right)=0\\ \frac{IUT\left(i,j\right)\ast T\left(j\right)\ast AT\left(j\right)}{{\sum }_{r=1}^{m}IUT\left(1,r\right)\ast T\left(r\right)},else\end{array}\right.\ast US\left(i\right)}{{\sum }_{l=1}^{n}US\left(l\right)}$(1) the Offset is calculated with the help of the following inputs: the task completion matrix—which can register values of 0 and 1 (0 for tasks that have not been completed and 1 for completed tasks), represented in the formula as AT; the values for the interdependencies between technical characteristics, with values of 0 or 1 (0 for technical characteristics which are independent, and 1 for technical characteristics which are depended on others), represented as TT; the values for customer requirements coverage by the technical characteristics percentage, represented as IUT; the values representing the evaluated degree of difficulty for the technical characteristics, represented as T; and the values representing the evaluated degree of importance for the customer requirements, represented as US.

The logical diagram showing the inputs and the methodology which leads to the determination of the Offset can be observed in Figure 5. As it can be observed the customer requirements provide inputs for the proposed model both independently and through the Technical characteristics which are derived from the customer requirements. Based on these inputs, the model can determine the value of the Offset which shows how much visible functionality has been achieved.

Figure 5. Simplified logic diagram for Offset relevance.

The simplified logic diagram is representative of a singular sprint in the development process and presents the mechanism used by the development team to select the technical characteristics to be fulfilled in order to meet customer requirements, all based on the Offset value. For the entirety of the project, the process will be repeated before the beginning of each new sprint, leading to a development loop, which takes into account, at the beginning of each sprint, modifications in customer requirements or in technical capabilities of the development team.

Based on the value obtained from the Offset, the best way for fulfilling customer requirements is selected, by choosing to meet the technical characteristics which help achieve the highest value of the Offset at the end of the development period.

The expected value for the Offset can be set at the beginning of the sprint planning activity, therefore providing an automated technical characteristics selection for a sprint.

5. Results

The QFD model has been implemented in the development process of three IT startups, out of which, one is a product-based startup (Vitraly), one is a service-based startup (Lemur) and one is a mix of product and service-based startup (Check4Green). More details about each startup can be seen in Figure 6.

Although the startups offered different types of products and services, they shared some common aspects such as all three startups have been launched as a result of intrapreneurship activities, all startups had small to very small development teams, all of them targeted European markets, two of the three startups offered subscription-based services.

As a result of the feedback received after the delivery of the first product, one of the startups had to pivot in order to meet customer expectations and remain competitive. Another had to integrate the Scrum methodology in its development process prior to the implementation of the model while the other two were already familiar with the methodology.

Figure 6. Analyzed startups.

In order for the development team of the Check4Green startup to implement the proposed model, some aspects of the Scrum methodology required implementation. Therefore, the project manager assumed the role of PO, an SM has been assigned, the customer requirements have been gathered in a US form and the tasks have been divided to be completed in sprints by the DT.

For the case of the Check4Green startup, a number of five US have been selected to be completed in the course of three sprints, each with a planned Offset value of 33.33%. Figure 7 shows the results obtained after the implementation of the proposed model for the development process of the Check4Green startup.

The development team of the Check4Green startup was composed of six entry-level team members.

Figure 7. Check4Green QFD model implementation.

From the total number of 26 tasks to be completed in order to meet the customer demands, 16 tasks had interdependency relationships with one or more other tasks. For the first sprint, a number of 8 tasks have been chosen to be completed leading to an Offset value of 32%. The second sprint had 11 tasks assigned for completion, and the registered value of the Offset variable was that of 72%, meaning a sprint Offset value of 40%. For the third and last sprint, a number of 7 tasks were assigned for completion and the Offset reached the expected value of 100%, meaning the third sprint had an Offset value of 28%. The Offset can be seen as a means of assuring transparency for both the customers and the development team, regarding the amount of functionality that the product being developed can offer to the customers at any given point and the amount of work completed by the development team.

For the case of the Vitraly startup, a number of four US were selected, with a total number of 13 tasks, out of which 10 tasks had interdependency relations. Because there were tasks that contributed to multiple US, the actual number of unique tasks is 13. However, represented as number of tasks per US, the number of tasks totals 20 tasks, divided as 5 tasks in each US. A visual representation of the results obtained after the implementation of the proposed model in the development process of the Vitraly startup can be observed in Figure 8. The development team of the Vitraly startup cumulated five entry-level members.

Figure 8. Vitraly QFD model implementation.

The tasks were divided to be completed in the course of three sprints, each with an expected Offset value of 33.33%. Five tasks were selected to be completed in the course of the first sprint, which led to an Offset value of 29.8%; 4 tasks were completed during the second sprint, leading to an Offset value of 59.9% and a sprint Offset value of 30.1%; and 4 tasks were completed during the last sprint leading to an Offset value of 100% which matched the expected one, and a sprint offset of 40.1%.

In the case of the Lemur startup, due to the limited number of resources in terms of personnel, three US have been selected to be completed with a total of eight tasks, five of which were affected by interdependency relationships. Figure 9 shows the results of implementing the proposed model in the development lifecycle of the analysed startup, highlighting the way that the model worked for the pivoting point that the startup faced.

The development team of the Vitraly startup cumulated five entry-level members, out of which, only two remained after a three-month period.

Figure 9. Lemur QFD model implementation.

The tasks have been divided to be completed in the course of two sprints, each with an expected Offset variable value of 50%. This led to a number of two tasks being completed in the first sprint, and an Offset value of 49.99%, and the remaining six tasks have been completed in the second sprint, leading to an Offset value of 100%, and a sprint Offset value of 50.01%.

After the deliverables reached the customers, and based on their feedback, the development team decided to pivot and work on completing the most important US that derived from the pivot, which led to a total number of five tasks required to be completed out of which two were affected by interdependencies with one or more other tasks. The new tasks that required completion were divided into two sprints, each with an expected Offset value of 50%. Two of the five tasks were affected by interdependencies. For the first sprint after the pivot, the team completed two tasks, leading to an Offset value of 48%, while the remaining three tasks have been completed in the course of the second sprint and led to a final Offset value of 100%, and a 52% sprint value.

The proposed model proved to be useful for the development teams for multiple reasons among which: it enabled the teams to organize better and understand the role of each team member for the well-being of the project; it helped the teams understand customer requirements better and prioritize the requirements that needed immediate satisfaction; it allowed the DT to visualize the technical characteristics needed to be completed; it helped the DT in choosing and optimizing the task selection process for each sprint, based on the criteria dictated by the PO and SM based on the requirements of the customer; it ensured a customer-centered delivery of products and improved the delivery process with the help of the iterative development approach. In the context of the three analyzed startups, the customer was represented by the PO and it reflected the expected requirements of the target audience that the startups targeted.

5.1. Threats to validity

In the study, the threats to validity have been distributed into two categories: internal and external. The internal threats are related to the possible observer bias, given that one of the authors was part of the development teams of the analyzed startups, as well as the fact that more than half of the development team of one startup was part of the development team of one or both of the other startups.

The external threats to validity refer to the fact that the proposed model has only been implemented in the case of three startups, which shared more than half of the same development team built, and the fact that all of the analyzed startups were similar in dimensions (in terms of team members and complexity of their products). It would therefore be useful if the proposed model would be implemented and tested by various size startups in order to gain more feedback as to how it can be better adapted to startup requirements.

6. Conclusions and further enhancements

The model was tested with the help of its implementation for the case of the three analyzed startups, one of which was a product-based startup, another being a mix of product and service startup and the third being a service startup.

The model allowed the startups to better understand their customer’s requirements and plan the completion of the technical characteristics in order to deliver products in a timely manner. It covered the requirements of the analyzed startups in each stage of the development process, and furthermore, it adapted for the requirements of a startup that required a pivot in order to remain competitive, and a startup that required important updates and improvements to its products and services.

The model covers aspects regarding the project management and quality management tools and practices of IT startups and it could accommodate additional aspects and functionalities. A particularly interesting aspect to be considered would be the possibility of implementing a risk management framework or model developed specifically for the particularities of IT startups. This, in turn, would lead to a quality management approach in accordance with the requirements of the ISO 9001:2015 standard which puts an emphasis on risk-based thinking. Furthermore, the risk management model could help provide insights into the capabilities of the development team which in turn could lead to an adaptation of the QFD model, and the Offset value in particular, which would be recalculated to give an even more accurate estimation of the visible functionality that can be achieved taking into account the risks that the startup is exposed to, and/or must manage.

Further enhancements to the currently proposed model are based on the possibility of integrating a risk management framework in the model to help understand how various parameters could affect the output of the Offset and how to model decisions based on this.

Additional information

Funding

The authors received no direct funding for this research.

Notes on contributors

Raluca Dovleac

Raluca Dovleac is an Assistant Professor at the Management and Industrial Engineering department of the University of Petrosani. She holds a PhD degree in Engineering and Management, and her research is focused in the field of startups and agile methodologies, with an emphasis on quality management practices used by startups and within agile methodologies.

Andreea Ionica

Andreea Ionica is professor at the University of Petrosani, Department of Management and Industrial Engineering in Romania. She is also doctoral adviser in the field of Management and Engineering.

Monica Leba

Monica Leba is professor at the University of Petrosani, Department of Computer and Electrical Engineering in Romania. She is also doctoral adviser in the field of System Control Engineering.