Requirements gathering: the journey

Abstract

Requirements gathering in Information Systems is a critical part of any project, as any issues with the elicited requirements have an impact on the project as a whole and in some cases can lead to project failure. To address the issues with gathering effective requirements this paper proposes viewing the requirements gathering process as a human centred journey. The journey being critical to the requirements gathering experience of participants. This allows for a better understanding of the requirements gathering process and its effectiveness. This study helps with the decision-making process by assisting with engineering an effective requirements gathering approach which ultimately follows a design science approach and its objective is to build a prototype of a ‘requirements gathering journey’ canvas.

Keywords:

• Requirements gathering
• requirements
• elicitation
• effectiveness
• journey
• technique

Introduction

Information Systems (IS) project failure is a phenomenon which has been occurring for many years if not decades. According to the Standish reports, the success rate of software projects is only 28%. (Muqeem & Beg, 2012, p. 157). Only 61% of IS projects meet the requirements of the customer specifications, and 63% of projects exceed their budget estimations (Abugabah & Alfarraj, 2015, p. 2). The cost of these project failures is also extremely high, as high as 100 billion dollars in the US in 2000 (Browne & Rogich, 2001, p. 543). One major reason why projects fail in IS can be attributed to a lack of clear and specific information requirements (Byrd, Cossick, & Zmud, 1992, p. 118). Hickey & Davis (2003, p. 168) state that the success or failure of a system development effort depends heavily on the quality of the requirements. We can also see that problems with the elicited requirements can often be traced back to the technique used to gather these requirements (Tsumaki & Tamai, 2006, Davey and Cope 2008).

The purpose of requirements gathering (RG) is to ‘capture Information through the use of multidisciplinary views. Such views express what is to be built.’ (Christel & Kang, 1992, p. 34). RG can also be viewed as a coming together of different people’s perspectives and knowledge to generate a solution to their problem. According to Vijayan & Raju (2011) RG is both the ‘hardest and most critical’ aspect of any project. For a process that is so difficult, it is unclear why so little thought is put in to the technique that is used to elicit the requirements required to make a project successful. ‘Once a [RG] technique that does not fit the current project is selected, the project will proceed along the road to failure’ (Tsumaki & Tamai, 2006, p. 508).

When choosing a technique, the effectiveness of that technique is important, effectiveness in this instance refers to the ability of the technique to produce the actual requirements that are necessary to make the project successful. In past research effectiveness has been evaluated using a quantitative approach as can be seen for example in Dieste & Juristo (2011), Dieste, Juristo, & Shull (2008) and Hoffman, Shadbolt, Burton, & Klein (1995). Whilst this is one approach to determining effectiveness it does not take into account the project as a whole, it views techniques in isolation, and does not consider the context. To add this context, we propose viewing RG as a journey and ultimately as an experience. Context is important when it comes to determining technique effectiveness, as a technique that is effective for one journey is not necessarily effective on another.

In order to help understand this journey and experience, we propose a ‘Requirements Gathering Journey’ (RGJ) mapping canvas. This canvas will enable the visualisation of the RGJ so that it can be both understood in its entirety and analysed to draw insights about what techniques work and do not work. These insights, will be useful in understanding the current RGJ for a given project but it can also potentially inform decisions around technique selection for future projects.

This paper starts with providing a conceptual foundation and a discussion of the traditional RG landscape. Next we justify reconceptualising the RG process as a journey. The paper then goes on to talk about an approach to designing the RGJ canvas and the evaluation approach for the canvas. The next section presents the form of the resulting RGJ canvas including a RGJ ontology and an example of a completed canvas. This is followed by an explanation of the design principles behind the RGJ canvas. The paper ends with a brief conclusion.

The conceptual foundation

In this section, we start by discussing the current RG landscape, the importance of RG process technique selection and evaluation, and RG as a journey.

Traditional requirements gathering landscape

As we have seen project failure can be attributed to the quality of the RG process and ultimately the quality of the requirements elicited. ‘By improving requirements elicitation, the [RG] process can be improved, resulting in enhanced system requirements and potentially a much better system.’ (Christel & Kang, 1992, p. 1). There have been several variations of RG process models as can been seen in Hickey & Davis (2004), Zhang (2007), Sommerville (2005) and Christel & Kang (1992) and we feel they are all missing steps for RG process design, technique selection, and technique evaluation. Whilst Hickey & Davis (2004) in their model include technique selection, they don’t include RG process design or technique evaluation. RG technique selection is an important aspect as a wrong choice can have a very serious impact on a project. RG technique evaluation is also important in order to evaluate the ex-post effectiveness of the process.

When it comes to choosing a RG technique the elicitor should aim to choose the most effective elicitation technique - see Table 1 for a sample list of techniques. Technique selection is not an easy task and there are several ways that a RG technique can be selected. The most prevalent technique selection method is often based upon inertia and what worked well in the past for the elicitor or for the project manager on the project.

Table 1. Sample list of techniques.

Technique	Description
Surveys	Seeks answers to a list of questions presented to one or more informants (Carrizo et al., 2014, p. 668)
Prototyping	Focused on creating a simplified version of a proposed system (Carrizo et al., 2014, p. 668)
Open-ended interview	Involves face-to-face interaction between informant and elicitor where the goal and the line of questioning are neither structured or completely defined (Carrizo et al., 2014, p. 668)

Another RG technique selection method is the utilisation of frameworks developed in past research such as Carrizo, Dieste, & Juristo (2008) and Tsumaki & Tamai (2006). General project characteristics have been identified that point to the effectiveness or otherwise of particular techniques to elicit the most amount of requirements. Similarly in Tsumaki & Tamai (2006), their framework has focused on establishing which technique is most effective for a particular set of project characteristics.

Technique effectiveness has generally been measured in one way, and that has been to base it on the number of outputs, in this case the number of requirements that a particular technique has elicited (e.g. Dieste & Juristo (2011), Dieste et al. (2008) & Hoffman et al. (1995)). The effectiveness of a particular technique has generally focused on the number of requirements elicited with the assumption that a technique that delivers more requirements is more effective than one that delivers less requirements. When Hoffman et al. (1995) was undertaking analysis they also took into account the length of time it took to get the requirements and therefore defined effectiveness around the number of outputs that were generated in a period of time.

In summary the traditional evaluation of RG techniques effectiveness has been done using a quantified approach. What this means is that the evaluation of effectiveness does not take into account the context of the project and it views RG technique effectiveness in isolation of this context. The context of the project has been secondary and indeed has not been fully considered. Carrizo, Dieste, & Juristo (2014, p. 647) in response to this suggests ‘… that the important factor for technique selection is the context in which the technique is applicable.’

Reconceptualisation of the requirements gathering process as a journey

Some people associate RG with a journey as can be seen in (Hickey & Davis, 2003, p. 174) where an expert says ‘I suspect I would be on a long journey just to ascertain the project’s goals’. We continue this tradition and suggest that RG should be reconceptualised from a process as presented in the likes of Zhang (2007), Sommerville (2005) and Christel & Kang (1992) to a journey. The journey can be viewed as a number of phases. For example, ‘Classic software development follows a well-defined series of phases, traditionally called a waterfall model’ (Hickey & Davis, 2004, p. 66).

Across the phases of a project, different paths can be travelled, as can be seen in Hickey & Davis (2003, p. 172) where ‘when the interviewee told us stories, we carefully listened for aspects of the stories that might trigger particular elicitation paths.’ The idea of a path as part of this journey can also be seen in Chakraborty, Sarker, & Sarker (2010, p. 216) where the ‘[RG] process characterised as a path that transitions from an initial output characterised by opaque specification, informal representation, personal views to desired output characterised by complete specification, formal representation and common views’. These paths are important as they constitute the RGJ and ultimately the discussion that occurs as part of this journey. The discussion that occurs will be as a result of the use of different requirements elicitation techniques as can be seen in Laporti, Borges, & Braganholo (2009) and Davis, Dieste, Hickey, Juristo, & Moreno (2006).

Whilst RG is a journey it ultimately creates an experience including an emotional experience throughout the RGJ. This emotional journey will have a different level of impact upon a project depending on it being positive or negative. From Beaudry & Pinsonneault (2010) we can see that different emotions have different impacts on IT projects, the same can be said for the RGJ. As if someone was to have an unhappy experience during the RGJ, that person then will associate negatively with the RGJ, which may impact negatively on the requirements gathered. Likewise, if someone was to have a happy experience during the journey then they will associate positively with the journey, which may impact positively on the requirements gathered. Beaudry and Pinsonneault (2010, p. 698) state that ‘pleasure, the degree to which a user feels good or happy with a new IT system, has been found to be positively related to several antecedents of IT use’. In journey mapping it is clear that the ‘user experience tends to include wider human experience dimensions (such as pleasure, fun, and other emotions)’ (Nenonen, Rasila, Junnonen, & Kaernae, 2008, p. 1). What this means in terms of RG is that people are at the centre of the journey, and how they feel and interact during the RGJ will ultimately have a large potential impact on the effectiveness for the RG.

Customer journey mapping is extremely relevant in terms of understanding RG as a journey, as the similarities are striking. The complexities in RG are also evident in customer journey mapping where ‘… customer journeys are often complex, with multiple interactions taking place over extended timeframes’ (Cruickshank, 2011, p. 2). The complexities in the RGJ can be attributed to multiple different stakeholders, as well as the interactions with these stakeholders taking place over a long period of time. Viewing RG as a journey will ‘…provide a clear view of a customer’s view of how they interact with an organisation’ (Cruickshank, 2011, p. 2) What this means for the RGJ is that it is easy to see what people thought of the journey. The viewing of RG as journey, similar to customer journey maps allows us to ‘deliver a visual representation of the customer experience, allowing gaps to be quickly identified … in a simple and effective format.’(Cruickshank, 2011, p. 2). It essentially helps us arrive at a more effective RG journey based on the experience of those involved.

The approach to designing the canvas

In this section we talk about the research approach to designing and evaluating the canvas.

Approach

Design science ‘… addresses research through the building and evaluation of artefacts designed to meet the identified business need’ (Hevner, March, Park, & Ram, 2004, p. 79). Design science is the chosen approach as ‘Good design science research often begins by identifying and representing opportunities and problems in an actual application environment’ (Hevner, 2007, p. 3). The problem in this case is the RG process and the opportunity has been to build a canvas to present RG as a journey to address the aforementioned problems. This paper also seeks to apply existing academic and practitioner knowledge to extend the current knowledge base. We seek to move towards an artefact that ‘… must enable the solution of heretofore unsolved problems. It may extend the knowledge base … or apply existing knowledge in new and innovative ways.’ (Hevner et al., 2004, p. 87). This study also seeks to set out a set of design principles upon which this artefact is being built. ‘Furthermore, because the authors are able to articulate the design principles upon which its construction was based, these serve as hypotheses to be tested by future empirical work’ (Hevner et al., 2004, p. 97).

Evaluation approach

The need for evaluation in design science is critical. Knowing how and what to evaluate is not necessarily easy. Hevner et al. (2004, p. 85) mention several different criteria against which to evaluate an artefact such as ‘functionality, completeness, consistency, accuracy, performance, reliability, usability, fit with the organisation, and other relevant quality attributes.’ These criteria are useful in that they give some guidance and have been used to evaluate the artefact to date. Two forms of evaluation - ‘Expert Evaluation’ and ‘Subject-based evaluation’ - have been used (as identified by Peffers, Rothenberger, Tuunanen, & Vaezi (2012, p. 402)). In undertaking the evaluation, we have used both low and high fidelity prototypes – see Table 2.

Table 2. Artefact evaluation.

Artefact	Design/Build	Evaluation	Desired Outcome
Low Fidelity Prototype	Designed and built a low fidelity prototype using post-it notes to see if the requirements journey could be visualised	Evaluated through Expert evaluation using a 1-h show and tell. The purpose of this evaluation was to evaluate the artefacts completeness and usability	The desired outcome of this evaluation was to see if it was possible to visualise the RG journey
High Fidelity Prototype	Designed a high fidelity prototype incorporating the feedback received from the evaluation	Evaluated through Expert evaluation and subject based experimentation	The desired outcome of this evaluation was to visualise the RG journey and receive feedback around the artefacts completeness and usability

The form of the canvas

In this section, we detail an ontology of the RGJ, as well as a later version of the completed RGJ mapping canvas.

Requirements journey ontology

‘An ontology provides a vocabulary of terms and relations with which to model the domain’ (Studer, Benjamins, & Fensel, 1998) The ontology in Figure 1 presents an outline of the terms identified from the literature and from observations during the evaluation of the canvas.

Figure 1. Requirements gathering journey ontology.

In Figure 1, we can see that there are four main elements, ‘user’, ‘timeline’, ‘journey’ and ‘techniques’. These elements are at the core of the proposed canvas and within these, there are further sub elements which correspond to the parent element. These sub elements also form part of the canvas. A description of all elements can be seen in Table 3.

Table 3. Requirements journey ontology explanation.

Term	Description
User	Describes the person taking part in the RGJ
Timeline	Describes the overall duration of the RGJ, from start to finish
Phase	Describes the different distinct stages within the timeline, with each phase being a distinct part of the timeline
Event	Describes the different activities within each phase, e.g. workshops, meetings, etc.
Journey	Describes the passage through the RG process
Discussion	Describes the depth of conversations that take place throughout the journey
Communication	Describes the scale of conversations that take place throughout the journey
Personal Feeling	Describes how the person felt throughout the journey
Understanding	Describes the level of understanding that occurs throughout the journey
Techniques	Describes the different techniques that are used throughout the RGJ. Under techniques there are also a sample of different technique categorisations used by (Dieste & Juristo, 2011)

Completed requirements gathering journey canvas

In Figure 2 we can see a completed RGJ mapping canvas. This canvas is physically (and conceptually) divided into a number of areas which corresponds to a matrix style grid. Horizontally across the top of the canvas we have a timeline, which is representative of the RGJ. Vertically down the left hand side we can see that there are two distinct areas, the top half of the canvas representing the RGJ, and the lower half representing the technique(s) that were used as part of that journey.

Figure 2. Completed requirements journey canvas.

The matrix grid is made up of the timeline being segmented by different phases of the journey vertically, while the journey and the different techniques used segment the matrix horizontally. Horizontally across the top of the canvas we can see ‘phases’ in Figure 2 for example we can see 4 phases (the number of phases is project dependent). Underneath these phases we can see ‘events’. These events help to create the matrix as they are more granular than the higher level phases which are part of the overall timeline. On the vertical axis we can see different headings which correspond to different distinct areas of the canvas.

The first distinct area is ‘Journey’ as can be seen in Figure 2. This area is used to represent different ‘Journeys’ through the RG process which are to be drawn as trend lines. The next distinct area that can be seen is the grey colour graded area of the canvas. These boxes are used to represent the technique(s) that were used during the RG process, a sample list of techniques can be seen in Table 1. The matrix works both horizontally and vertically, on the vertical axis we have the names of the techniques that have been used and on the very top of the canvas, horizontally we have the phases/event that the technique used corresponds too. When using the canvas, the corresponding cell is where the user would enter the appropriate information.

In Figure 2, we can see that there are circular balls with numbers in the cell next to where the technique use has been identified. The number between ‘1’ and ‘3’ is represented of the user’s perceived effectiveness of that techniques use. The numbers being representative of low (1), medium (2) and High (3).

In Figure 2 we can see a completed canvas that represents a project that the main author has been involved in. This has been completed through the analysis of a RGJ of a research project in the connected health space, which has multiple stakeholders and multiple project partners. From undertaking the activity of completing this canvas it has become clear that there are multiple different views of the same RGJ. This difference in perspective can make this modelling exercise invaluable as it promotes reflection and analysis of what happened during the journey.

Why the canvas works

The design of the RGJ mapping canvas has been based on an ontology (refer Figure 1) and guided by design principles. The design principles that the RGJ mapping canvas have been based upon are described below.

Design principles for the canvas

The principles set out below are at the core of the canvas. These principles are: timeline, the user, communication, understanding, discussion and personal feeling. These principles have come mainly from past research as well as practice and are the design principles utilised in the canvas. These are used in the canvas as there impact on the RGJ is important, as can be seen from both previous research and practice. On the canvas some of these are represented using trend lines whilst others are the foundation of the canvas itself. Whilst there are many more potential principles and trend lines which could be included these are the ones that are core to the RGJ.

Timeline

The representation of the journey as a timeline is a core part of the canvas. The journey is to be represented as a timeline, as naturally the flow of the project will be segmented into different time blocks. These time blocks will have particular events that will occur at different times throughout the journey. E.g. requirements workshops

The role of the user

The user is the single most important aspect of the journey. The user is any stakeholder that was involved in the journey. The role of the user is a core part of the canvas as it is from the user’s perspective that the canvas is to be filled in and this is the basis upon which it is to be evaluated. The role of the user is also a guiding design principle for the canvas as it will be filled in and evaluated by the user.

The role of communication

Communication is important as poor communication has an impact on the project as a whole as a ‘Savant Institute study found that ‘56% of errors in installed systems were due to poor communication between user and analyst’ (Christel & Kang, 1992, p. 10).

Communication is also important in the RG process in that it is a common challenge most projects have during the elicitation process ‘The common challenges that analysts face during elicitation process are to ensure effective communication between analyst and the users.’ (Vijayan & Raju, 2011, p. 9). Whilst communication is important, it’s the scale of communication that is important for the artefact.

The role of discussion

Discussion is important, as the amount occurring through the RG process varies widely throughout the journey as can be seen in Laporti et al. (2009, p. 377) where ‘Since people usually do not discuss in a linear form, new interactions may lead to discussions.’ Discussion will occur due to the use of particular techniques (refer Table 1 for sample list of techniques). The amount and depth of discussion arising from technique use will vary however as can be seen in Davis et al. (2006, p. 184) where they have found that ‘the use of visual aids or prototypes focuses the discussion on the displayed artefact.’

The role of personal feeling

Personal feeling is important, as the RG process is a journey. At different times throughout the journey, people will feel different things which may have an impact on the RG process as a whole. People drawing a trend line of their personal feeling journey gives a unique insight into how a person felt at a particular time during the journey. This is important as it allows us during the analysis process to view the RGJ from a different angle, that of putting the person first rather than the project which is akin to user-centred design where the user is put first, should the people undertaking the RG process not be put first as ‘It is necessary to think carefully about who is a user and how to involve users in the design process. Obviously users are the people who will use the final product or artefact to accomplish a task or goal.’ (Abras, Maloney-Krichmar, & Preece, 2004, p. 4).

The role of understanding

Understanding is included as the level of understanding of requirements has a direct impact on the success of the RG process and thus the project as a whole as can be seen in Christel & Kang (1992, p. 10) where ‘Problems of understanding during elicitation can lead to requirements which are ambiguous, incomplete, inconsistent, and even incorrect because they do not address the requirements elicitation stakeholders’ true needs.’. Understanding is also included as ‘… most problems in software development stem from a poor initial understanding of the customers’ needs.’ (Geisser & Hildenbrand, 2006, p. 109).

Understanding is a very important inclusion, because the more ambiguity there is the more likely there is to be incorrect assumptions made by people. ‘Due to the problems of understanding and scope discussed earlier, user needs may not be clearly expressed initially in the requirements, and the developer or requirements analyst may make some incorrect assumptions based on this ambiguity.’ (Christel & Kang, 1992, p. 13). Also we know that during the RG process there will be a certain level of ambiguity around what is required from the project at different times ‘Requirements elicitation is a difficult process in which one has to deal with ambiguity, informality, incompleteness and inconsistency, in which the ‘knowledge’ of the requirements is not clear.’ (Vijayan & Raju, 2011, p. 10).

Conclusion

In this paper we proposed the idea of RG being viewed as a journey with consideration of the associated experience of those involved. To help understand this journey and experience a RGJ mapping canvas and supporting ontology has been proposed which allows the RGJ to be visualised in order to analyse techniques to determine their impact on the effectiveness of the overall journey. This analysis of the RGJ will have the benefit of providing a project context for the analysis but can also be used to inform decision-making around elicitation technique selection on future projects.

Disclosure statement

No potential conflict of interest was reported by the authors.