What can you do to get a good first understanding of requirements?

Disclaimer This was an essay assignment from my “Requirement’s engineering” course at the University of Amsterdam (SE Master’s). I’ve thought to leave it here. Maybe it helps someone. Date of the assignment was around December 2023.

Introduction

When gathering requirements, in an ideal world, we would only have one stakeholder who will spill all their wisdom over a requirements engineer without any ambiguity. The project team would exactly know how the future system should look like and with which steps they need to proceed. The hardest part, then, would probably be the coding. However, “the hardest single part of building a software system is deciding precisely what to build.” [Brooks, 1987]

Therefore, basically, to come up with a suitable specification of a system. In no other stage of product development the ambiguous and noisy human factor is so present as it is in the requirements elicitation phase. Moreover, “no other part of the work cripples the resulting system if done wrong”. [Brooks, 1987].

The above indicates that gathering requirements is not a straightforward task but needs a sharp eye and a cautious work ethic. Requirements are “not flying around” somewhere in the world, simply to collect. Furthermore, it is not even a process which can be completed in one session, as stakeholders are rather keen to propose their dreams or agendas to requirements engineers instead of the actual requirements. [Dekkers, 2023]

There are many definitions of what requirements could be. This essay will see them as “pre- scriptive statements about software functionalities, qualities and development constraints” [Lamsweerde, 2009].

The following will explore the possible components that should help gather a good first understanding of requirements in an upcoming project. “Good understanding” is a hard term do define, because how do you measure the abstract term “good”? To establish a common ground and have an indication, this essay is going to see “a good understanding”, if a person can describe a system clearly, accurately and in unambiguously terms (for example by using appropriate models).

Explore the Domain

Gather the Sources

An integral part of the requirements elicitation process is understanding the domain the system-as-is and the system-to-be are operating or will operate. This is because one essential property of requirements is that they lie inside a domain, thus making it the essential starting point of a requirements engineer’s journey. Moreover, one needs this knowledge (defined as the “potential to influence action” to understand the problem, its roots, and the goals of a current system or stakeholders [Lamsweerde, 2009, cf 1.1.6 The requirements lifecycle: Processes, actors and products].

When trying to understand the system-as-is, it is essential to gather sources, as these are the basis for further work. Sources can be understood as “A place where there is relevant knowledge for the project” [Dekkers, 2023, cf Lecture Notes]. These sources are not limited to “human” stakeholders (people or organisations with interest and influence in a system-to-be, [Lamsweerde, 2009, cf 1.1.6 The requirements lifecycle: Processes, actors and products]), but can also be gathered from processes, physical environments, or the outside world which is not part of a system.

At this stage, it is also crucial not to rely on assumptions, but rather on hard facts. For instance, numbers and testing or validation if a proposed problem is really a problem. A personal example which went drastically wrong during a case study: The primary stakeholder kept saying there was a problem with customer support, so users did not know where to ask for help and were not using the system as intended.

After weeks of stumbling around with tunnel vision and focusing too much on the customer support issue, it turned out that the user interface was overloaded with information and intimidated the users. Analysing it further, it became clear that the users did not achieve the ability during an initial “frontal-lecture” style onboarding to grasp how to use the system.

Speaking about tunnel vision and how to mitigate it, it is useful not only to consult the supporting party of a system-to-be but also to look out for “rebels”, “visionaries” and experts who might have a completely different view on the proposed changes. However, it is also good to talk about less skilled people in the domain as “Domain knowledge improves knowledge elicitation, particularly in interviews. However, it also has disadvantages, as domain expertise may encourage tacit knowledge omission” [Sutcliffe and Sawyer, 2013, cf. Introduction].

Debating on each other’s viewpoints encourages to deal with opposite arguments and to be actively open-minded [Kahneman et al., 2021, Chapter 21 –Noise and Biasing in Forecasting].

Furthermore, it can point requirement engineers in different directions, which could be explored further.

Last but not least, during this phase, one must also consider the constraints and policies a particular system is bound to, as these will become important when evaluating possible scenarios. [Lamsweerde, 2009, cf. Chapter 1 – Domain Understanding]

Regarding human sources, interviews, questionnaires, workshops and other common require- ments engineering methods are a good starting point. However, engaging with the people in their day-to-day interaction with the system can also be useful, for instance, in a so-called “apprenticeship” interview. In an apprenticeship interview, a master-apprentice relationship is established. The requirements engineer takes the role of an apprentice (someone who wants to learn something from a master) and tries to understand why the master works with a system-as-is the way he does [Beyer and Holtzblatt, 1995]. Here, a requirements engineer or an analyst shall not use custom defined “objective”-categories, but rather listen to the people and use their “categories [which] they use implicitly to communicate” [Siddiqi, 1996, cf. WHAT CONSTITUTES A REQUIREMENT?].

However, this type of interview activity is not always applicable.Regarding the case mentioned above, the case team needed to use a slightly derived version of the apprenticeship method, as we were attending onboarding with real clients and the primary stakeholder. Therefore, the primary stakeholder would not have been amused if we had disturbed his interview with questions. Instead, we asked to reflect on the process directly after the onboarding. We tried to get the reasoning for certain actions by asking for argumentation, which builds the core part of this type of activity [Beyer and Holtzblatt, 1995, cf. Seeing the work reveal what matters].

To sum it up, at this stage, one should act like a sponge, trying to get everything one can regarding information. Stakeholders should be used as guides for other sources, for instance, by asking for a mandate from other people or “facts” [Dekkers, 2023, cf. Lecture Notes]. It is crucial not to fall into the pit of tunnel vision. However, it still keeps one’s focus on the problem. The outcome of this stage should be an understanding of the domain in which a problem is rooted and what the roots of a problem could be [Lamsweerde, 2009, cf. Domain Understanding].

Deal with Problems

Tacit Knowledge

After the given examples, it would be naive to assume that there will be no problems or challenges during the domain exploration. To explore this point more, we need to understand the types of knowledge we can deal with in exploring domains. What a requirements engineer should keep in mind during the elicitation process are the types of knowledge he could potentially encounter during his journey. For the sake of completeness of data, it is helpful to be sensitized to the categories.

In their paper [Sutcliffe and Sawyer, 2013] categorize knowledge from an analyst perspective in an analyst/user-dialogue into the categories1 of “known knowns” which is the knowledge that is expressible, articulated and relevant. Known knowns are considered to be the easiest to elicit, as stakeholders will talk about it without problems.

“Known unknowns” is knowledge which is potentially accessible but not articulated. In other words, it is knowledge an analyst is aware of (so that it is needed), but it is tacit knowledge to the user. This type of knowledge can arise if a stakeholder drops a cue during an interview that needs further investigation, for example, if a user talks about a routine.

Often overseen but crucial in terms of requirements engineering are the so-called “unknown unknowns”, which is the knowledge that is “not expressible, articulated or accessible but still potentially relevant”. Unknown unknowns can be a result “of [a lack of] domain knowledge on both sides or inadequate design exploration so the user-stakeholder is simply unaware of the technical solution possibilities.”. A requirements engineer should not underrate it and plan additional time trying to explore these unknowns, possibly with the stakeholder.

Currently, there is no “main” or most promising methodology to elicit tacit knowledge. Never- theless, [Sutcliffe and Sawyer, 2013] propose to broaden the knowledge about the stakeholder and find out their background to possibly get insights about their cultural, political or skill views. Alternatively, exploration of possible requirements by observing the interaction from the user with the system (as also mentioned by [Beyer and Holtzblatt, 1995]) because “the combination of contextual long-term observation and interviews produces rich contextual descriptions” is promising. Whether the application of unknown unknowns wil be successful, depends on the sample size, diversity or duration of interviews. The categories mentioned earlier imply that gaining trust as an interviewer during the elicitation process is so important because one enhances one’s chances of getting closer to the tacit knowledge of a stakeholder. For example, knowledge might not be articulated because it might be “suppressed for political, social or emotional reasons”. Alternatively, suppose requirements engineers do not get enough trust from a stakeholder. In that case, stakeholders might not even let them observe them during their interaction with a system (which we learnt is one of the most promising techniques to elicit tacit knowledge). In the case accompanying this essay, the team was not able to get so much trust from the client that we never got a mandate to talk to the actual users of the system. It remained and known unknown to us. The team knew there were issues, but we could not gather them since we were not allowed to go into interaction. To mitigate the risk of unknown unknowns, although they will still be present there, is to get a good grasp of the domain and the existing solutions. Furthermore, it also stresses why it is so important to consider many angles of a given domain, especially to identify blind spots.

The Human Factor

After listening consciously to the interview partner, reading through reports and analyzing other data, requirements engineers have, in the best case, plenty of information they need to digest and make sense of it. This leads to one of the main problems in requirements engineer- ing: The huge amount of information. Therefore, when revising the sources, a requirements engineer should turn on his critical thinking and personal filter.

In particular, humans are not the best species when it comes to telling the truth, which makes the requirements engineering process hard. This has nothing to do with maliciousness but rather the nature of our brain. Humans try to reduce their cognitive load by reducing the information they need to interact with.

For instance, biases play a significant role, which can be introduced from people or the environment. A typical example is the conclusion bias, where people jump to conclusions and do not consider other facts because they do not want to accept another worldview. Underlying principles for that is excessive coherence, that humans are relatively fast in creating first assumptions but slow in changing them, or base neglect, where people “fade out” the importance of the whole data set in favour of their argument [Kahneman et al., 2021, cf. Chapter 13].

An even more realistic example which could impact you as a requirements engineer: Have you ever just assumed that one person is competent or telling the truth just because they looked attractive? Then you have fallen for the so-called Halo Effect.

However, how can critical thinking be applied during a elicitation session? In general, by trying to activate the analytical thinking of the interviewees. Asking to elaborate on content or a statement. For instance, when someone says, “We need this knowledge system”, ask, “Why?” or if the people have tried anything else.

Scepticism goes hand in hand with being actively open-minded, always in the state that your judgment about a particular situation is incomplete. Implement the characteristics of so-called “superforecasters” who “careful thought and self-criticism, the gathering and synthesizing of other perspectives, the granular judgments and relentless updating.” They like a particular cycle of thinking: “try, fail, analyze, adjust, try again.” [Kahneman et al., 2021, Chapter 21 – Perpetual Beta].

Scepticism in evaluating the given information is often seen as negative. How can a require- ments engineer dare to question a compelling vision that, for example, the sales manager provided? Nevertheless, “Scepticism is not pessimism” as scepticism enables us to gather more accurate and valid data to build our assumptions and requirements out of the domain and prevents us from making mistakes by finding blind spots in the given information.

Bring it down to paper

After having an overview of the high-level or abstract goals of a system and general re- quirements, which are consolidated with the stakeholders, an important step is also the documentation of them as it helps to create transparency for all project parties. According to Lamsweerde, a requirements document can be considered “good” if it meets mul- tiple properties, such as “unambiguity”, “measurability” or “pertinence” [Lamsweerde, 2009, cf .1.1.7 Target qualities and defects to avoid].

There are more. However, even if only focusing on these three, one can understand how documenting the first draft of requirements can help to think about the last findings and potentially refute them again. With this activity, requirements engineer may see contradictions or spots where they should readjust their focus. After exploring the domain and talking to people, there is a version of all the previous “learnings”.

When documenting requirements, requirement engineers shall ask themselves how the re- quirements relate to the system goals. If this document is unclear, how should the future developed system be clear for the customer? In sum, the requirements document presents the requirements and the concise understanding of a requirements engineer towards the system.

Develop your scenarios

“There is no silver bullet” to fix problems in software engineering [Brooks, 1987]. Therefore, there won‘t be only one possible solution for your system-to-be. By creating possible scenarios based on the previously declared requirements, one might refute previous claims or get other ideas, as with each thinking, you might get new associations.

One important point, though, when creating scenarios is that when deciding whether to build a new software system from scratch or not “The most radical possible solution for constructing software is not to construct it at all”. Evaluate all existing solutions-(components) on the market (the ones that succeeded, but also the ones which failed) before making a final decision. Relying on existing software “is much cheaper than [building] a fresh [one]. […] And delivery is immediate” [Brooks, 1987].

Furthermore, when creating scenarios, back them all up with your previous knowledge and reasoning (especially the relation to goals), as an analytical view of things can enhance decisions made later. To evaluate your solutions, you could get independent views on them (for example, from stakeholders).

Now what?

After exploring the domain, understanding the goals, documenting first requirements and developing potential scenarios, a good first understanding of the requirements and the system- as-is should be given.

However, the requirements journey is still ongoing. The emphasis of this essay lies on the basic components of a “good first understanding” as requirements engineering can be seen as a spiral process that begins by understanding the domain, evaluating and negotiating with the stakeholders, specification and documentation, and consolidating requirements. These steps do not have to appear in sequence but can go back and forth. For instance, during the documentation, it appears that a requirement contradicts the properties of the domain, so one needs to re-evaluate it and so on [Lamsweerde, 2009].

Furthermore, documented requirements will likely be incomplete the first time gathered, as it is “virtually impossible to make all the correct requirements and implementation decisions the first time around” [Brooks, 1987]. A requirements engineer can‘t avoid that, but he can cope with it. The involved project team and the stakeholders should discuss a level of accepted incompleteness. After the establishment, techniques and appropriate stopping conditions should be found or created. Again, to avoid tunnel vision.

After having a good first understanding of requirements and multiple scenarios for the “system-to-be” are created, requirements engineers should avoid falling into the pit that they’d be done with their job. “All successful software gets changed”, for instance, because the targeted domain, law, or technology might change. [Brooks, 1987] Created scenarios must be evaluated constantly and potentially adjusted. However, a requirements engineer should not try to fix a performance issue until there is one.

Last but not least, there is not only a “system-to-be” in the requirements engineer process but also a “system-to-be-next” (a future system) [Lamsweerde, 2009, cf. the systems-to-be-next].

Keeping this in mind and having a good understanding of requirements while creating a solution for a system-to-be can save time and money in the future.

References

[Beyer and Holtzblatt, 1995] Beyer, H. R. and Holtzblatt, K. (1995). Apprenticing with the customer. Communications of the ACM, 38(5):45–52.

[Brooks, 1987] Brooks(1987). No Silver Bullet Essence and Accidents of SoftwareEngineering. Computer, 20(4):10–19. Conference Name: Computer.

[Dekkers, 2023] Dekkers, H. (2023). Lecture 2 – lecture: Starting the project.

[Kahneman et al., 2021] Kahneman, D., Sibony, O., and Sunstein, C. R. (2021). Noise: a flaw in human judgment. Little, Brown Spark, New York. OCLC: 1255402641.

[Lamsweerde, 2009] Lamsweerde, A. v. (2009). Requirements engineering: from system goals to UML models to software specifications. John Wiley, Chichester, England ; Hoboken, NJ. OCLC: ocn243941685.

[Siddiqi, 1996] Siddiqi, J. (1996). Requirement engineering: The emerging wisdom [guest editor’s introduction]. IEEE Software, 13(2):15.

[Sutcliffe and Sawyer, 2013] Sutcliffe, A. and Sawyer, P. (2013). Requirements elicitation: To- wards the unknown unknowns. In 2013 21st IEEE International Requirements Engineering Conference (RE), pages 92–104. ISSN: 2332-6441.

[Van Lamsweerde, 2000] Van Lamsweerde, A. (2000). Goal-oriented requirements engineer- ing: a guided tour. In Proceedings Fifth IEEE International Symposium on Requirements Engineering, pages 249–262, Toronto, Ont., Canada. IEEE Comput. Soc.