Requirements Engineering and Origami

You might have seen the pictures of green swans on our twitter feed from time to time. You have probably been wondering: What the heck is going on at Qualicen? How is this related to Requirements Engineering?

We know from pedagogics that people learn best from their own experience. We apply this in our courses all the time. Specifically, we let people experience many engineering pains with little experiments. One of these is an experiment that was originally designed by Magne Jørgensen, a Norwegian expert on software engineering estimations. Read more about estimations in Magne Jørgensens and Torleif Halkjelsvik’s great Open Access Book Time Predictions: Understanding and Avoiding Unrealism in Project Planning and Everyday Life.

How does our experiment work?

• I give a 5-10 minutes task to a group of people. Usually, I take folding origami flowers or swans as a task. For this, I hand out detailed instructions to fold a specific origami figure, for example, this one:
• I start a bidding process and ask my participants to make a secret, independent bid on a small piece of paper: How long will it take you to fold this?
• The lowest bid gets the chance to make money: For every 30 seconds that the winning bid is faster than their own prediction, that person wins $1. Of course, there must be also penalty: For every 30 seconds that the winning bid is slower than their own prediction, that person loses $1.
• I remind everyone that they can ask as many questions as they want.
• I tell the audience that I can see into the future and write down three things that are going to happen next. To formally document my visionary skills, I give a secret note to one of the participants. In the end, I’m a show master, I guess…

Next, people are looking through the instructions. Some people start taking out a pen and adding up the costs of each step. Some go by gut feeling.

Problems visible right away…

Right away, I see three common mistakes happening almost every time:

1. Participants ask no or only very few questions. It is such a simple task, right? Plus my colleagues around me are listening…
2. Participants calculate time per step and add up the numbers without adding buffers for mistakes and reading up on previous or next steps. But, as Magne and Torleif say (in another context): 2+2 is usually more than 4.
3. Participants try to make a prediction without trying the individual steps. Everything is simple in theory. But you only see the challenges in the details when you go forward.

What’s happening next?

I then collect the bids from the participants and line them up. This creates a nice tension in the room…

I’ve done the experiment with as few as four participants. I promise: You will always have that one person that estimates that he is going to solve the problem in three minutes. I’ve even had someone promise to do it in less than 30 seconds! They try folding the origami. Meanwhile, I pull out my three predictions (that I also stole from Magne):

1. An inexperienced person will win the bid: As they write in their book: “If a client selects someone based on low time predictions or price, they may select those with the least relevant experience, because the providers with the lowest bids may have forgotten to include essential activities in their time predictions.”
2. The winning bid will not make it on time: People force themselves to make a prediction without looking into the details. They do not ask questions, they are not used to trying to understand the context. In this case, if people don’t ask me, I hand them a rectangular instead of a square paper. And right away, people won’t be able to make the bid. This is because in a competition situation, people tend to estimate optimistically instead of realistically. And again: Because I’ve chosen the lowest bid, I most probably took someone who didn’t make space for unexpected challenges.
3. The quality of the result will be very poor: Well, as a consequence of the over-optimistic bid, the result always looks something like this:

(Yes, this is a flower! This was a result of me even knowing about the aforementioned problems…)

How is this related to Requirements Engineering?

This is very common in requirements engineering: We’re often asked to estimate projects based on requirements documents. If the documents are bad, the estimations will suffer. And even if they are awesome (for example, if they are created by Qualicen),  there is still no guarantee for success.

In the experiment, we see how missing knowledge, human habits, and a bad selection process leads to overoptimistic predictions. After having conducted the experiment close to 20 times, it “failed” only once.

In short, what we see is how quickly missing information leads to project failures. In addition, this shows that just thinking hard (i.e. Requirements Engineering) is not enough. We have to combine RE with prototyping in order to create good predictions.

TL;DR

• We design our courses in a way that you can experience the pain of the real problem.
• When estimating, you have to challenge the requirements and ask plenty of questions.
• Taking the lowest bid leads to increased risks for overestimations.