3-value Boundary Value Analysis: Misconception and Reality

Not long ago, I encountered some examples of the 3-value Boundary Value Analysis (BVA) provided by various tutorials and community websites. Surprisingly, I found many cases where the supposedly “correct” answers did not implement the 3-value BVA accurately, either leading to missing test cases or resulting in exhaustive testing.

In this article, I aim to explain why misconceptions about the implementation of the 3-value BVA exist. By critically analysing the ISTQB Foundation syllabus and providing practical examples, we can better understand the correct approach to implementing the 3-value BVA.

Defining the Test Object

Let’s start by defining our test object: a password input field.

• It will deny passwords that are shorter than 6 characters or longer than 16 characters, displaying an error message

• For passwords 6–10 characters long, it will display the message “fair enough”

• For passwords 11–16 characters long, it will display the message “excellent”

Equivalence Partitioning

First, we need to identify the equivalence partitions and their boundary values. The ISTQB Foundation mentions the following about the equivalent partitions:

And about their boundary values:

Based on the above, the equivalence partitions identified for our test object are:

Where ? can be the maximum characters allowed to be written in the input field, or the max value of the integer type (e.g. for Java is Integer.MAX_VALUE)

2-value BVA

Before proceeding to the 3-value BVA, we’ll start with the simpler and indisputable 2-value BVA. The ISTQB Foundation provides the following information about it:

Based on the above, we have identified the following coverage items:

In terms of test cases, we will have the following:

So, in 2-value BVA the coverage items are the boundary values of each partition.

3-value BVA: The Misconception (?)

Now, let’s try to apply the 3-value BVA. Once again, the ISTQB Foundation provides the following information about 3-value BVA:

Based on the above, we have identified the following coverage items:

Let’s simplify the above table and generate our test cases:

It is odd, right?

I guess that I’m not the only one who thinks that the above solution is quite odd. I imagine my OCD friends would already feel uneasy about the fact that for each boundary with the 3-value BVA we result in 4 coverage items.

On the other hand, the following example taken from the ISTQB Foundation, implies (?) that with the 3-value BVA we should only have 3 coverage items:

Which is correct?

Consider this question: Which values should we pick as coverage items for the boundary 10|11?

a. 9, 10, 11

b. 10, 11, 12

c. 9, 10, 11, 12

I feel that most people would unconsciously choose the answer a. But after reading this, they would choose the answer c, but still they wouldn’t be fully convinced.

In my opinion, this confusion arises because the above ISTQB Foundation example is incomplete and requires further explanation to be fully understood.

Let’s think of it

In the ISTQB example, the decision “if (x ≤ 10)” represents the upper boundary of the 6–10 partition. If we examine this boundary in isolation, we should test the values 9, 10 and 11:- For 9 and 10 the test object has the behaviour “fair enough”- For 11 the test object has the behaviour “excellent”

But what about the lower boundary of the adjacent partition (11–16)? Since the decision “if (x ≤ 10)” exists, there must be another decision block, like “else” or “else if”. That decision block can be tested by the values 10, 11 and 12:- For 10 the test object has the behaviour “fair enough”- For 11 and 12 the test object has the behaviour “excellent”

At first glance, it seems logical that to efficiently test the boundary 10|11 from both sides, we should select 9, 10, 11, and 12 as coverage items.

Let’s dive deeper

In terms of coding, the boundary 10|11 can be expressed in various ways, but eventually all summarise into these options:

• Expressions that can be used to express the left side of the boundary 10|11 (partition “Fair enough”):

passwordLength <= 10
passwordLength < 11

• Expressions that can be used to express the right side of the boundary 10|11 (partition “Excellent”):

passwordLength > 10
passwordLength >= 11

Errors, errors, errors…

How many errors can someone do at the above logical expressions? If you think about how many ways there are to set things right, well, there’ll always be a whole lot more ways to muck it all up!

At the above logical expressions, the errors can occur:

• at the comparison operators (<, <=, >, >=, ==)

• at the boundary values (e.g. using 1 value below/above)

• a combination of the above

So, let’s rewrite the above logical expressions containing all the possible errors. At the same time, let’s evaluate which of the coverage items (9, 10, 11, 12) would catch the bug.

6–10 partition upper boundary

As we already said, this boundary is expressed with one of the following logical expressions:

• length ≤ 10

• length < 11

So for the coverage items 9, 10 the above logical expressions should evaluate to TRUE, and for the coverage items 11, 12 to FALSE.

Let’s see in action how an error at these expressions relates to the coverage items. The tables below show different cases of possibly wrong implementations and the test results.

Cases of wrong implementation of the expressions length <= 10 and length < 11 and comparison of expected/actual results for the coverage items 9, 10, 11, 12

T: expression is true for the coverage item

F: expression is false for the coverage item

*/*: expected result of the coverage item for the original expression / actual result with the wrong expression

V: the error occurs at the value of the logical expression

O: the error occurs at the operator of the logical expression

only for length <= 10

only for length < 11

common for length <= 10 (A) and length < 11 (B)

11–16 partition lower boundary

As we already said, this boundary is expressed with one of the following logical expressions:

• length > 10

• length ≥11

So for the coverage items 9, 10 the above logical expressions should evaluate to FALSE, and for the coverage items 11, 12 to TRUE.

Cases of wrong implementation of the expressions length > 10 and length >= 11 and comparison of expected/actual results for the coverage items 9, 10, 11, 12

T: expression is true for the coverage item

F: expression is false for the coverage item

*/*: expected result of the coverage item for the original expression / actual result with the wrong expression

V: the error occurs at the value of the logical expression

O: the error occurs at the operator of the logical expression

only for length > 10

only for length >= 11

common for length > 10 (A) and length >= 11 (B)

Understanding the results

From the results above we see that:

For the 6–10 partition upper boundary:

• Coverage item 9: It’s the only one that catches the case 18

• Coverage item 10: It’s the only one that catches the cases 1, 5, 15

• Coverage item 11: It’s the only one that catches the cases 6, 11

• Coverage item 12: When it fails, any other of the previous coverage items fails too. So it doesn’t offer value in testing this side of the boundary

For the 11–16 partition lower boundary:

• Coverage item 9: When it fails, any other of the previous coverage items fails too. So it doesn’t offer value in testing this side of the boundary

• Coverage item 10: It’s the only one that catches the cases 19, 29

• Coverage item 11: It’s the only one that catches the cases 24, 28, 33

• Coverage item 12: It’s the only one that catches the case 36

Here are also some aggregated statistics:

• Each coverage item (9, 10, 11, 12) detects 55.5% of cases (20 out of 36)

• Each boundary value (10, 11) uniquely detects 13.9% of cases (5 out of 36)

• Each neighbour value (9, 12) uniquely detects 2.8% of cases (1 out of 36)

• 2-value BVA detects 94.4% of cases (34 out of 36)

• 3-value BVA with 3 coverage items detects 97.2% of cases (35 out of 36)

• 3-value BVA with 4 coverage items detects 100% of cases

3-value BVA: The Reality

According to the above results, we conclude that what seemed like a misconception is actually the correct approach to testing the boundaries of adjacent partitions.

The real misconception lies in the belief that 3-value BVA always results in 3 coverage items. If we limit ourselves to this approach (e.g., testing only 9, 10, 11 or 10, 11, 12), we risk reducing the bug detection rate by 2.8%.

Conclusion

To summarise, here’s a simple rule of thumb:

• 2-value BVA: Always results in 2 coverage items (the boundary values themselves).

• 3-value BVA:

  • For a single partition boundary (tested in isolation), results in 3 coverage items (the boundary value and its two neighbours).

  • For boundaries between two partitions, results in 4 coverage items (each partition’s boundary value and their respective neighbours).

Which to choose?

Which approach should you choose? The choice depends on the nature and criticality of your product. From my side, I hope this article has provided you with valuable insights to help you select the approach that best fits your needs and has given you a new perspective on the 3-value BVA test technique. :)