Testing Techniques

Black box -Test Techniques

Equivalence Partitioning:

In this method the input domain data is divided into different equivalence data classes. This method is typically used to reduce the total number of test cases to a finite set of testable test cases, still covering maximum requirements.

 

In short it is the process of taking all possible test cases and placing them into classes. One test value is picked from each class while testing.

 

E.g.: If you are testing for an input box accepting numbers from 1 to 1000 then there is no use in writing thousand test cases for all 1000 valid input numbers plus other test cases for invalid data.

 

Using equivalence partitioning method above test cases can be divided into three sets of input data called as classes. Each test case is a representative of respective class.

So in above example we can divide our test cases into three equivalence classes of some valid and invalid inputs.

 

Test cases for input box accepting numbers between 1 and 1000 using Equivalence Partitioning:
1) One input data class with all valid inputs. Pick a single value from range 1 to 1000 as a valid test case. If you select other values between 1 and 1000 then result is going to be same. So one test case for valid input data should be sufficient.

2) Input data class with all values below lower limit. I.e. any value below 1, as a invalid input data test case.

3) Input data with any value greater than 1000 to represent third invalid input class.

So using equivalence partitioning you have categorized all possible test cases into three classes. Test cases with other values from any class should give you the same result.

 

We have selected one representative from every input class to design our test cases. Test case values are selected in such a way that largest number of attributes of equivalence class can be exercised.

 

Equivalence partitioning uses fewest test cases to cover maximum requirements.

Boundary value analysis:

It’s widely recognized that input values at the extreme ends of input domain cause more errors in system. More application errors occur at the boundaries of input domain. ‘Boundary value analysis’ testing technique is used to identify errors at boundaries rather than finding those exist in center of input domain.

 

Boundary value analysis is a next part of Equivalence partitioning for designing test cases where test cases are selected at the edges of the equivalence classes.

 

Test cases for input box accepting numbers between 1 and 1000 using Boundary value analysis:
1) Test cases with test data exactly as the input boundaries of input domain i.e. values 1 and 1000 in our case.

2) Test data with values just below the extreme edges of input domains i.e. values 0 and 999.

3) Test data with values just above the extreme edges of input domain i.e. values 2 and 1001.

 

Boundary value analysis is often called as a part of stress and negative testing.

 

Note: There is no hard-and-fast rule to test only one value from each equivalence class you created for input domains. You can select multiple valid and invalid values from each equivalence class according to your needs and previous judgments.

 

E.g. if you divided 1 to 1000 input values in valid data equivalence class, then you can select test case values like: 1, 11, 100, 950 etc. Same case for other test cases having invalid data classes.

 

This should be a very basic and simple example to understand the Boundary value analysis and Equivalence partitioning concept.

Decision Tables:

If you’re not familiar with decision tables, let’s go through a simple example. Assume your local baseball squadron offers free tickets to kids and discounted tickets to senior citizens. One game a year, free hats are given to all fans.

 

To represent this logic in a decision table, create a spreadsheet and list all inputs and expected results down the left side. The inputs in this case are the fan’s age and sex. The expected results are ticket price and hat color.

 

 

Each row of a decision table should contain the different possible values for a single variable. Each column, then, represents a different combination of input values along with their expected results. In this example, the first column represents the expected results for boys under 5 — “Free Admission” and “Blue Hat”. The last column shows that female senior citizens get $10 tickets and a pink hat.

 

  Rule 1 Rule 2 Rule 3 Rule 4 Rule 5 Rule 6
Inputs            
Age < 5 Y Y        
5 =< Age < 65     Y Y    
Age >= 65         Y Y
Sex M F M F M F
Results            
Free Admission Y Y        
$10 Admission         Y Y
$20 Admission     Y Y    
Blue Hat Giveaway Y   Y   Y  
Pink Hat Giveaway   Y   Y   Y

 

There are three major advantages to decision tables.

  1. Decision tables define expected results for all input combinations in an easy-to-read format. When included in your functional spec, decision tables help developers keep bugs out of the product from the beginning.
  2. Decision tables help us design our test cases. Every column in a decision table should be converted into at least one test case. The first column in this table defines a test for boys under 5. If an input can be a range of values, however, such as “5 =< Age < 65″, then we should create tests at the high and low ends of the range to validate our boundary conditions.
  3. Decision tables are effective for reporting test results. They can be used to clearly show management exactly what scenarios are working and not working so informed decisions can be made.

State transition testing:

  • State transition testing is used where some aspect of the system can be described in what is called a ‘finite state machine’. This simply means that the system can be in a (finite) number of different states, and the transitions from one state to another are determined by the rules of the ‘machine’. This is the model on which the system and the tests are based.
  • Any system where you get a different output for the same input, depending on what has happened before, is a finite state system.
  • A finite state system is often shown as a state diagram (see Figure 4.2).
  • One of the advantages of the state transition technique is that the model can be as detailed or as abstract as you need it to be. Where a part of the system is more important (that is, requires more testing) a greater depth of detail can be modeled. Where the system is less important (requires less testing), the model can use a single state to signify what would otherwise be a series of different states.

 

State transition model has four basic parts:

  • The states that the software may occupy (open/closed or funded/insufficient     funds);
  • The transitions from one state to another (not all transitions are allowed);
  • The events that cause a transition (closing a file or withdrawing money);
  • The actions that result from a transition (an error message or being given your cash).

Hence we can see that in any given state, one event can cause only one action, but that the same event – from a different state – may cause a different action and a different end state.

 

For example, if you request to withdraw $100 from a bank ATM, you may be given cash. Later you may make exactly the same request but it may refuse to give you the money because of your insufficient balance. This later refusal is because the state of your bank account has changed from having sufficient funds to cover the withdrawal to having insufficient funds. The transaction that caused your account to change its state was probably the earlier withdrawal. A state diagram can represent a model from the point of view of the system, the account or the customer.

 

Figure below, shows an example of entering a Personal Identity Number (PIN) to a bank account. The states are shown as circles, the transitions as lines with arrows and the events as the text near the transitions. (We have not shown the actions explicitly on this diagram, but they would be a message to the customer saying things such as ‘Please enter your PIN’.)

 

The state diagram shows seven states but only four possible events (Card inserted, Enter PIN, PIN OK and PIN not OK). We have not specified all of the possible transitions here – there would also be a time-out from ‘wait for PIN’ and from the three tries which would go back to the start state after the time had elapsed and would probably eject the card. There would also be a transition from the ‘eat card’ state back to the start state. We have not specified all the possible events either – there would be a ‘cancel’ option from ‘wait for PIN’ and from the three tries, which would also go back to the start state and eject the card.

State transition example

In deriving test cases, we may start with a typical scenario.

  • First test case here would be the normal situation, where the correct PIN is entered the first time.
  • A second test (to visit every state) would be to enter an incorrect PIN each time, so that the system eats the card.
  • A third test we can do where the PIN was incorrect the first time but OK the second time, and another test where the PIN was correct on the third try. These tests are probably less important than the first two.
  • Note that a transition does not need to change to a different state (although all of the transitions shown above do go to a different state). So there could be a transition from ‘access account’ which just goes back to ‘access account’ for an action such as ‘request balance’.

White box -Test Techniques

What is Statement coverage?

  • The statement coverage is also known as line coverage or segment coverage.
  • The statement coverage covers only the true conditions.
  • Through statement coverage we can identify the statements executed and where the code is not executed because of blockage.
  • In this process each and every line of code needs to be checked and executed

Advantage of statement coverage:

  • It verifies what the written code is expected to do and not to do
  • It measures the quality of code written
  • It checks the flow of different paths in the program and it also ensure that whether those path are tested or not.

Disadvantage of statement coverage:

  • It cannot test the false conditions.
  • It does not report that whether the loop reaches its termination condition.
  • It does not understand the logical operators.

Example:

 

To understand the statement coverage in a better way let us take an example which is basically a pseudo-code. It is not any specific programming language, but should be readable and understandable to you, even if you have not done any programming yourself.

Consider code sample :


READ X
READ Y
I F X>Y THEN Z = 0
ENDIF


Code sample

 

 

To achieve 100% statement coverage of this code segment just one test case is required, one which ensures that variable A contains a value that is greater than the value of variable Y, for example, X = 12 and Y = 10. Note that here we are doing structural test design first, since we are choosing our input values in order ensure statement coverage.

 

Now, let’s take another example where we will measure the coverage first. In order to simplify the example, we will regard each line as a statement. A statement may be on a single line, or it may be spread over several lines. One line may contain more than one statement, just one statement, or only part of a statement. Some statements can contain other statements inside them. In code sample , we have two read statements, one assignment statement, and then one IF statement on three lines, but the IF statement contains another statement (print) as part of it.

 

1 READ X
2 READ Y
3 Z =X + 2*Y
4 IF Z> 50 THEN
5 PRINT large Z
6 ENDIF
Code sample

 

Although it isn’t completely correct, we have numbered each line and will regard each line as a statement. Let’s analyze the coverage of a set of tests on our six-statement program:

 

TEST SET 1
Test 1_1: X= 2, Y = 3
Test 1_2: X =0, Y = 25
Test 1_3: X =47, Y = 1

 

Which statements have we covered?

  • In Test 1_1, the value of Z will be 8, so we will cover the statements on lines 1 to 4 and   line 6.
  • In Test 1_2, the value of Z will be 50, so we will cover exactly the same statements as Test 1_1.
  • In Test 1_3, the value of Z will be 49, so again we will cover the same statements.

 

Since we have covered five out of six statements, we have 83% statement coverage (with three tests). What test would we need in order to cover statement 5, the one statement that we haven’t exercised yet? How about this one:

Test 1_4: X = 20, Y = 25

 

This time the value of Z is 70, so we will print ‘Large Z’ and we will have exercised all six of the statements, so now statement coverage = 100%. Notice that we measured coverage first, and then designed a test to cover the statement that we had not yet covered.

 

Note that Test 1_4 on its own is more effective which helps in achieving 100% statement coverage, than the first three tests together. Just taking Test 1_4 on its own is also more efficient than the set of four tests, since it has used only one test instead of four. Being more effective and more efficient is the mark of a good test technique.

 

What is Decision Coverage/Branch Coverage?

Decision Coverage is also known as Branch Coverage.

 

Whenever there are two or more possible exits from the statement like an IF statement, a DO-WHILE or a CASE statement it is known as decision because in all these statements there are two outcomes, either TRUE or FALSE.

 

With the loop control statement like DO-WHILE or IF statement the outcome is either TRUE or FALSE and decision coverage ensures that each outcome(i.e TRUE and FALSE) of control statement has been executed at least once.

 

Alternatively you can say that control statement IF has been evaluated both to TRUE and FALSE.

 

The formula to calculate decision coverage is:

Decision Coverage=(Number of decision outcomes executed/Total number of decision outcomes)*100%

 

Research in the industries have shown that even if through functional testing has been done it only achieves 40% to 60% decision coverage.

 

Decision coverage is stronger that statement coverage and it requires more test cases to achieve 100% decision coverage.

Let us take one example to explain decision coverage:

READ X
READ Y
IF “X > Y”
PRINT X is greater that Y
ENDIF

 

To get 100% statement coverage only one test case is sufficient for this pseudo-code.

TEST CASE 1: X=10 Y=5

 

However this test case won’t give you 100% decision coverage as the FALSE condition of the IF statement is not exercised.

 

In order to achieve 100% decision coverage we need to exercise the FALSE condition of the IF statement which will be covered when X is less than Y.

So the final TEST SET for 100% decision coverage will be:

TEST CASE 1: X=10, Y=5
TEST CASE 2: X=2, Y=10

 

Note: 100% decision coverage guarantees 100% statement coverage but 100% statement coverage does not guarantee 100% decision coverage.

Condition Coverage or Predicate Coverage

Condition coverage is also known as Predicate Coverage

Condition coverage is seen for Boolean expression, condition coverage ensures whether all the Boolean expressions have been evaluated to both TRUE and FALSE.

Let us take an example to explain Condition Coverage

IF (“X && Y”)

In order to suffice valid condition coverage for this pseudo-code following tests will be sufficient.

TEST 1: X=TRUE, Y=FALSE
TEST 2: X=FALSE, Y=TRUE

Note: 100% condition coverage does not guarantee 100% decision coverage.

Multiple Condition Coverage

Multiple Condition Coverage is also known as Condition Combination Coverage.

In Multiple Condition Coverage for each decision all the combinations of conditions should be evaluated.

 

Lets take an example:

if (A||B)
then
print C

 

Here we have 2 Boolean expressions A and B, so the test set for Multiple Condition Coverage will be:

TEST CASE1: A=TRUE, B=TRUE
TEST CASE2: A=TRUE, B=FALSE
TEST CASE3: A=FALSE, B=TRUE
TEST CASE4: A=FALSE, B=FALSE

Code Coverage

In computer science, code coverage is a measure used to describe the degree to which the source code of a program is tested by a particular test suite. A program with high code coverage has been more thoroughly tested and has a lower chance of containing software bugs than a program with low code coverage. Many different metrics can be used to calculate code coverage; some of the most basic are the percent of program subroutines and the percent of program statements called during execution of the test suite.

 

Code Coverage is a measurement of how many lines/blocks/arcs of your code are executed while the automated tests are running.

 

CC is collected by using a specialized tool to instrument the binaries to add tracing calls and run a full set of automated tests against the instrumented product. A good CC tools will give you not only the percentage of the code that is executed, but also will allow you to drill into the data and see exactly which lines of code were executed during particular test.

 

Print Print | Sitemap
© 2016 Automation Learn. All rights reserved.