Exhausted with Bugs? 8 Test Design Techniques You Need to Know
What Is a Test Design Technique, Its Types and How to Put It to Practice with Real Examples from an Existing Project
Here’s the idea — I will lead you through theory to practice while explaining test design techniques (TDT) that should be used when testing applications. You’ll see real examples from an existing project of our IT outsourcing company.
First of all…
What is testing?
Testing is the process of collecting and providing information about how the software works, how it meets the needs of users. It allows you to make decisions whether the application is ready for use or not.
Often this information is of a negative nature, because in the process of studying the product, defects are detected. At the same time, the information obtained helps prevent future defects.
Testing stages
The testing process can be divided into the following stages:
- Planning and control
- Test analysis and design
- Implementation and execution of tests
- Evaluation of exit criteria and reporting
- Test closure activities
Any project is always limited in time for its implementation, budgetary funds, and scope.
Under these restrictions (and common sense) it is impossible to test all possible conditions and combinations. Not having a specific plan, you can leave important sections of the program untested. Therefore, before the tests are performed, test cases are drawn up at the stage of test analysis and design — that is, “step-by-step instructions for inspections”: the sets of input data, preconditions for execution are selected, and the expected results are formulated.
It is important to choose such test cases that will have the greatest probability of finding defects. For this purpose, you had better use TDT — procedures, which allow meaningful selection of test cases.
The goal
Our goal is the minimum of tests and the maximum of checks and defects detected.
Test design helps to decide:
- what to test and what not to test
- what data to use to test the operation of the application and
- how the system should react to the input data.
3 Categories of Test Design Techniques
Test design techniques can be defined as high level verification steps that are created to design a product or software that is free from all kinds of defects. Test design techniques can be derived from business scenarios and are categorized based on the type of testing.
There are 3 categories of TDT:
1. Experience-based: technique based on experience.
TDT#1. Error guessing — depends on knowledge, skills and experience. Often in the same programs, errors of the same type occur. There is no special approach — the tester foresees what the system might not handle (division by 0, entering letters into numeric fields).
TDT#2. Exploratory testing — means minimum planning time, maximum test executions. At the same time, the program is being studied, tests are designed and executed.
These methods can be used when there are no specifications, or with an absolute lack of time. They can identify the most critical defects.
If you use only this technique, the test coverage will not be sufficient, and it will be difficult to provide reliable information about how the program performs the necessary tasks. Techniques based on experience should be used only as an addition to the techniques of the “black” and “white” boxes.
2. Structure-based: based on the structure (white box methods). They use the internal structure of the program to determine test cases.
The tester needs to know the internal structure of the software, because these methods are directed directly at code and flow.
Therefore, the source code must be available, and in certain cases, it should be possible to add code.
3. Specification-based: based on specifications, analysis of documentation (black box technique).
Consider the program as a “black box” with inputs and outputs.
The tester does not take into account how the system or a component works inside this “box”, and concentrates on what the program does, and not how it does it.
To describe the tasks that need to be addressed, software products or their components, models are used — formal or informal. From these models, test scenarios are systematically derived.
TDT#3. Equivalence class partitioning is a technique in which test cases are chosen to verify data that represents different equivalent classes.
An equivalent class is a set of input (or output) values for which the behavior of a component or system must be the same.
Each class must be tested at least once.
Testing on one condition in each class, we assume that the system will treat all the conditions equally in this equivalence class. If one condition works, then we assume that others will work (and vice versa respectively). It is important to check not only the specification (valid classes), but also outside (invalid classes).
For example, there is a module for hiring. Based on age, there are the following conditions for admission of a candidate:
Data within each age range (and beyond their boundaries) are equivalence classes.
One test value is selected to exercise the data within each equivalence partition.
TDT#4. Boundary value analysis is a technique in which boundaries between classes are tested. A boundary value is a value that is at the smallest incremental “distance” on each side of the boundary, which means it’s next to the minimum or maximum value of the range.
When developing a large number of problems occurs precisely at the boundaries of input variables — at the boundaries of equivalence classes. This is because borders are often not clearly defined or misunderstood. Often in the code, the matter is in comparison operations. For example, a value of > 100 should be accepted, and ≥ 100 is written in the code, and a value of 100 becomes erroneously valid.
Therefore, it is necessary to test the boundaries themselves and the nearest adjacent values (inside and outside the equivalent class). It is important for the tester to have information about which values are acceptable, what “step” between values, in order to correctly determine the minimum “increment” or “decrease”.
Technically, since each boundary belongs to a certain class, we can say that by testing only the boundary values, we test each equivalence class. But this approach can lead to a problem: if this value leads to an error, we can not say whether only the boundary value was error or the entire equivalence class.
TDT#5. State transition testing is a technique that is used when not only the current input, but also the history of previous actions or inputs affect the result the system displays and the behavior of the system. That is, with the same actions user can have different results.
For example, you need to withdraw 100 UAH from an ATM. After certain entries, the customer of the bank receives cash. Repeating the same sequence of inputs a second time — he may not receive money due to the fact that the balance will no longer be the amount requested. Refusal to issue money is due to the fact that the state of the bank account has changed from sufficient to fulfill the request to insufficient, due to the previous transaction (event).
To illustrate the dependence on the “history”, state diagrams are used. The system is analyzed in terms of:
- states — conditions in which the system expects events. The states “remember” the inputs received by the system in the past and determine how the system should react to subsequent events.
- transitions — changes in states caused by a certain event.
- events (inputs) — what causes the system to change the state.
- actions — what happens as a result of a change in state, they cause output.
Test cases should be chosen so that all transitions have been checked at least once.
For example, The JobsToday project is a platform for recruitment by employers and job search by candidates across Europe.
Testing the appearance of a modal window with the options “Use Payout / Pay for a Retail Job” — it should be displayed only if the user has a package of jobs with an unexpired term of use and a sufficient number of available jobs.
1. Precondition: The user has an active Job Package, in which there are 2 available jobs.
Click the “Post Job” button.
Expected Result: There should be a pop-up with options: “Use Package / Pay per Job”.
Choose “Use Package” — the job was successfully published (used the penultimate work from the package).
2. Precondition: The user has an active Job Package, in which 1 job is available.
Click “Post another Job”.
Expected Result: There should be a pop-up with options: “Use Package / Pay per Job”.
Choose “Use Package” — the work was successfully published (used the latest work from the package).
3. Precondition: The user does not have active Job Packages left.
Click “Post another Job”.
Expected result: a pop with options: “Use Package / Pay per Job” does not appear. The user has been redirected to the Payment of Retail Jobs page.
The user’s actions are the same as in the 2nd test, but the 2nd event caused the system to change the state-available jobs were available, but after the 2nd test-transition-there were no jobs left, so the popup no longer appeared.
4. Precondition: The user does not have active Job Packages left.
Click the “Post Job” button.
Expected result: a pop up with options: “Use Package / Pay per Job” does not appear. The user is redirected to the Payment of Retail Jobs page.
The user’s actions are the same as in the 1st test, but the 2nd event caused the system to change the state-available works were available, but after the 2nd test-transition-there were no jobs left, so the popup no longer appeared.
TDT#6. Decision table testing is a technique that allows you to test combinations of inputs and output conditions.
Focuses more on business logic and rules. It is convenient to use when these rules are represented as a set of simultaneously executing conditions (united by AND, OR, NOT), and these conditions have results that are associated with them.
The idea of the method is in the construction of cause-effect relationships, the selection of such combinations of inputs, which will sufficiently allow to check the correctness of the system operation.
Here’s what we’ll see on the screen in each case:
At the top of the table, the causes (parameters of the input data) are entered, in the bottom part — the consequences (the expected result). Each column can be considered as a ready test case.
Often, decision tables contain many different combinations of conditions, and there is not enough time, and it does not always make sense to test all possible combinations.
TDT#7. Pairwise testing is convenient to use in such cases. With this technique, each tested value of each of the tested parameters is combined with each of the tested values of all other parameters at least once.
According to IBM research, the majority of defects (97%) occur when there are no more than two factors interacting. And the probability of appearance of a defect, depending on the combination of 3 and more specific parameters, is very small. This type of testing is convenient to carry out with the help of special programs — PICT from Microsoft, Combinatorial Test Services — IBM, and others.
For example, we need to check the correctness of the search filter system. To sort out all possible combinations of conditions is inexpedient.
TDT#8. Use case testing — test cases are compiled on the basis of scenarios for which the user interacts with the application. This method is convenient for use in acceptance testing. Describes the main flow by which the user will go and, accordingly, allows to find defects that can be seen by users.
Use case is composed from the perspective of the user, describes what he does and sees. The input data that should be used to obtain a certain result is not described.
A set of such scenarios can serve as a basis for the functional requirements of the system.
Scenario: U — User; S — System; SA — Super Admin
Steps:
1. U: Orders a Package from “Order Packages” page of the Company Admin Panel
2. S: Validates the payment and displays the Confirmation message
3. S: Displays the User’s order in the “Online orders” table of the Super Admin Panel
4. SA: Assigns the ordered Package to the User
5. S: Displays the assigned Package in the “Individual package” table of Super Admin Panel with Status=Online
6. S: Displays the assigned Package on “My Packages” page of the Company Admin Panel
Conclusion
Each technique focuses on certain characteristics, each has its own advantages and disadvantages. For example, a technique that examines the upper and lower input boundaries will find boundary value defects, but may not take into account the combination of inputs into different fields.
Or techniques based on the structure can find malicious code, but will not find the parts that are missing in the specification. And if there are inaccuracies in both the code and the specification, then the use of techniques based on experience will bring more results.
When choosing a test design technique, it is recommended to consider factors such as application type, standards, customer requirements, level and type of risk, availability of documentation, knowledge, and experience of the tester.
If you liked this, show your support by clapping us to share with other people on Medium.
Follow us on Facebook, Instagram, LinkedIn, Behance, Medium and visit our corporate blog for more news and articles on smart solutions.
Any questions? Feel free to contact us!
