Types of Software Testing

Types of Software Testing | 4 Tests by Development Stage

There are several types of software testing depending on the development stage. Generally, following the development flow, first, unit testing is performed on a module or function basis. Next, integration testing is conducted to connect multiple modules and verify their operation. Finally, system testing (or comprehensive testing) is carried out for the final verification of the entire deliverable. After passing these tests, acceptance testing (or validation testing) is performed by personnel other than the developers or by users (clients or orderers) to determine whether the product should be accepted. The following sections introduce these four tests by development stage.

What is Unit Testing?

Unit testing is testing performed on a function or module basis. It verifies whether each function or module works correctly on its own according to the specifications and whether it can withstand abnormal inputs. Specifically, it checks whether the expected output is returned when an expected input is given to a function that processes sensor values. It also checks whether abnormal behavior occurs when values outside the expected range are entered. Furthermore, it verifies whether the output at boundary values (before and after thresholds, maximum, minimum, 0, etc.) is correct. Execution methods include static testing, which is performed without running the program, and dynamic testing, which is performed by actually running the program. The former checks for logical errors, while the latter checks for bugs that only occur during execution. Generally, both static and dynamic testing are performed not only in unit testing but also in the integration and system testing described below.

The Role of Integration Testing

Integration testing is a test to connect multiple modules that have completed unit testing and verify whether they “work correctly when combined.” In other words, it checks whether there are any inconsistencies in the interfaces between modules or if there are any errors in data transfer.

Key Points of System Testing (Comprehensive Testing)

System testing (or comprehensive testing) is conducted after the unit and integration testing described above. This test verifies whether the entire completed system meets the required specifications. It comprehensively checks not only functional aspects but also whether performance and security aspects meet the requirements. Generally, this is the final test performed by the system development side and serves as the final verification for the developers.

How to Proceed with Acceptance Testing (User Testing)

After the final verification by the development side through system testing, acceptance testing (or validation testing) is performed by the user side or personnel other than the developers to determine whether the product should be accepted. It verifies the operation in the actual usage environment or an equivalent environment to ensure that the quality, including performance and usability, is satisfactory to the user.

Software Testing Methods | Classification by Purpose and Method

Testing by Purpose, Such as Functional, Performance, and Security Testing

In the testing at each development stage described above, various tests are performed depending on the purpose. These include various tests such as functional testing (verifying operation according to specifications), performance testing (examining processing speed and load tolerance), security testing (detecting vulnerabilities and verifying safety). Which test is performed at which stage—unit, integration, or system—is determined on a case-by-case basis, but generally, it is necessary to conduct a wide range of these various tests, especially in system testing.

Difference Between White-Box Testing and Black-Box Testing

On the other hand, classifications of testing methods include white-box testing and black-box testing. The former is testing conducted with awareness of the source code and internal structure—in other words, testing to verify whether the software is built and operating as intended by the design. The latter, black-box testing, is testing that focuses only on functions and specifications as seen from the outside without being aware of the internal structure, and can be described as testing to verify whether the software functions correctly.

Importance of Regression Testing

Furthermore, regression testing (or regression testing) is also important in software development. This is a test to verify, when modifications or changes are made to software during the development process, “whether what was previously possible can still be done without change, and whether there are any changes in the behavior of the software in parts that have not been modified or changed” (conducted in unit, integration, and system testing). Therefore, it is ideal to perform it as much as possible with every modification or change, but because the frequency increases, it is important to enable it to be conducted in an effective manner that is worth the implementation cost.

• View PDF for details

Characteristics and Key Points of Embedded Software Testing

Importance of Testing in Embedded Software

The basic flow of testing in embedded software is the same, and all of the above points are performed in the same way. However, because hardware operates in conjunction with software in embedded systems, higher safety and reliability are often required than for PC software, and if a problem is found after the product is shipped, fixing it requires significant time and cost. Therefore, testing at each stage must be performed with particular care, and at the same time, testing from a different perspective than that of general-purpose software is required.

White-box testing and performance verification using actual devices

Because embedded systems operate through the coordination of software and hardware, another characteristic is that testing to confirm whether the two work together correctly is essential. In many cases, after implementing functions and performing unit testing (or simultaneously), the software is run on the actual target CPU (such as a microcontroller) to confirm operation on the actual device. Since operation on an actual microcontroller often differs from that on a PC, operation on the actual device is confirmed through white-box testing. After critical parts are implemented, it is recommended to perform performance testing at an early stage to confirm whether expected performance is achieved on the actual hardware.

Coordinated operation/communication verification of multiple CPUs and real-time capability

In addition, in complex systems, there is often not just one CPU but multiple CPUs, and many products operate as a single system through the coordination of each CPU. For example, a single automobile is equipped with dozens to around 100 CPUs that operate in coordination. Confirming whether they operate to satisfy the specifications of the entire product while communicating correctly (communication between CPUs) is also an important and difficult point in system testing for embedded devices. Furthermore, the requirement for real-time capability is a prominent characteristic of embedded software compared to general-purpose software. Testing to confirm this point (timing testing) can also be said to be particularly important in the development of embedded software.

Importance of early performance verification

In this way, testing of embedded software requires consideration of the hardware that works in coordination within the system, and confirming that they combine to meet the specifications. In addition to functional aspects, it is necessary to verify performance aspects such as timing verification, real-time capability (processing speed), and load tolerance. Since performance aspects are also related to hardware operation, once a problem occurs, it can lead to hardware redesign and potentially delay the development period. Furthermore, performance-related bugs often have low reproduction frequency or are difficult to reproduce. Therefore, if these occur in the late stages of development, the scope of impact of fixes and changes will be large. To avoid such situations, it is desirable to confirm at as early a stage of the development phase as possible that the software operates correctly and that its performance is achieved on the intended hardware.