- Describe the Test-Driven Development approach
- What is the "testing pyramid"?
- What is "unit testing"?
- Why are unit tests needed?
- What characteristics of a good unit test do you know?
- What unit test patterns exist?
- What is "integration testing"?
- How does integration testing differ from unit testing?
- What types of test objects exist?
- How does a stub differ from a mock?
- What are "fixtures"?
- What fixture annotations exist in JUnit 4 or 5?
- What is the
@Ignoreor@Disabledannotation used for in JUnit? - What frameworks for supporting automated acceptance testing do you know?
- What load testing tools do you know?
Test-Driven Development (TDD) is a development style in which the evolution of a system is driven by tests in short cycles:
- Write a single test.
- Write only the necessary amount of code to make the test pass.
- Refactor the code to make it "clean."
In programming languages such as Java, these cycles take no more than five minutes. In older languages with slow compilation and less support for automated refactoring, such a cycle takes longer — around 20 minutes.
The "testing pyramid" is a metaphor representing a pyramid made up of different levels of tests — unit, integration, and user-level (end-to-end) tests. At the base of the pyramid are unit tests, which should make up 70–80% of the total number of tests. Next come integration tests, accounting for 15–20%. At the top of the pyramid are user-level (e2e) tests, which should make up about 5%. This structure allows for the greatest benefit from test automation.
Unit/component testing is a process in programming that allows individual modules of a program's source code to be checked for correctness. The idea is to write tests for every non-trivial function or method. This makes it possible to quickly verify whether a recent code change has led to regression — that is, to the appearance of bugs in already tested parts of the program — and also makes it easier to detect and fix such bugs.
Unit tests serve several important purposes.
Less time spent on functional testing. Functional tests require a large amount of resources. Typically, you have to open the application and perform a series of actions to verify the expected behavior. Test engineers do not always know what those actions are and have to consult specialists in the area. The testing itself may take several seconds for minor changes or several minutes for larger ones. Finally, this process must be repeated for every change made to the system. Unit tests, on the other hand, take milliseconds, are executed with the simple press of a button, and do not necessarily require knowledge of the entire system. Whether the test passes depends on the test runner, not the user.
Protection from regression. Regression defects are introduced when changes are made to the application. Quite often, test engineers test not only the new feature but also previously existing features to verify that they still work as expected. With unit testing, you can re-run the entire test suite after every build or even after changing a single line of code. This gives you confidence that your new code has not broken existing functionality.
Executable documentation. It is not always obvious what a particular method does or how it behaves with certain inputs. You might ask yourself: how will the method behave if I pass it an empty string? Or a NULL value? If you have a set of unit tests with clear names, each test can clearly explain what the output will be for given inputs. Moreover, it can verify that this actually works.
Less coupled code. If code is tightly coupled, it is poorly suited for unit testing. Without creating unit tests for the code, this coupling may be less obvious. When you write tests for code, you naturally decouple it; otherwise, it would be harder to test.
A good unit test should be fast — in well-designed projects there can be thousands of unit tests, and they should execute very quickly, within milliseconds. It should be isolated — unit tests are self-contained, can run in isolation, and have no dependencies on external factors such as the file system or a database. It should be repeatable — running a unit test should produce consistent results, always returning the same result as long as no changes are made between runs. It should be self-checking — the test should automatically determine whether it passed or failed without user involvement. It should be timely — the time spent writing a unit test should not significantly exceed the time spent writing the code under test. If you feel that testing code takes too much time compared to writing it, consider a structure that is more suitable for testing.
AAA (Arrange, Act, Assert) is a good pattern for writing unit tests (input data, action, expected result). A single unit test should test one thing. Consequently, each test case should contain only one AAA set. A test case should not be too large (more than 10 lines of code) if it follows the AAA pattern.
BDD-style (Given, When, Then) uses three other keywords to describe each test case: Given, When, and Then. The "given-when-then" approach is almost identical to the "arrange-act-assert" approach. They both simply define a transition from one state to another in a Finite State Machine (FSM).
The differences between AAA and BDD-style are as follows. BDD-style looks at the module from the "outside," focusing on its external behavior. When using BDD, you must define a domain-specific language (DSL) when writing your test specifications. Because of this, it usually requires using a different framework.
Integration testing is testing that verifies the operability of two or more system modules working together — that is, several objects as a single unit. In interaction tests, a specific, defined object is tested along with how exactly it interacts with external dependencies.
From a technological standpoint, integration testing is a quantitative evolution of unit testing, since, like unit testing, it operates on the interfaces of modules and subsystems and requires the creation of a test environment, including stubs in place of missing modules. The main difference between unit and integration testing lies in the goals — that is, in the types of defects being detected, which in turn determine the strategy for selecting input data and analysis methods.
Consider the following example: suppose there is a class that, under certain conditions, interacts with a web service through a dependent object, and we need to verify that a certain method of the dependent object is actually called. If we pass a real class that works with the web service as the dependency, this constitutes integration testing. If we pass a stub, this constitutes state testing. If we pass a spy, and at the end of the test we verify that a certain method of the dependent object was indeed called, this constitutes an interaction test.
A dummy is an object that is typically passed to the class under test as a parameter but has no behavior — nothing happens to it and none of its methods are called. Examples of dummy objects include new object(), null, "Ignored String", and so on.
A fake object is used primarily to speed up resource-intensive tests and serves as a replacement for a heavyweight external dependent object with a lightweight implementation. The main examples are a database emulator (fake database) or a fake web service.
A test stub is used to obtain data from an external dependency by substituting it. The stub ignores all data coming from the object under test and returns a predetermined result. For instance, if the object under test reads from a configuration file, you pass it a ConfigFileStub that returns test configuration strings without accessing the file system.
A test spy is a variant of a stub that can record calls made to it by the system under test in order to verify their correctness at the end of the test. It captures the number, composition, and content of call parameters. If there is a need to verify that a particular method of the class under test was called exactly once, a spy is exactly what is needed.
A mock object is similar to a spy but has extended functionality, with pre-defined behavior and reactions to calls.
A stub is used as a stand-in for services, methods, classes, etc. with pre-programmed responses to calls.
A mock uses substitution of call results, verifies the fact of interaction itself, and records and controls it.
Fixtures represent the state of the testing environment that is required for successful test execution. The main purpose of fixtures is to prepare the test environment with a pre-fixed/known state in order to guarantee the repeatability of the testing process.
@BeforeClassin JUnit 4 /@BeforeAllin JUnit 5 — defines code that must be executed once before the set of test methods is run.@AfterClassin JUnit 4 /@AfterAllin JUnit 5 — code executed once after the set of test methods has been run.@Beforein JUnit 4 /@BeforeEachin JUnit 5 — defines code that must be executed each time before any test method is run.@Afterin JUnit 4 /@AfterEachin JUnit 5 — code executed each time after any test method has been run.
@Ignore in JUnit 4 or @Disabled in JUnit 5 indicates that the given test method should be skipped.
Cucumber, JBehave, Spock.
Apache JMeter, The Grinder, Gatling, HP Performance Tester (LoadRunner).