The disagreement over whether to write a unit test or an end-to-end evaluation for an element of a software system is something I have encountered a number of times. It mostly appears as a philosophical conversation along the lines when we can only write one test for this feature, should we write a unit test or an end-to-end test? Basically, time and resources are limited, so what type of test would be most effective?
In this article, I'll provide my view on this question. I must be aware that my experience has been in building software infrastructure for industrial applications -- streaming data system for near-real-time data. For someone who has worked in another domain, where calculating and analysing the whole software process is simpler, or at which the functional environment is more forgiving of mistake, I could understand the way their experience might be different. I've worked on hosted solutions in addition to infrastructure that's installed on-premises and operated by the client.
These programs are composed of a number of unique components, must perform consistently and reliably, and fulfil crosscutting requirements concerning security, scalability, and functionality. These programs will need to evolve to add new functionality or bug fixes, without introducing regressions in present functionality or behaviour. Testing these systems end-to-end is obviously a challenge since there are a number of dependencies that have to be in place so as to test even a small part of the general system. Reproducing the diversity of problems encountered in functional settings can also be hard.
End-to-End Tests
An end-to-end test is generally either an operational test that confirms that a particular facet of the system behaves as expected, or an approval test which not only verifies the aspect functions correctly, but also validates the wider system continues to meet prerequisites concerning functionality, scalability, security, maintainability, and so on. End-to-end tests typically require deploying the machine and its dependencies, which may take a substantial amount of resources and time. The evaluations can take a very long time to conduct and are subject to variability, given the amount of moving components.
As an engineer, a product manager, a development lead, or even an engineering manager, I'd always need every critical part of the system to be tested as part of an end-to-end test. I just wouldn't live with this. This evaluation not only verifies the feature works but in addition, it confirms that it works under normal operational problems. The concluding test is indispensable. I would like to know that the system I am providing matches the prerequisites and will continue to satisfy the requirements because the system evolves in the future.
One other important consideration concerning end-to-end tests is that the most complicated and subtle bugs in application systems can't be seen in isolation and therefore are just struck when exercised as part of an integrated platform. I have written previously about some hard bugs I encountered that couldn't be detected with unit tests, or even a functional evaluation exercising aspect in isolation. End-to-end tests allow you the opportunity to study the application under conditions where these bugs arise and are invaluable for building robust and reliable software systems.
Unit Tests
A unit test is a test that independently exercises a small component of source code, like a method or a course. A unit test typically has no dependencies, can be implemented in milliseconds, and so is absolutely trustworthy.
End-to-end tests serve a vital function, but when a system is analysed mostly or exclusively with finishing evaluations, difficulties arise. The Google Testing Blog post Only Say No to More End-to-End Tests does a fantastic job of characterising the problems of relying on too many finishing tests. Considering that the inevitable complexity of end-to-end tests, it can take days or weeks to get opinions. When an end-to-end test fails, it is frequently rather tricky to identify the component that resulted in the failure. Even then, the failure can frequently be a result of the variability introduced by the test infrastructure itself. The Google post suggests that evaluations are most efficiently arranged as a member of a pyramid where the biggest number of tests are unit tests, followed by a moderate variety of integration tests, then a lesser number of end-to-end tests.
I worked for quite a few years on a software program that was tested almost exclusively with finishing tests. An inverted pyramid at best, but probably closer to just the tip of the pyramid. It would require a day or more to conduct these tests. There were regular test failures as a consequence of the test infrastructure. When a test failed, it was often very hard to ascertain why. The group spent a great deal of time characterising test failures, an expensive activity that became accepted as part of the daily routine. Rinse and repeat.
I think unit tests serve another purpose and are complimentary to complete tests. Unit tests are more about programmer productivity and imagination, rather than verifying that the machine is functioning properly. For a programmer, the opinions cycle derived from finishing evaluations is too long. If the activation energy required for experimentation and opinions becomes too large, it becomes a barrier to mining, learning, and making progress. Unit tests provide nearly instant feedback. They're supportive of experimentation, as there isn't any need to deploy the machine to conduct the tests. Unit tests can readily be executed on the developer's workstation and integrate seamlessly with most IDEs.
Unit testing, unlike finishing evaluations, can readily be implemented when code is committed to source control. In case a unit test fails, then the commit could be rejected. This usually means that the code under source control can be maintained in a country where it's always functional. This has great advantages, especially when dependencies are shared across multiple projects or teams.
Another free aspect of unit tests is that they support considering how the code has been factored. This is not something end-to-end tests generally promote. Test-Driven Development (TDD) has been a popular practice for helping developers write better code. While TDD doesn't necessarily prescribe unit evaluations, they are generally essential to it. I am not a purist in terms of TDD. I seldom write formal device tests upfront. As I'm working, I generally write evaluations to support experimentation, preferring to formalise my evaluations once I've settled on a course forward. After I began unit testing, however, it certainly helped me develop better interfaces and simpler implementations than I would have otherwise.
In the introduction, I said two particular concerns regarding writing unit tests in addition to end-to-end tests. The first was that the overlap between unit tests and finishing tests. There'll not necessarily be a one-to-one relationship between a unit evaluation and an end-to-end test, but there will continually be duplication. I believe it's effective to test the exact same functionality under different states. I actually embrace this particular reproduction. I'd rather something be tested twice, instead of not at all, and that I believe considering the same evaluation from different perspectives, at different times, by different people, ends up enhancing the overall system. Frequently it is not the test artefacts themselves but the action of testing that ends up enhancing the total system and the skills of the folks who work on it.
The second issue was that unit tests make the system difficult to evolve. I find easy unit tests that concentrate on testing one thing per evaluation method are rarely hard to refactor if refactoring is even necessary. I have never discovered unit tests for a burden concerning evolving a system. In fact, I feel a system analysed largely with end-to-end tests finally becomes more difficult to evolve, as individuals become afraid of making adjustments when they can't easily characterise the consequences. I am able to understand when unit tests involve a lot of mocking that one feels it's near impossible to evolve the system. However, by using a great deal of mocking, one inevitably ends up tying the evaluations straight to the implementation. I rarely, if ever, use mocks. When a test requires mocking, I normally reconsider the plan and alter my approach so that mocking becomes unnecessary, or I reevaluate whether the attribute would be effectively tested with just an end-to-end evaluation.
My Answer?
Whenever possible, I compose both a unit evaluation and an end-to-end evaluation. I see unit tests as complimentary to end-to-end tests. End-to-end tests verify the behaviour of the system as a whole, while unit tests encourage programmer productivity and creativity. I embrace the diversity of analysing the identical aspect from several perspectives. I enjoy how unit evaluations inform software design and business and keep the code base healthy when they need to pass in order to commit code.
Comments
Post a Comment