Unit vs Integration Testing
Note: This post was originally posted on my other blog back in January, and has popped back into my technical subject “radar” so I am re-publishing it here.
I have danced around with the idea of honestly implementing unit tests into my development toolkit and processes, but have struggled with getting my head around the concepts and their real-world application to my development tasks. I understand the reasons and benefits of unit testing, having read countless tutorials on the subject, from getting started with unit tests to test-driven development.
In my most recent attempt, I am revisiting the subject but this time asking more direct questions about what to test and even more importantly how to test my code. Additionally, putting on my pragmatist hat, I ask the question “what kinds of tests will provide the most benefit in my particular circumstances”?”
What are my particular circumstances? Well, I am a 1-man development team supporting software that primarily performs validation and transformation tasks on data extracted from various file formats, loads the data into a database and in some circumstances, the data is parsed. Much of the data is received in fixed-width, flat file formats, and many of those particular files originated on legacy mainframe systems. So, the primary tasks involve properly reading the data from, and writing the data to files while performing transformation (including cleanup of low values, etc…) activities on this data.
Having said that, the targets of properly created unit tests, by definition, in my case are not what I would consider to be, in and of themselves, valuable in terms of the feedback a unit test against them would provide, again, assuming the unit test fits the classic definition. This has been the clear and indisputable reason behind my adoption of unit tests per se. However, as I started digging deeper, asking more specific questions pertaining to the specifics of my scenario, I uncovered what is, in my experience, a neglected sibling to unit tests, and that is the integration test.
The light bulbs came on, one by one, and while I was drilling down and through Stack Overflow I came across this answer to a question that has striking similarities to those that I have been asking. Here are the bulleted definitions:
ddaa wrote:
Unit test: Specify and test one point of the contract of single method of a class. This should have a very narrow and well defined scope. Complex dependencies and interactions to the outside world are stubbed or mocked.
Integration test: Test the correct inter-operation of multiple subsystems. There is whole spectrum there, from testing integration between two classes, to testing integration with the production environment.
Smoke test: A simple integration test where we just check that when the system under test is invoked it returns normally and does not blow up. It is an analogy with electronics, where the first test occurs when powering up a circuit: if it smokes, it’s bad.
Regression test: A test that was written when a bug was fixed. It ensure that this specific bug will not occur again. The full name is “non-regression test”.
To this, I will add:
- Acceptance test: Test that a feature or use case is correctly implemented. It is similar to an integration test, but with a focus on the use case to provide rather than on the components involved.
It has been integration testing that I have been looking for. Sure, unit tests have a place in my tool box, and will be used to be sure, but that tool alone does not satisfy my need in a test, nor does it provide but maybe 10% of the total value I am looking for in a testing process.
As for the tools, I am starting out (at least) with the testing tools that come with Visual Studio 2010 Pro. I know there are tons of tools out there and that I can’t throw a rock, blind-folded, and not hit one of them, but I will start with what I have. BUT, a question I have (and will save for another post) is can unit testing tools be used to create integration tests? I came across a post today, which I can’t remember where to find it for the life of me, that said that yes, you can.
UPDATE: Here is the response in Stack Overflow question that alludes to using unit testing tools to automate integration tests:
Kent Boogaart wrote:
I am reticent to pollute my code with types and concepts that exist only to facilitate unit testing. Sure, if it makes the design cleaner and better then great, but I think that is often not the case.
My take on this is that your unit tests would do as much as they can which may not be 100% coverage. In fact, it may only be 10%. The point is, your unit tests should be fast and have no external dependencies. They might test cases like “this method throws an ArgumentNullException when you pass in null for this parameter”.
I would then add integration tests (also automated and probably using the same unit testing framework) that can have external dependencies and test end-to-end scenarios such as these.
When measuring code coverage, I measure both unit and integration tests.
Either way, at the end of the day, I still find it curious as to why there is SO much emphasis put on unit testing, tools, etc…and relatively nothing covering integration testing tools and techniques. If I am missing something here, somebody—anybody—please fill this in for me :-) In the meantime, I will continue building upon what I have unearthed today to add to my toolbox and continually improve my development processes.
