Someone recently asked me about mocks. As I hadn’t yet put my thoughts in writing, I do so here. Hopefully there’s some helpfulness scattered here somewhere…
“I guess we can only integration test this code – it pulls in the world”
As you start to write unit tests for your code, you quickly find that while there are a few methods that are easy to test, you quickly get bogged down with collaborators. “Oh, I guess I have to set up a real database server with test data out there somewhere or this method can’t be tested…” or “Rats, I don’t want to have my test run the part that initiates a transaction with this remote internet service, because it will result in charges to the company”, or “These blocks of code only run if the collaborator throws some obscure error, and I don’t have any way of reliably creating ”
More generally, here are some of the challenges to dealing only with integration tests*:
- Setting up the infrastructure for the test takes time and effort – sometimes involving configuring and maintaining database servers or setting up accounts with third parties
- Tests tend to run slowly because they must talk to the database, place a phone call, really perform the complex transaction processing, etc.
- When a test breaks, additional time and effort is needed to isolate the cause, as there are many moving parts
- Due to the infrastructural setup required, the test itself is more complex and prone to bugs. Time must be spent diagnosing in which class the problem lies (potentially through many layers of the system, sometimes spanning servers, DLLs, and codebases)
Dealing gracefully with collaborators while unit testing is what mocks are all about.
*(See also the earlier article, Why not just integration tests?)
Dealing with the Collaborators
As the EasyMock page says:
Unit testing is the testing of software units in isolation. However, most units do not work alone, but they collaborate with other units. To test a unit in isolation, we have to simulate the collaborators in the test.
A Mock Object is a test-oriented replacement for a collaborator. It is configured to simulate the object that it replaces in a simple way. In contrast to a stub, a Mock Object also verifies whether it is used as expected.
Examples
The syntax varies with the mock framework: a couple of examples from the HippoMocks 3.0 tutorial follow:
mocks.ExpectCall(barMock, IBar::c).With("hello").Return(42);
and (here’s a big reason I can’t ignore mocks):
mocks.ExpectCall(barMock, IBar::c).With("hello").Throw(std::exception());
See that? Expect the c() method to be called and throw the specified exception. This can be really valuable in testing your error handling – you don’t have to laboriously set up a complex net of conditions to get a subsystem to really throw an exception – you just tell the mock object to throw it and that’s that.
Mocks not a replacement for integration tests
Mocks aren’t a total replacement for integration tests. Many times the problem is not in class A nor class B, but in their collaboration. Integration tests are valuable to prove that the collaboration works as expected. But it is faster and easier to diagnose a problem in the collaboration of units that have been thoroughly unit tested in isolation. To put it the opposite way, when you’re working with a group of collaborating units that haven’t been thoroughly tested in isolation, “the problem could be anywhere”!
Limitations and downsides of mocking
It’s a Simulation
When using mocks to test your unit in isolation, you’re simulating interactions with the unit’s collaborators. A risk inherent in simulation is the risk that the simulated interaction diverges from interactions that use the real collaborator.
Coupling between the test and the collaborators’ implementation
In insulating a unit from its collaborators, mocks couple your tests to those collaborators. When you change the way calls to a collaborator work, you have to adjust mocks’ expectations as well. This coupling can decrease the flexibility of your code if not understood and managed (this answer to “When should I mock?“‘s Mock shortcomings section gives examples of this problem).
I’ve experienced this kind of brittleness, and it is an important consideration. My hope is that such brittleness can be avoided by relaxing the expectations down to the minimum needed (see for instance the “2. More Complex Stuff” section of the HippoMocks 3.0 Tutorial ). It’s a challenge to do this right the first time, though, because the brittleness of a test using mocks may not be apparent to you until time passes and the collaborator changes its implementation, breaking a test. It may be a learning process for mock users, to develop a standard practice for the right kind of looseness that supports .
Why Not Just Fakes, or Ad-Hoc Mocks?
There are ways to get independent unit testability without using a mocking framework.
- You can create your own fakes (i.e., create a testing implemention of your collaborator’s interface) — certainly handy at times
- You can create your own mocks without using a framework (the article Should we use a mocking framework?)
The thing to watch about these approaches is the amount of boilerplate code that’s required. During my experience in Java-land, using EasyMocks in conjunction with Unitils reduced the boilerplate code down to a really small amount. It was great.
Which Framework?
I’ve written up some initial results on that front as well.
Conclusion
While using mocks without care can lead to brittle tests, their power for easily testing the unit under test’s responses to interactions with a collaborator cannot be ignored. I hypothesize that by setting up the most relaxed expectations that fulfill the test’s need, the brittleness problem can be avoided; but more study (or at least more Googling) needs to be done to prove or disprove this. In the meantime, I believe the right course of action is to proceed (with our eyes open) with the use of mocks, since the current alternatives, including having integration tests but no unit tests, must leave us in the position of leaving certain code untested. I don’t see how it wouldn’t anyway…
Shoot, now I need to find a good C++ code coverage tool, to prove it! : )