Mockito ArgumentMatcher Pitfalls

Avoiding the pitfalls of tightly coupled mocking

March 23, 2020 - 4 minute read -
testing mocking mockito argument-matchers kotlin

In Mockito, you can configure your mock’s behaviour based upon the arguments provided to its member functions, this is done using ArgumentMatchers.

In this post I want to highlight some “rules of thumb” for using ArgumentMatchers which will help you avoid some pitfalls and instead build more resilient tests.

ArgumentMatchers?

Broadly speaking, Mockito ArgumentMatchers are used to define the conditions under which a mock’s behaviour should apply. They can also be used for argument verification, although here I’m going to focus on their use in defining behaviours.

The condition matching is often done implicitly. In this case, the mock service will return a list of songs when the genre argument is equal to “jazz” -

`when`(service.getLatestSongs(genre = "jazz")).thenReturn(listOf(song1, song2, song3));

We can also be more explicit and directly use Mockito.eq() in the functionally equivalent -

`when`(service.getLatestSongs(genre = eq("jazz"))).thenReturn(listOf(song1, song2, song3));

There are a range of ArgumentMatchers available, such as startsWith(), contains(), argThat() etc to match against specific conditions for the given input variable types. Also see AdditionalMatchers for even more.

We can also match broadly against the input argument, so that the mock behaviour should apply, essentially regardless of the input argument value. This mock will return the list of songs, for any genre -

`when`(service.getLatestSongs(genre = anyString())).thenReturn(listOf(song1, song2, song3));

There are other broad matchers such as: any(), anyInt(), anyByte(), anyBoolean(), anyIterable() etc.

Note for Kotlin, I recommend using Mockito-Kotlin.

Use specific or broad matchers?

This classification prompts the question: when should we use specific matching and when should we use broad matching?

One mindset is that by liberally using the specific matchers when defining our mocks, we help ensure the correctness of our class under test - after all, if the mock only responds when given the genre “jazz”, we are implicitly verifying that our class under test interacts with the SongService correctly by providing it with that argument. If we were using the anyString matcher, then we could be calling getLatestSongs with complete rubbish - which obviously wouldn’t be what we want!

However, in my experience, this overuse of specific argument mocking matchers can make your tests brittle and difficult to maintain.

The pattern increases the complexity of an individual test case by making it become more of a “white box” test; it more explicitly mirrors the implementation details of the class under test. For good measure, this is often repeated in each test case, adding to the pain. When the implementation changes, your tests require more refactoring.

Also when using more complex types, there is the burden of maintenance and an expanded footprint for your test fixtures to ensure that meaningful and correct values are used in tests. This can get quite tedious with a large suite.

If you really, really care about how your class interacts with its collaborators, then at least test that explicitly. For example -

verify(service.getLatestSongs(genre = eq("jazz"))); // side note; here "eq" is being used as a verification matcher

The intention of this assertion is clearer than when defined in the mock and also any failures are more readily identifiable. When using specific matching, if our provided argument doesn’t exactly match the condition (is equal to “jazz”), then the mock would return its default value: null. Chances are that this would eventually cause your test to fail, but the reason might not be particularly clear.

When to use specific matchers?

Of course, specific matchers can be useful. You can define different mocking behaviour, based upon the input, for example different responses to requests for songs of different genres -

`when`(service.getLatestSongs(genre = eq("jazz"))).thenReturn(listOf(song1, song2, song3));
`when`(service.getLatestSongs(genre = eq("rock"))).thenReturn(emptyList());

TL;DR

  • Don’t use specific argument matchers to implicitly verify collaborator interactions.
  • Instead, if deemed necessary, define targeted tests which explicitly verify these collaborator interactions.
  • By default in other cases, use the broadest appropriate argument matcher when mocking.

Related source code can be found here.

This post is Creative Commons Attribution 4.0 International (CC BY 4.0) licensed.