How to write thorough unit tests

This is the second post of my series on writing better tests. In my previous post, I discussed the characteristics that make unit tests good. I mentioned that, amongst other things, good tests must be thorough, meaning they need to check everything that is likely to break.

This post focuses on thoroughness, and will help you in addressing it with a repeatable approach for your Force.com projects.

Why is thoroughness so tricky?

Of all the characteristics of good tests, thoroughness is probably the most difficult to achieve. There are some recurring challenges that most technical leads might have encountered at some point.

  • It doesn't come for free. There are things the platform can do for you, and unfortunately thoroughness is not one of them. It isn't something that can be easily automated. It costs time and money, and will affect your velocity.

  • The happy path problem. When writing tests developers tend to follow the path where everything works, which is what leaves room for bugs in the first place. Thinking how to break your own logic is counterintuitive, and requires a form of reasoning that most people find difficult.

  • Inconsistent approaches. Every developer in the team brings a unique way to approach problems. The challenge for testing is that developers might approach testing differently, not only in terms of style, but in terms of depth. Different people might have a different perceptions of how deeply things should be tested.

  • Over-testing. Tests have a cost, therefore every one of them should be meaningful. Over-testing happens when developers write irrelevant tests (i.e. testing things that have been already tested), when tests go in excessive depth (i.e. retesting the platform), or beyond the scope of the customisation.

A checklist for writing thorough tests

As I do pretty much for everything, I came up with a checklist to summarise what I have learnt from experience. It's a simplification that provides a solid baseline that can be adapted to a wide variety of Force.com projects.

This checklist gives you:

  • Key scenarios you should cover in your unit tests
  • Key variations for each scenarios
  • Things you should assert for each scenario and variation
  • A reusable reference to ensure your tests are thorough, whilst avoiding over-testing
Test scenario Key variations Checks
Positive scenario (the happy path)
  • Entry conditions met (code expected to be executed)
  • Result exists (is not null)
  • Result is correct
  • Result is in the expected format
  • Result is within the expected value range
Negative scenario
  • Entry conditions not met (code not expected to be executed)
  • Bad/invalid inputs
  • Result does not exist
  • Expected exception is thrown
  • Expected error is returned
Boundary conditions For numeric inputs:
  • Null
  • Zero
  • A negative value
  • Min possible value
  • Max possible value
For date inputs:
  • Null
  • Yesterday / Today / Tomorrow
  • 1/1/1970
  • 29/2 on a leap year
  • Last day of a month with 30 days
  • Last day of a month with 31 days
  • Overlapping dates
For string inputs:
  • A null string
  • An empty string
  • A string with white spaces
  • A very long string
  • A string with uncommon unicode characters
  • A string with potential code injection (JS, SOQL, SOSL)
For collection inputs:
  • A null collection
  • An empty collection
  • A non-null collection (10 items)
  • A collection with some missing items here and there (10 items)
  • Result exists (is not null)
  • Result is correct
  • Result is in the expected format
  • Result is within the expected value range
Error conditions
  • Class instance not properly initialised (i.e. when an init() method is required)
  • A required custom setting is missing
  • Result does not exist
  • Data integrity is preserved (no partial changes to data)
  • The expected exception is thrown
  • The expected error is returned
Large datasets (for triggers and methods with input collections)
  • A batch of 200 records
  • No governor limits are broken
  • The result set exists
  • The result set has the right cardinality (expected number of items)
  • The first item in the result set is correct
  • The last result in the result set is correct
  • The items in the result set are in the right order
Security
  • User is authorised
  • User is not authorised
  • User is authorised, but some involved fields should not be accessible
  • Result does/does not exist
  • Field-level security is not bypassed

A few recommendations:

  • Break down each test scenarios and all relevant variations into separate methods
  • Keep each test method simple and with minimal logic (try to avoid loops, ifs, etc.)
  • Use assertions to perform all relevant checks
  • Boundary conditions: focus on the relevant variations that apply to your case