Whether you’re doing TDD or not, having tests that verify your code is a great source of confidence that whatever feature you added, or whatever thing you’ve just refactored, didn’t break any other part of your codebase.
At COBE, we recently shifted from creating excuses for not unit testing to creating actual unit tests. I decided to write about our experience in this two part article. This first part will deal with the whys and whats of unit testing itself, while the second part will be a bit more practical and I’ll talk about how we make our codebase testable and how we write the actual tests.
I bet at one point you had a class in your codebase that was a giant beast, and everybody was afraid to touch it, even though it badly needed a refactoring.
Having unit tests provides you with enough confidence to refactor those kinds of classes. Even while working on new features, running your existing tests will give you a pretty good idea about whether or not you’ve broken anything in your codebase.
Once you have tests for the behaviour of that class, you could change any line in there and know almost instantly and fairly certainly whether you’ve just broken something or not. The fear is then gone, and you can refactor and change that class as you want.
Fear trumps creativity, and in turn trumps quality. No programmer should ever be afraid of code, we are the ones that write it.
I think there’s two types of documentation in your code: active and passive.
Passive documentation is comments or any external documentation you or a team-member wrote for your code. The disadvantage of this type of documentation is that you have no idea how up-to-date it is, and there’s nothing forcing anybody to read it. (Let’s be honest, we all need to forced sometimes.)
Active documentation is “documentation” that shows up during the development process in a visible way. This can be anything from proper access control for your methods to marking a method as deprecated.
I consider tests a kind of active documentation, because they describe how the application should behave, and tell everybody when it’s not. This is incredibly useful, especially when working with a complex codebase and a lot of people.
You should write tests that test the behaviour that you need. Don’t test implementation. I first heard this concept put into words in Orta Therox’s excellent ebook on iOS testing. Basically it means that you should treat methods (and sometimes classes) as black boxes, and all you know is what you put into the method or class, and observe the output.
In other words, think about what the end goal of each class or method is, and test that. Don’t test the implementation details in between.
This gives you the freedom of refactoring. Provided you don’t change method names, your tests should run even if you change the whole implementation of the class/struct you are testing.
Practically, this means you test endpoints. I look at each class as a little electrical component, it has a + on one end, and a - on another end, and I find the most useful tests are the ones that only observe the difference between those points, not what’s in-between.
Ideally, everything. But we’re not in an ideal world. If you think of your application in layers, tests are always going to cover a subset of those layers. Some layers are just too high, and some are just too low to test.
My assumption is that you can’t unit test external frameworks. Most apps begin and end with an external framework. That is to say, your app begins with a database of some kind (maybe Alamofire or Core Data), and ends with UIKit. (or vice versa)
By extension, this means that classes that talk to the external framework directly are not unit-testable. This is why the classes that talk to actual external frameworks should be as thin as possible. I will talk more about this in Part 2, where we’ll discuss testable architecture.
Everything between those external frameworks should be unit tested. But sometimes we don’t have the luxury of time. In that case, I propose this:
The more complex the class, the more you should write tests for it.
Bugs in large complex classes are harder to spot, and it’s much easier to break them
with changes, so you should prioritise these. Get rid of that fear early on.
If you found a weird bug, write a test for it.
It will
help you debug, and also nobody will ‘reimplement’ that bug in your code.
If you want to write a comment telling somebody what to consider when
refactoring a class, write a test instead.
First of all it’s much more
visible than a comment since there’s a little red diamond in Xcode, but also
it’s more useful to future programmers since they can just run tests to see if
they broke anything.
The less you see the class, the more you should test it.
UI tests
are really useful in that they save a lot of time, but UI bugs are fairly easy
to spot. You literally see them. Other parts of your code, however, could have a
silent bug that your users might experience, but your testers or you don’t.
Write tests before you refactor something
Trust me, it will make
your life easier, and will probably end up saving you time.
A general rule is that you should write tests for the class you want to write tests for the least. The most useful tests are often the hardest ones to write.
When I think of pure functions, I often think of mathematical functions, i.e. those that take each element from set A, and assign it exactly one element from set B.
These are functions you can test simply by calling them with some parameter, and comparing the result to the expected outcome. This is an ideal function for a unit test, and it’s the easiest to write. Unfortunately, I find these functions are in the minority, and often limited to simple utility classes.
These tests verify that one class is propagating an action or a piece of information to another class. For instance, if I click a login button, I want to make sure a method in my networking class is called with the correct parameters. Or, if I change some setting, I want to make sure my UserSettingsManagerOrWhatever is called. I find I write these tests the most.
These are really useful for future programmers working on your codebase. Also, when you write tests for each layer of your application, it serves as a verification that all your classes are calling each other as they should, and that bugs lie in the implementation, not the glue code.
Sometimes functions are not mathematical functions. Sometimes you have more than one kind of input or output. For instance, most methods dealing with the network have a different kind of output when there’s no internet connection, or something breaks on the server.
This is why most methods with some sort of completion handler must have tests that verify both a successful path (the “happy path”), and the not-so-happy path, to see if they handle this correctly.
Unit tests are, by definition, isolated. The only reason a unit test should fail is if something changes inside the actual class you are testing. Nothing outside of the class should be able to break the test. This is what makes unit tests useful, because you know when a test fails where the problem is. (Although you might not know what the problem is.)
In the case of simple mathematical tests, this isn’t a problem, because each function, being a pure function, is completely isolated from anything else in your code.
A simple mathematical function
Functions with side-effects, however, are not that easy. If the side effects are limited to the same class, your life is still fairly simple — you need to set up the class in the beginning, run the code you are testing, and see if the class is changed in a way you expect.
Method with side-effects inside a class
If that method is calling a different class, we use mocks. Mocks are substitutes for your classes dependencies, that you provide in your tests and can observe. Mocks are essentially the output, and sometimes the input of the class you are testing in that case. All you need to do is provide your mocks to the class, and see if the correct mock methods are called, or something is altered in the mock.
Method that is calling another class
Let’s say you have some boolean property in your class. Naturally, you’ll have to test if the class responds correctly to true and false state of that property. When you add another one, there are four possible combinations of those properties. Add another one and suddenly there’s eight different states your class can be in!
State is your enemy when programming, whether or not you’re writing tests. Make sure to keep it in check and only depend on state that you absolutely have to.
We ship great code we are proud of, and we test as much as possible, but there are other considerations. Sometimes you’re better off learning something new than writing a test for a one-line method.
On the other hand, Xcode registers every line of code that is called during tests as ‘covered’. A line of code may be called without being tested, and some lines of code must be called more than once. When testing, make sure to take all possible considerations into account. Test all paths trough your method, test if it works with no data, or lots of data, or unexpected data.
Coverage can only tell you which parts of your code aren’t covered. You cant trust it to tell you which parts are.
In some cases, future programmers (including you), might use a class in a way you don’t intend to. Make sure you test those cases, and make sure your class responds to those correctly.
If you have multiple delegate methods, make sure your class calls the correct one. Or make sure the delegate method gets called the correct amount of times. Also, make sure your classes don’t propagate wrong data, like a login call with an empty password. Sometimes it’s useful to verify something is not happening.
This a general overview of unit testing and how we do it on a more abstract basis. In Part 2, I’ll talk more about how we design our iOS code in a way that allows unit tests.