Not writing automated tests? You have a problem waiting to happen

Unit testing, Integration testing, A/B testing, stress testing… you name it. There are many ways to ensure your software is working correctly in an automated way.

The times of manually checking if your functionality “runs fine” are long gone. In any Software development team, the testing part is (or should be) as integral as writing the code itself.

I personally advocate testing so heavily that I reject PRs on the base of missing or having wrongly written tests. There’s certain anxiety on the back of my neck when having to deploy code that’s not through-fully tested. I just can’t help it.

For a better Software development world, you shouldn’t tolerate it either.

As applications and teams grow bigger, the needs for automatic testing suites grow with them.

You may say “But Ron, we have a QA team to do the testing for us”.

Sure, having a QA team is a nice thing, and surely they will be able to test your features in due time, but the ownership of code should come from development. It is your responsibility as the person who writes the code to ensure it’s working as best as possible, and to reduce the load on the QA team.

Additionally, relying on your QA team promotes a “throwing over the wall” behaviour, in which developers don’t follow up to make sure their code is production ready the moment it leaves their machines.

The less tests you write, the more potential issues you’re introducing. I’ve learnt this the hard way. It’s almost like divine justice being served. Every time I’ve decided to skip testing certain features, I’ve introduced an error in production. Fortunately other measures like heavily alerting and small deploys usually had helped me find the issue quickly and roll it back, but it’s still painful.

How to best write automated tests

There’s this concept of a testing pyramid. It looks something like this:

The idea is that the majority of your tests are going to be unit tests. They’re easy to write, they are fast to run and in general they should even help you clarify your functionalities.

Afterwards you have integration tests, which are more complex and make sure the different parts of your system are interacting correctly.

At last you have UI and potentially manual tests.

There are many variations of this pyramid, with more detail or steps in between but the concept is the same. You go from less to more complex tests, moving up the pyramid.

Your application should at the very least have the bottom of the pyramid covered. Unit tests are essential for every system.

If you engage in practices like TDD or BDD, it’s even better. The practice of writing your tests first helps you find functionality gaps and guide you through your feature building. You can consider the different scenarios your app needs to handle before you even start writing your first line of code.

Additionally, you should make sure to pick the right testing framework for you. Pick something that makes your life easy. You don’t want to be spending all your time figuring out how your tests are being set up, but writing them.

Do you need 100% test coverage?

Simple answer is no. Test coverage is somewhat a useless metric.

There it is, I said it!

More often than not, test coverage can give you a false sense of security and make you sloppy when considering edge case scenarios.

Entertain my idea with the following example

def debit(account, amount)
  account -= amount
end

def credit(account, amount)
  account += amount
end

Two over-simplistic functions to debit and credit a given amount from an account, similar to a banking transaction.

Logically It is expected that you add the amount to the account in the credit scenario, and the opposite on the debit scenario, right?

There are a lot of validations to be performed, like making sure the account is higher than 0 on debit for example.

Now, consider we write the following tests

it "debits the amount" do
  expect(debit(100, 20)).to eq(80)
end

it "credits the amount" do
  expect(credit(100, 20)).to eq(120)
end

Our tests have now 100% coverage!

See the problem here?

Coverage in itself is a horrible metric, it doesn’t consider edge cases, just that you’ve “covered” all parts of the code.

The following logically incorrect situations won’t be considered at all:

debit(100, 500) == -400
credit(100, -200) == -100

When considering the use of test coverage, bear in mind that it’s not a substitute for logically and thoroughly thinking about all your test cases, edge cases and impossibly looking cases.

You can just as easily have an application with merely 70% test coverage but having a strong set of tests that make sure the code is handling all expected and most unexpected scenarios.

Drawbacks of writing tests

Writing tests is time consuming. It doesn’t move your application forward like, say, a new feature. When your line manager asks why that feature is not ready yet, and you say you’ve been spending a lot of time writing tests it’s sometimes not pretty.

It’s a necessary time consuming task that needs to be included as an essential part of your development process. You and your team need to understand that if writing the code for a feature takes 1hr and your automated tests take 2hrs, that feature took 3hrs until done, not 1hr.

Additionally, test scenarios need to be rewritten every time a feature is updated, and new scenarios need to be added every time a bug that wasn’t considered in the tests is discovered. It’s an ongoing time consuming process.

That being said, do not think for a second that it’s unnecessary.

What test suite should you use?

Use something that makes it extremely easy, even enjoyable to write tests.

Sometimes writing automated tests is dragging and plainly boring. When pressed for time, you will hate having to take care of the tests, but with the correct testing framework, you’ll make the process overall better.

What if I’m the only one writing tests on my team?

Hold your teammates accountable, advocate the benefits. Be repetitive and annoying if necessary.

The cost of letting untested issues into production is just too high. Once that broken feature reaches your users, it’s going to negatively impact the business. It may even mean losing users altogether.

When it comes to making sure everyone is pulling their weight and making sure all systems are operational, writing tests should be among your top priorities.

An additional benefit is that you avoid regressions when collaborating on features. If a great testing suite is in place, you can guarantee that whatever you do is not going to affect your teammate’s previous work if all tests are ok after you’re done.

Conclusion

Testing is as important as writing the features itself when it comes to Software development. Do not ignore them, or you have a problem waiting to happen. No matter how good you think your coding skills are, you increase your probabilities of making mistakes if there isn’t an automated way to check everything is working correctly.

In big and interconnected systems, the need is even bigger. As your platform grows in size and complexity, you need a mechanism that ensures you can safely build on top of what you already have, and that’s automated testing.

If you enjoyed the article don’t forget to subscribe for more content and to share.