Getting Started with Test-Driven Development [and Key Benefits]

Test-driven development (TDD) is a software development practice that has been steadily growing in popularity. Its three-step process can give you the confidence to refactor and improve your code using a workflow that’s proven to boost your team’s productivity in the long run.

While it sounds promising in theory, many developers struggle to apply this concept and make a number of mistakes while applying TDD. Unfortunately, this means a lot of developers give up on it before they’re able to see some of the benefits. Run a search on any search engine and you’ll see a number of forums and articles about developers’ frustrating experiences with poorly executed TDD. 

But through my experience as a full-stack senior developer, I’ve seen first-hand how the TDD process, when implemented correctly, can massively benefit teams. In this article, I’ll unpack how TDD methodology works, the pros worth noting, some common pitfalls to be wary of, and tips to successfully implement this practice into your development process.

What Is Test-Driven Development?

Introduced by Kent Beck in the late 1990s as part of the Extreme Programming (XP) methodology, the core concept behind test-driven development is to write the test before the actual production code. This gives you the ability to think of how your code will interact with a hypothetical client, encouraging you to think about the design earlier. As such, this process promotes modularity, loose coupling, high cohesion, and good separation of concerns. TDD, then, is intended to produce software that is higher quality with less defect density and a higher Return on Investment (ROI) in the long term. 

Let’s dig into what that all means. 

What Makes Software High Quality?

The desired outcome of applying TDD is high-quality software. But what does “high quality” mean? In truth, the exact definition will vary from person to person. I like to define it from the client’s perspective. Put simply, it needs to fit the client’s requirements and expectations. It’s not just about building software, but doing it in an efficient way that will bring a good ROI to keep stakeholders happy.

The client and developer will likely have different metrics to evaluate quality. But from the developer’s point of view, the aim is internal quality: how easy it is to maintain the software in the long run. 

In the developer world, high-quality software is widely agreed to include the following properties, according to Dave Farley: 

These aspects lead you to higher maintainability. 

The idea that higher quality is more expensive may be true for most things, but not for the internal quality of software development. Martin Fowler wrote an interesting article on the subject and suggests that high-quality software is actually cheaper to produce. Simply put, his reasoning is that by keeping a higher level of quality, your team will be able to iterate faster. 

Writing software is not like building a house. You won’t be able to have all the specifications written in stone. It’s more like writing a book and the software evolves as you learn during the process. Faster iteration is the rule of thumb:

Even if the requirements are changing, with higher quality you can change the software at a faster pace and converge to the most appropriate solution for the client. 

Is Quality Software One of the Benefits of TDD?

There is evidence to suggest TDD can improve software quality. According to a study by Microsoft, TDD helps to reduce a software’s defect density without significantly reducing the productivity of the development team. The table below illustrates Microsoft’s findings: 

My experience is similar to what Microsoft’s study showed. Here’s how I’ve found TDD promotes quality improvement:

An added benefit is that the tests also become up-to-date software documentation. This is especially useful for software that quickly becomes outdated (or is simply nonexistent), as it keeps existing employees up to speed and helps you onboard new developers quickly.

How Does the TDD Methodology Work?

The TDD process is composed of three golden rules:

Notice that, when mentioning unit tests, I’m not talking about them as a type of test or the smallest piece of code that can be logically isolated in a system. Rather, think of the Unit as a cohesive and modular portion of the software. This could be a class, an entire module, a view from your web framework, or a smart component from your UI implementation.

Red-Green-Refactor

This workflow is also referred to as the Red-Green-Refactor process – the mantra of TDD. Let’s explore each step:

Red – Write the Test

In this stage, you’ll introduce a new test scenario for functionality that doesn’t exist yet. It’s important to resist the temptation to write more than one test case at a time. That way, you can focus on learning more about your scenario as you write it.

Your test scenarios may be using unit tests, integration tests, functional tests, or even end-to-end tests for this task. It depends on each case.

Green – Make the Test Pass

Now, implement the code but don’t over-optimize it. You want the test to pass so you have a strong foundation for you to refactor.

It’s ok to take short paths or even copy and paste at this moment. You’ll have time to improve it in the next step!

Refactor – Improve the Code

Once the test passes, it’s time to improve it by applying relevant refactoring processes. Remember: you’re testing a cohesive portion of the software. Internally, you can apply best practices to ensure your code is readable and maintainable. But from the test perspective, the code simply needs to fulfill the tests, which means it must behave the same way.

The important aspect here is the software behavior. By using the TDD approach, you’re focusing on the behavior rather than the implementation. This is the key to maintaining software quality: you won’t be over-optimizing it or adding unnecessary components, and your software will do what it says it does!

The main benefit, then, is that you’ll catch problems earlier and spend less time debugging code in a real environment. You’ll also find the time you spend refactoring will become less as you conduct sequential tests. As pointed out by Robert Martin in Refactoring: Improving the Design of Existing Code, “as you test each step, your code is always in a state of ‘working’ and you don’t need to refactor that much.”

For those who like hiking, I like the analogy of a Prusik knot, used for climbing. Two sliding ropes are used to allow the climber to go up with confidence and safety. The red-green-refactor from the TDD process can give you the same feeling!

Hands-on Example

Each language has its own tooling for testing. For instance, Python has unittest and pytest. JavaScript and TypeScript have jest, mocha, jasmine, and a couple of other alternatives. Other languages have their own alternatives as well.

Let’s look at a Python example. 

Assume we’re implementing the good old TODO list application and want to implement the REST’s create verb. A test would look like this:

1def test_should_create_task_for_a_valid_payload(client):
2    payload = {'content': 'Get used to TDD'}
3    response = client.post('/api/tasks/', payload)
4    assert response.status_code == 201
5    created_task = models.Task.objects.get()
6    assert task.content == payload.content
7    assert response.json() == {'id': task.id, 'content': task.content}
8

This code is written with Django-rest-framework in mind, but the idea would be the same for any other language, framework, or database you use. For instance, this is a similar example using NextJS and DynamoDB.

Implementing a view that fulfills this test case will now be a simple task, more or less like the code below:

1class TaskSerializer(serializers.ModelSerializer):
2    class Meta:
3        model = models.Task
4        fields = ('id', 'content')
5
6class TaskViewSet(viewsets.ModelViewSet):
7    queryset = models.Task.objects.all()
8    serializer_class = TaskSerializer
9

I won’t go deeper into this example as it’s very specific to each language, but one thing to note is to keep the implementation to a bare minimum to fit the test scenario.

For instance, if you need to add authentication or validation, then finish the first red-green-refactor cycle only to repeat it. Focus on each individual behavior, one at a time.

Also, as my task object gets bigger in complexity and as I add more test cases, I also add abstractions to common operations (like defining a generic payload or comparing the created object against the payload). For instance:

1def test_should_create_task_for_a_valid_payload(client, payload):
2    response = client.post(‘/api/tasks/’, payload)
3    assert response.status_code == 201
4    created_task = models.Task.objects.get()
5    assert_task_matches_payload(created_task, payload)
6    assert_response_matches_task(response, created_task)
7

You should try keeping the test as readable and focused as possible. Also, the test name is very important. As the first abstraction level of the test, it should resemble what the test really does.  For instance, if you would test the content shouldn’t be blank, instead of naming the test test_create_blank_content, call it test_should_refuse_blank_content.

Notice that you should avoid testing implementation details. Your test suite must be resilient to allow you to refactor the code in the third step of the red-green-refactor process.

In the end, we can borrow the given-when-then process from Behavior-Driven Development (BDD) to define the simpler abstraction we want:

Your unit (such as a class, module, or function) would be a black box that receives the parameters and returns something or causes this side effect (in our case, the API output and the database side-effect).

Debunking the Cons of TDD 

A quick search through Quora and StackOverflow threads will point out that many developers find TDD practice complex and unnatural to follow. But when implemented correctly, these potential downsides are unlikely. Here are some of the common challenges developers expect with TDD, and how they can be avoided.

It’s true that implementing test-driven development may be slower at the outset. However, it quickly evens out: because you save so much time debugging, you will likely save time in the long run. As you get better at implementing and following the TDD approach, you’ll be able to develop tests more quickly, and your productivity will increase. 

Because you’re testing as you go, you will be encouraged to use high cohesion, modularization, and loose-coupling. Together, these all will help to improve code quality. At the end of the day, though, test-driven development won’t protect you from yourself. Write clean code and follow best practices!

If you’re new to TDD, it might take you a little while to easily write tests. But if your code is modular, your tests will be easier to implement. If your code is tight-coupled, you will have hard-time testing. It could be even a sign that your implementation should be reviewed!

If you focus only on the UI behavior and you have help from your tooling, it is possible to test UI! For instance, React developers can use the combination of testing-library and semantic HTML to write tests for smart components (the ones that carry the software business logic).

To successfully test your UI,  avoid testing implementation details. You may want to consider using alternative tools like Storybook and visual regression test tools for dumb components (which are simply presentational).

Let’s suppose we are implementing the task creation form, using React. A test would look like this:

1it('should allow create task for valid data', () => {
2  render(<TaskCreate />)
3  const contentField = screen.getByLabelText('Content')
4  fireEvent.change(contentField, {target: {value: 'the value'}})
5  fireEvent.click(screen.getByRole('button', {name: 'Submit'}))
6  await screen.findByRole('alert', {name: 'successful message'});
7}) 
8

This example uses jest and testing-library. This could be achieved with end-to-end testing using cypress, puppeteer, or a similar tool. I personally prefer using testing-library and integration tests to keep a fast and responsive continuous integration process.

Notice that, by querying UI elements by their semantic HTML behavior (e.g. the element’s role), I’m making the test more resilient. It doesn’t matter what UI toolkit I use. If I change from bootstrap to material-UI, there’s a good chance the test won’t need any refactoring.

This even demonstrates how the TDD methodology influences you on separating concerns (in this case, by isolating the smart components from the dumb/presentational ones).

5. “My product is constantly changing/evolving so my tests become irrelevant and need to be re-written”

This might be an indication that:

When you change the business logic, the tests might change to reflect it, but they rarely need to be re-written from scratch, they will evolve with the application implementation.

Let’s first look at an example to demonstrate it. Suppose you have a simple registration API view implemented in Python/Django with the following test:

1def test_registration_should_create_user(client):
2  payload = {'name': 'Michel', 'email': 'michel@email.com'}
3  response = client.post('/registration/', payload)
4  assert response.status_code == 200
5  user = User.objects.get()
6  assert response.json() == {'id': user.id, **payload}
7  assert user.name == payload['name']
8  assert user.email == payload['email']
9

Now let’s suppose you have to accept the user country and reject users from a specific country. You would implement it in two TDD cycles, one for accepting the country, other to reject specific countries:

1def test_registration_should_create_user(client):
2  payload = {'name': 'Michel', 'email': 'michel@email.com', 'country': 'BR'}
3  response = client.post('/registration/', payload)
4  assert response.status_code == 200
5  user = User.objects.get()
6  assert response.json() == {'id': user.id, **payload}
7  assert user.name == payload['name']
8  assert user.email == payload['email']
9  assert user.country == payload['country']
10

As you could see, the tests were upgraded to support the new behavior. We would need to add one more test to support rejecting some specific country:

1@override_settings('REJECTED_COUNTRIES', ['XY'])
2def test_should_fail_for_reject_country(client):
3  payload = {'name': 'Michel', 'email': 'michel@email.com', 'country': 'XY'}
4  response = client.post('/registration/', payload)
5  assert response.status_code == 400
6  assert response.json() == {'detail': 'Country rejected!'}
7

The important point is that you are testing an isolated unit, using the concept of unit presented earlier in the article (a cohesive and modular portion of the software, in this case, the whole registration view).

If the logic to rejecting the country got really complex, you might want to isolate it in its own “unit”. You can abstract the country validator in your view, for example, using dependency injection.

1@override_settings('COUNTRY_VALIDATOR': lambda country: false)
2def test_registration(client):
3  payload = {'name': 'Michel', 'email': 'michel@email.com', 'country': 'XY'}
4  response = client.post('/registration/', payload)
5  assert response.status_code == 400
6  assert response.json() == {'detail': 'Country rejected!'}
7

Notice that you could do the same with mocking, which allows you to replace a function or class at runtime during tests. You must be careful with mocking though as it can lead you to write highly-coupled code. See an example of a bad test below:

1@mock.patch('country_validator.CountryValidator.is_valid', return_value=False)
2def test_registration(client):
3  payload = {'name': 'Michel', 'email': 'michel@email.com', 'country': 'XY'}
4  response = client.post('/registration/', payload)
5  assert response.status_code == 400
6  assert response.json() == {'detail': 'Country rejected!'}
7

The test code might work, but you replaced an internal method from a dependent class. The registration code shouldn’t be aware of the CountryValidator internal shape because it can change without your control!

Even if the business logic changes dramatically, TDD will still guide you through the necessary changes. And if rewriting from scratch, you still have benefits of TDD (e.g. avoiding debugging manually or focusing on one problem at a time). In the end, with TDD practice, you should save time!

My Tips for Practicing TDD

Throughout my time practicing test-driven development, I’ve learned the following practices can improve the experience: 

I won’t go into detail about BDD, as it goes beyond the scope of this article, but what you do need to understand about it is: separate these three parts of your test into independent logical blocks.

Conclusion

As we’ve seen, there are many reasons to use test-driven development in your projects. From reliability and reduced bug density to easier maintenance and scalability, the benefits TDD can provide are massive. 

Is TDD hard? Maybe, but the trick is getting it into your routine. Like any new habit, the first time can feel like you’re doing a lot of extra work and not getting anything out of it.  But as you get used to it, TDD will become a lot easier and smoother. You’ll quickly find the benefits greatly outweigh the cost, and you likely won’t want to revert to any other approach. 

For me, getting started with TDD is similar to my early days as a programmer. If someone asks me, “is programming hard?” I’ll quickly say no; after all, it’s something that I do every day. But if I really take a moment to think about my early days and all the work I had to put in, I realize it wasn’t actually simple. But with a little hard work and dedication, it’s become that way.

Now you know a bit more about test-driven development and how to apply it, I hope you’ll consider it for your next project. With practice and persistence, I’m confident you’ll find it worth the time to explore.