How to Create and Maintain Technical Documentation for Small to Complex Projects

There are many types of documentation in the software world. Of these, technical documentation is one of the most frequently neglected.

The fast-paced nature of the industry leads to the conception that time spent documenting is time lost developing and delivering features. Over time, a development team’s documentation debt (a type of technical debt) can build to a point where the idea of tackling it becomes daunting. As a project scales and teams experience churn, information can become siloed or even lost. This can then impact the efficiency of the team. On the flip side, well-documented projects are more likely to run smoothly, especially at scale.

In this article, we’ll discuss the importance of technical documentation, why it is often neglected, and the impact it can have on development teams.

Why Projects Need Technical Documentation

Common tasks and important concepts explained by the docs don’t require one-on-one conversations for knowledge transfer, making things like onboarding and process management easier. Projects become resilient to unexpected turnover since technical documentation lets knowledge reside within a project, not within the individuals currently working on it. If a team member leaves, typically their knowledge goes with them, but this isn’t the case when things are properly documented. Well-documented projects are also more attractive from an acquisition perspective, and the technical documentation can assist in the associated due diligence process.

The core misconception is that writing software documentation is difficult or time-consuming. When done correctly, it’s straightforward to write and returns multiples on effort in terms of time saved. 

The Risk of Not Having Technical Documentation

Every project has a lead or senior developer that knows the ins and outs of the system like no other. These are the people that have been part of the project since the very first line of code, or that have been working on it for a long period of time. They are incredible assets, with the ability to see both the bigger picture as well as the details and help the team navigate the codebase.  

We should be upfront here and recognize that documentation is not a direct replacement for that person’s knowledge, no matter how well-written and organized your docs are. But in the case that your developer disappears, it’s the next best thing. 

And believe me, your developers will disappear eventually. Hopefully, they won’t be literally hit by a bus or any other sort of tragedy, but they are likely to move to a management position over time as a technical project manager or technical leader and lose touch with the codebase, move to another company, or simply pursue other interests.

The Bus Factor

If you’re relying on a single person to know how your application works, even temporary departures such as vacations or parental leave can result in a costly stall in your project. 

But even if you have multiple people in such a capacity, there’s still room for a catastrophic failure if many of them decide to leave at the same time for whatever reason. We usually refer to the number of key technical people that would have to disappear for a project to go down with them as the “bus factor”.

A high bus factor, i.e. increasing the number of people with access to this level of technical expertise, is not only desirable but also can be the key to keeping the project running smoothly. This can be achieved by encouraging communication and cross-training, decreasing complexity, and, above all, writing comprehensive documentation. 

Five Essential Items that Good Technical Documentation Will Include

There are several “must-haves” of technical documentation for every software project. For smaller projects, they may be all that’s needed in terms of documentation; for larger and more complex projects, they’re the bare minimum starting point. If you’re missing any of the following, it may be difficult to operate in a day-to-day fashion, when bringing on new staff, and especially if there is unexpected turnover within the team. Make sure your bases are covered by documenting the following topics:

1. Source Control

Where the source code is located and how to access it, be it GitHub, GitLab, BitBucket, or elsewhere.

2. Hosting

Where the application is hosted and the account credentials. This should also include things like backups plus where the associated domains and SSL certificates are registered (if separate from the primary hosting account) and when they need to be renewed.

3. Deployment

Document the process to deploy a new release of your product. Cover how much of this is automated vs. manual, what steps and triggers are involved, how to debug a failed deployment, and how to roll back changes should the need arise.

4. Development Environment Setup

How to get a local copy of the application up and running on a developer’s computer. What needs to be installed, where to find things, and how to build, test, and access the app locally. Going through this process is a day-one task for new developers so having it documented will help save time whenever you grow your team.

5. Important Services and Credentials

For SaaS services like project management software, email services (such as MailChimp), analytics, etc. Make sure you’re cataloging these tools and how to access them.

When it comes to documenting accounts and passwords, I recommend using something like a Google Sheet with the following headings:

• Service Name
• Login URL
• Username (if applicable)
• Email
• Cost
• Notes

Make a habit of adding accounts to this sheet as they’re created. Having the cost listed (be it per user, monthly, or estimates based on usage) can also be helpful for calculating things like burn rate. For security reasons, passwords should be managed separately using a password manager; you may end up with some information duplicated between the two, but I believe maintaining both has value.

Technical Documentation to Speed Up Daily Work

Documenting these topics isn’t essential, but they add a lot of value, especially for larger and more complex projects. They can speed up day-to-day development operations, and cover some of the trickier policy and process aspects related to things like data handling. Consider covering the following:

Coding Style Guide

Document your standards for naming conventions, file structure and code organization, linter settings, and general style rules pertaining to things like braces and indentation. Taking the time to formalize this adds a lot of value and leads to developers producing clean and consistent code, which speeds up the code review process, resulting in fewer bounced-back tasks. Basing your guide on and referencing an existing standard like PHP’s PSR can make this type of documentation quick and painless to produce.

Development Workflow

Cover topics like what project management methodology is in use (SCRUM, Kanban, etc.), how it’s being implemented, and the meeting schedule. If you are interested in these methodologies, we have an article comparing Scrum vs Kanban to tackle their main differences for continuous deployment. What is the process for task assignment through to acceptance and deployment, and what steps need to be done at each phase along the way. Also, cover the DevOps side of things, and to what level Continuous Integration / Delivery / Deployment is being leveraged.

Data Flow and Regulatory Compliance

Depending on your space, location, and nature of your project, you may be subject to regulations like COPA, GDPR, HIPAA, or others. If so, it makes sense to document how you’re abiding by these standards and what internal rules and processes you have in place to ensure you’re doing so. In addition, it can be wise to document your general rules for the handling of data and sensitive information in your application, and what steps should be taken to keep that information secure in both development and production environments.

What to Consider for More Complex and Mature Projects

If you’ve documented everything we’ve discussed so far, you’re in good shape. Still, there’s room for improvement and added efficiencies through additional technical documentation. Mature and enterprise-level projects may want to cover the following topics, as they help to demystify complex logic and address business risk.

Supporting Diagrams

Creating visual documentation like ER Diagrams and flow charts to help clarify how your application works at a core level and how its different models relate. This can be especially useful in complex applications while also helping bring new developers up to speed quickly. 

Business Logic

If there are specific algorithms, files, or concepts that are hard to understand even with inline code comments, consider creating supplementary documentation for them.

Security Primer

Cover core concepts like SQL Injection, XSS, CSRF, and other common threats relevant to your project. Cite the OWASP Top Ten and tie in any information relating to mitigations and security processes that you may have in place. Also, include how you’re handling access control for infrastructure such as the use of whitelisting and things like SSH and cloud hosting keys.

Disaster Recovery Plan

Document how to handle service outages, hardware failures, data loss and corruption, security breaches, ransomware attacks, and any other potential scenarios you can think up. If you have a plan ready, it will be easier to respond should one of these events occur.

Additional Topics

If you’re having repeat conversations about how to do a certain development task, how a system or process works, or you’re unable to perform a certain business function because someone is absent, that’s a sign that you should write documentation about it.

Five Technical Documentation Tools for Software Development

What you’re using to write technical documentation is secondary to ensuring that you’re actually doing it. Still, there are many options out there, from simple to complex.

1. README.md

The standard go-to for information related to any git-based project. This can be a good starting point for basic developer documentation. If it becomes too large, you can break it out into several smaller markdown files.

2. GitHub Wikis

For more in-depth software developer documentation. Read more about them here.

3. Shared Office Suite

Either G Suite, or Office 365. Make use of the sharing functionality to build more robust documentation for either the developer or the operations side of your project. It may be helpful to have a primary index document that serves as a jumping-off point by linking to each related topic document.

4. Technology-specific Documentation Tools

Tools like Autodocs generate documentation from code or API documentation tools like Swagger.

5. Enterprise Tools

For large organizations, something like Atlassian’s Confluence may be helpful, especially if you have diverse content or a need for advanced permission controls and collaboration functionality.

Start with something that meets your needs and refine your process as you go. Break files out once they start getting too long or cover too many topics, and go from there.

How to Implement Technical Documentation in Your Development Process

The best way to make sure software documentation isn’t neglected is to make it part of your workflow. Track new accounts as they’re created, and add to the developer documentation as your system evolves. Part of the acceptance process for tasks should be asking two key questions:

1. Does this change warrant updating the documentation?
2. If so, has the documentation been updated?

Based on the answers, it may be necessary to update the docs before accepting the task. Enforcing this requirement stops the “I’ll do it later” mentality that leads to things sliding, as developers often want to complete tasks and then quickly move on to the next one. You may also want to have the software documentation quickly peer-reviewed (like a code review) by another person to make sure it’s clear and covers the bases before it’s accepted. 

If you haven’t been doing this and you’re looking to improve the documentation for an existing project, don’t get intimidated. Address one topic at a time based on importance, and work through things until you’re caught up. From there, make writing docs part of your daily workflow as discussed above.

Final Thoughts for Effective Documentation

Effective technical documentation is a valuable resource for projects. By centralizing information and defining processes, teams are able to be efficient in both their day-to-day operations and periodic tasks such as onboarding new developers. Failing to keep documentation up-to-date is a disservice to a project, and will lead to problems as it begins to scale.

There are a handful of essential topics to cover when writing docs. For clients/product owners, it is primary accounts and policies that are critical to the business. On the development side, it’s the process to get up and running, code style, and task workflow through to deployment. All this can be done without much effort by using freely available software documentation tools, and building it into your business process ensures that it gets done.

Every time you’re able to direct someone to the docs to answer a question or explain how to do something, that’s time saved. Going forward, aspire to make your technical documentation more robust by looking for opportunities where additional coverage would be helpful. Engage with your team throughout the process, get their input, and make documentation a part of your culture. From there, the path will be clear for you to build great things.