Documentation in Agile Projects: Why? How? And What?

I am writing this blog post to provide a crisp treatment on a topic that comes up quite often when we talk about Agile implementation, adoption, transition and so on.  The first and foremost driver and a myth that causes this discussion is that Agile means ‘no documentation’.  Yes that is a myth.  Agile does not mean ‘no documentation’.  Let us begin with that understanding and move on to understand why we need documentation in software projects.    We need documentation in software because of the following reasons.

• Project stakeholders need it  (a business decision)
• To define a contract model (external interfaces and integration parameters)
• To enable communication with an external group (distributed teams, other project teams)
• For audit / compliance reasons
• To think something through – when you write or document something you think further and think deep. It helps before you get into discussions! You can save a whole lot of time by doing this!

There can be other reasons too. Here are some and these are not so good reasons. Beware!

• The requester wants it! – She has the authority and thinks that it is required.
• The requester mistakenly perceives documentation as project success
• The process says so!
• Someone wants to reassure someone else that everything is ok!
• Status Quo! We are a large organization and this is our way! 

In traditional SDLC, we used to do Big-Upfront-Documentation.  It is a costly affair.    When we know that 45% of deliver functionality is not going to be used, the corresponding cost invested in documentation goes unused too.  So, we need to be aware of the cost involved in big-upfront-documentation.



Our goal is to communicate. One of the mechanisms to enable this is to create documents.  However, documentation is the least effective mode of communication as shown below.



Let me step back.  Our goal is to communicate but that is not the only goal at a project level. There are several goals as listed here.

• Communicate Well - Documentation is not the primary issue – communication is!
• Ensure Project Success – Comprehensive documentation does not ensure project success!
• Create Consumable Documentation – Documentation need not be sophisticated!
• Create well-written, concise documentation
• Try to be flexible enough – Avoid trying to fit everything into a standardized template
• Keep the audience in mind – Usability and readability are critical.

In traditional SDLC, we have seen so many documents – High-level Plan, High-level Requirements and Architecture, Detailed Project Plan, Detailed Requirements Specifications, Detailed Design Specifications, User Manual, and so on.  And in most projects, these documents did not remain up-to-date as the software went to production and later in to maintenance phase.   Maintenance teams did not find them valuable and they started creating their own help documents.  The story went on. And this is true in most projects.

In Agile world, you ask follow Agile Principles. You know that document creation involves efforts and efforts cost money.   Why invest money in documents that go out of date?  Why not do crisp documentation and create an up-to-date set of documents that can be valuable to end users and maintainers at logical intervals?  Does it make sense?

So, Agile teams invest in documentation when they need to

• Communicate information during development
• Specify something (how-to test, what to test, etc.)
• Permanently record information (decisions and trade-offs)
• Confirm regulations

And they know that there are 3 types or categories of documentation.  These categories are not the same and documents belonging to these categories do not need the same type of treatment.

1. Work-in-progress documentation   (requirements, design, etc.)
2. Product documentation (API documentation, user manuals, etc.)
3. Handoff documentation (supports long-term viability of the project)

Above all, irrespective of the SDLC we follow, it is meaningful to ask the following questions and seek answers. That will help us know what to document and what not to.

• When? How? How much to invest in documentation? What is necessary at what timeframe?
• Do we measure the productivity of document creators? How?
• Do we measure the cost of maintaining documents? How?
• What do we do with stale or outdated documents?

So what documents do Agile teams create? It depends on factors such as complexity, strategic importance, user base across geographies, and criticality of projects. Here is a sample set.

• Executive Overview/Vision/Project Charter
• Project Overview
• Product Backlog, User Stories
• Test Cases, Test Scenarios with Test Data
• Solution Design, Contract Models(Interface Specifications)
• Design Decisions/Trade-offs
• User Manual
• Support Manual

This is what most Agile teams do.  And the documents they create

1. Maximize stakeholder ROI
2. Stakeholders know TCO of the document
3. Remain Lean and sufficient
4. Are meant for specific customer or audience
5. Are sufficiently accurate, consistent and detailed
6. Fulfill the purpose

Are you facing challenges in accomplishing these? Let us discuss.