Documentation a necessary pain

The days of the waterfall development model and the telephone-book sized documentation that went with it are, for the most part, gone. But documentation still has an important role to play, at least according to some.

Documentation hasn’t always proved to be useful in the past, said Stephen O’Grady, an analyst at Nashua, N.H.-based Illuminata. Documentation is a difficult, time-consuming task that developers don’t like doing. And in many cases the tremendous amount of time they spent on documentation proved to be wasted hours, either because the application had a short shelf life or because the people maintaining the application were those who built it.

But those incidences aside, companies can’t summarily dismiss the idea of documentation, O’Grady said. If a company has high staff turnover and a product with a long shelf life, then they had better rely on documentation.

“You could really be stuck if you’re left with a complex distributed application, but there’s no documentation attached to it,” he said.

Scott Simpson has seen first-hand the consequences of not producing documentation. The president of Ottawa-based bitHeads, a software development outsourcing company, is alarmed by the number of organizations that he encounters that have little or no documentation to accompany their software packages. He believes that while there is no longer a need for the hefty documents produced in the old waterfall days, the need for a document explaining the overall architecture of a system remains indispensable. Such documentation can explain why programmers made the choices they did and why they chose to go one route when another might seem more logical to someone who’s not aware of all the circumstances, Simpson said.

He’s constantly surprised at how many companies come to him with software products that have no accompanying documentation. That means that before additions to or upgrades of the product can be built, his team has to sit down with those who know how the product works and glean some of their knowledge.

“You have to talk to an individual to get that information out, and that’s a pretty time consuming process,” he said.

If those with the expertise are no longer with the company, then reverse engineering techniques have to sometimes be applied in order to create an architecture documentation, Simpson said.

Companies are likely pushing documentation aside in order to get products to market faster, he said. In the pre-Web days the time between product releases was much longer. Now companies tend to make new releases available much more frequently in order to stay competitive. Documentation gets low priority.

It’s the opportunistic developers that give documentation low priority said Thomas Murphy, a senior program director at Meta Group in Stamford, Conn. He said a study recently conducted by a large tool manufacturer found that there are three basic types of programmers: the opportunistic, pragmatic and expert/paranoid.

The opportunistic developer is very task oriented. They want to get the job done quickly, and they have a small overall scope. They rely on RAD tools and wizards and take visual approaches. Because their emphasis is on speed, they push documentation aside, Murphy said.

The second type of developer, pragmatic programmers, aren’t thinking about just the here and now. They want to build something that will last – solutions that are well architected and flexible. These types of programmers will take the time to do documentation. They want to create a foundation that others can build on in the future.

The expert/paranoid developers want to create efficient high-performance code that’s going to achieve technical praise. They want to create an elegant solution. They believe in documentation, but they feel that their code should be self-documenting – they want to write code that’s so clean, it’s self-explanatory, he said.

When creating documentation, it’s important to keep in mind the real audience for the documentation – maintenance developers, said Scott Ambler, president of software consulting firm Ronin-International in Newmarket, Ont.

It’s management types that will likely ask for the telephone-book sized documentation, but they’re not the ones who will actually use the documentation, he said. The real audience for the documentation is maintenance developers, he said.

“And if you go to talk to maintenance developers and ask them what they need, they’ll invariably tell you that what they really want is cleanly written code and really tight and concise overview documentation.”