Archive for November, 2012

Botanical Nature of Modern Software Engineering: Re-reading Frederick Brooks

November 7, 2012

They say that “The Mythical Man-Month” is a Bible of Software Engineering at least in a sense that “everybody quotes it, some people read it, and a few people go by it.” (see One of the fundamental things Brooks asserts in the book as a way to maintain balance between tight project budget and quality of a software system is to build it by using developers organized as a surgical team. In such a team a single person (“surgeon”) is responsible for the conceptual integrity of the whole system and performs the most critical work himself. All other members of a team help “the surgeon” to achieve his goals. The system scales up through hierarchical division of problems. Brooks even suggests the optimal size of a team.

The fallacy of modern commoditized software development is in the attempt to overcome this principle by removing the surgeon from the development team in hope that team members will find a way to build the system (or part of it) themselves. Integration testing will uncover deficiencies that will be resolved by natural collaboration between QA and development teams that produced the parts of the whole application.

This approach leads to the situations colorfully illustrated by Brooks in “The Mythical Man-Month” ranging from the missing project dead lines, going over the budget, suffering software quality or just blatantly failed projects. However, there is another aspect of such anti-engineering practice that creates interesting side effects and redefines (in a sense) the nature of modern software development.

Some of the systems implemented in such an anti-Brooks way still make it to the market. They suffer from the broken dead lines, missed budgets, missing functions, suffered quality but they still come to us in the form of desktop, embedded or mobile applications, ERP or content management systems, security packages or integrated solutions. They have been implemented without single concept in mind by international teams that do not exist anymore because individual members of these teams switched companies, got reassigned to other projects, retired or left the industry. The knowledge of such systems or their parts has been completely lost because it did not originated in a mind of a single individual or documented by some central authority but existed rather as a number of dispersed folk tales in an ether of a short lived development group.

In such situation any information about the system that is a little bit deeper than mainstream use case or involves more than one component does not exist. It is not that it is just hard to get this information digging through the pages of documentation or looking for the right person who may know the answer or even contacting a group of people who can deduct the information from the knowledge they have. This information does not exist because it never existed. It is buried in the lines of code created by way too many people who are in most cases unreachable anymore, and almost always do not work in the same groups that created this information. For all practical purposes almost any complex system that is in use today is as little understood as any creation of a nature like a bacteria or a flower.

In other words, understanding certain interaction for the given software system requires study of this system using method of natural science: an iterative approach starting with an observation, proceeding to a hypothesis to an experiment and analysis. If computer science of 80th and 90th was closer to mathematics because, after all, people artificially created everything in this science, computer science today is closer to biology or chemistry.

Traditional software development in one form or another goes from requirements to design to coding and then QA and maintenance. In the situation when environment is fundamentally unknown it becomes hard if not impossible to specify usable requirements to test against them later. The software turns to a research tool that provides feedback about the condition of the environment in the client’s location.  A bug turns to a piece of knowledge about the possible states of the environment.