The longer the chain of responsibility the less likely there is anyone in the hierarchy who can actually accept it

Lack of communication

One of the key tenets of modern capitalism is division of labor. But is it a good thing for software development ?

Prior to the late 19th century a violin was produced from raw materials to completion by a single person, who himself may have been an expert violinist. He may have had members of his family work for him. Everyone involved in the process had deep connection to the craft and to the final product. Wikipedia article on Stradivarius states:

The name “Stradivarius” has become a superlative often associated with excellence; to be called “the Stradivari” of any field is to be deemed the finest there is. The fame of Stradivarius instruments is widespread, appearing in numerous works of fiction.

A modern violin on the other hand is assembled from many pieces, each one manufactured by someone else who may not even be musically inclined. At each point in the process, someone worked on this violin and contributed parts to it without knowing who the end user if the final product is going to be. Nobody in this chain of manufacturing can claim pride in the final product.

It also used to be that a stuffed bear is put together by a craftsman who would sew the toy together, stuff it with material, and decorate it on the outside. Today, a child can walk into a Build-a-Bear Workshop at their local mall, pick out the shell, and observe how a minimum wage worker sticks a tube into the shell and operates the machine that inflates the shell with stuffing.

A few years ago I needed my Swiss mechanical watch serviced. It used to be that a watchmaker had a shop downtown and serviced every single watch himself. To my horror, I realized that I had no choice but to drop my watch off at a mall kiosk, whose operator promptly put my precious posession in a UPS envelope and handed it off to a UPS truck driver.

Needless to say, when I got a call to come pick up my watch it turned out to be in a worse shape than it was when I dropped it off – in fact, the watch hands fell off the moment I turned the crown to adjust it. Eventually the watch repair company (Precision Time by the way) rectified the problem at their expense, but not without the hassle and stress for me.

In that entire process there was not a single person aside from me (the end user) who cared about the outcome. The kiosk owner had liability insurance, and so he couldn’t care less if my watch got damaged as he put it in the UPS envelope. UPS themselves don’t even know what’s in the package, and so they don’t care if it sits at the bottom of a pile in a dark wet corner of a truck. And finally, the person in a shop somewhere working on watches has to adhere to timelines and productivity goals that have nothing to do with the end result. There was not a single person in that entire chain who was able to make an informed decision about the entire process. It is no wonder that my watch came back damaged.

How is this relevant to software development, you might ask ? Well, in some companies the roles of business analyst, developer, tester, deployed and maintainer are distinct and separate. In the event of a production issue it takes orders of magnitude longer for developers to explain to maintainers what to do, than for them to do it themselves and fix the issue. All cognitive aspects of software engineering are split up and separated, keeping developers and maintainers as far removed from the knowledge as possible.

This hierarchical setup has historical roots. In 1974 Harry Braverman made the following observations in his Labor and Monopoly Capital: The Degradation of Work in the Twentieth Century (p. 227):

The upper level of the computer hierarchy is occupied by the systems analyst and the programmer. The systems analyst is the office equivalent of the industrial engineer, and it is his or her job to develop a comprehensive view of the processing of data in the office and to work out a machine system which will satisfy the processing requirements. The programmer converts this system into a set of instructions for the computer. In early computer installations, the programmer was generally a systems analyst as well, and combined the two functions of devising and writing the system. But with the encroachment of the division of labor, these functions were increasingly separated as it became clear that a great deal of the work of programming was routine and could be delegated to cheaper employees. Thus the designation of “programmer” has by this time become somewhat ambiguous, and can be applied to expert program analysts who grasp the rationale of the systems they work on, as well as to program coders who take as their materials the pre-digested instructions for the system or subsystem and simply translate them mechanically into specialized terminology. The training for this latter work occupies no more than a few months, and peak performance is realized within a one- to two-year period. In accordance with the logic of the capitalist division of labor, most programmers have been reduced to this level of work.

Below this level, computer work leaves the arena of specialized or technical skills and enters the realm of working-class occupations. The computer operator runs the computer in accordance with a set of rigid and specific instructions set down for each routine. The training and education required for this job may perhaps best be estimated from the pay scales, which in the case of a Class A operator are on about the level of the craftsman in the factory, and for Class C operators on about the level of the factory operative.

Of course, when Braverman published his book in the 1970s it was still very early in the history of computer industry. However, attempts to separate cognitive aspects of software craftsmanship from the work of a computer programmer did not stop there. The thinking was that by reducing the role of a programmer into a mere translator of very specific and very detailed requirements and design into machine language, and separating the programmers from any ability to make informed decisions, it was going to be possible to recruit a much cheaper workforce – or outsource it altogether.

Andrew Hunt and Thomas David warned us in Pragmatic Programmer: From Journeyman to Master (chapter 8 “Pragmatic Projects”):

Traditional team organization is based on the old-fashioned waterfall method of software construction. Individuals are assigned roles based on their job function. You’ll find business analysts, architects, designers, programmers, testers, documenters, and the like. There is an implicit hierarchy here—the closer to the user you’re allowed, the more senior you are.

Taking things to the extreme, some development cultures dictate strict divisions of responsibility; coders aren’t allowed to talk to testers, who in turn aren’t allowed to talk to the chief architect, and so on. Some organizations then compound the problem by having different subteams report through separate management chains.

It is a mistake to think that the activities of a project—analysis, design, coding, and testing—can happen in isolation. They can’t. These are different views of the same problem, and artificially separating them can cause a boatload of trouble. Programmers who are two or three levels removed from the actual users of their code are unlikely to be aware of the context in which their work is used. They will not be able to make informed decisions.

Just like a violin builder who can’t play music and doesn’t work with musicians (or is not a musician himself) can’t make a good violin, a programmer who doesn’t have a stake in the end product cannot build a good product. Environments and organizations with hierarchical bureaucracies discourage developers from proactively taking responsibility for the end product. The longer the chain of responsibility the less likely there is anyone in the hierarchy who can actually accept it.