Exploration of the Software Engineering as a Profession

In 1992 Ed Yourdon wrote Decline and Fall of the American Programmer followed by Rise and Resurrection of the American Programmer just four years later. The first book spelled doom and gloom for the American Programmers who were going to get replaced by cheaper counterparts in India, Russia, Philippines, etc. The second book revisited some of the predictions based on the changes that the software industry has undergone in the years between the books.

I have read both books as a freshman in college and both books were incredibly thought provoking. As a talented computer science student I did not feel seriously threatened by the predictions of the Decline and Fall, nor was I convinced by the conclusions from Rise and Resurrection. These books did spark controversy in the industry, but as all literature goes they were opinions rooted in facts of that time period. As I like to tell people who ask me questions any recommendation I make is based on facts known to me up to this moment and are not a guarantee of future results. Likewise, Decline and Fall and Rise and Resurrection had to be viewed in that prism.

Both books were based on popular management techniques of the time that emphasized separation of cognitive aspects of software development from programming. Indeed, popular software engineering project management techniques at the time were based on the experience from electrical and other engineering disciplines that put more weight on the design than on the implementation.

What I’d like to do is a modern exploration of the future of the software engineering in the United States as a craft and as a profession.

As it turned out, software engineering is not really an engineering discipline, and computer science is not really a science. In civil engineering, for example, a bridge that is safe and lasts for centuries takes months and years to design by highly qualified and well paid engineers and is then built to the specifications and design by individual craftsmen working in teams. A bridge is subject to forces beyond designers’ and engineers’ control. Once built, a bridge is extremely difficult to incrementally upgrade. That is obviously not the case with software.

Furthermore, unlike other engineering disciplines software has an incredible low cost of entry. While some engineering disciplines require years of education and apprenticeship, software engineering does not (but it could benefit from it). An architect would require a substantial capital investment to build a building. A software engineer, on the other hand, just needs food, a $1000 worth of equipment, and some spare time to build the next Twitter or Facebook.

Many of the predictions about outsourcing have not panned out either. Software engineers need to be domain area experts for example, something that is not easily accomplishable if you intend to have your software built by a generic pool of engineers overseas. Open-source is a great equalizer – whereas in the 1980s and 1990s one needed to hire an army of programmers to build boiler plate code, majority of the platform code is out there in the open today. Cloud platforms like AWS eliminate the need for an army of on-premise IT personnel – although they do create a temporary opening for outsourcing vendors to help customers migrate.

These are the topics that I’d like to explore over the next few months on this blog. Is there a future for software engineering as a profession in the United States ? What is the present state ? What are the forces at play ?