No Silver Bullet Essence and Accidents of Software Engineering

¶ … Silver Bullet

During the 1970's, companies had difficulty delivering software within the constraints of schedule, budget, and quality (Food for Thought, 2005). The problem grew worse over time. Many projects undertaken in the 1980's and 1990's were complete disasters, failing to deliver anything, grossly exceeding budget and schedule deadlines, and delivering poor quality. Also, during the 1980's a "software crisis" occurred in which the spending on software maintenance exceeded spending on creating new software products. So, why can't software be mass produced in a way that is reliable and consistent just as manufactured goods are delivered today? There are many theories regarding lack of software productivity. Brooks (1987) holds that the fundamental nature of software prevents meaningful automation. Cox (1996), on the other hand, makes the interesting assertion that software development issues stem from market dynamics, namely the way software is bought and sold. Most recently, experts have turned their eyes to the organization itself, claiming a lack of IT governance as the cause of software project failures (Weill and Ross, 2004). This paper finds that all these theories and many more not discussed all help to shed light on barriers to software productivity.

Brooks (1987) holds that the problems in software development productivity are attributable primarily to conceptual components and, therefore, no efforts on the task components that are nothing more than the expression of the concepts can bring about large productivity gains. Brooks states, "I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation." Therefore, he asserts that there is no technology or management technique that will produce even one order-of-magnitude improvement in software productivity, reliability and simplicity.

Further, Brooks (1987) describes a software product as being fundamentally different from a manufactured good. Unlike a manufactured product, scaling up of a software element is not about repetition of the same elements into large sizes, it must involve an increase in the number of different elements that interact with each other in a nonlinear fashion. Because software must conform to other interfaces, this complexity is impossible to simplify. Also, manufactured things aren't expected to change after manufacture, but software must constantly evolve. And, software has no ready geometric representation as do other goods such as silicon chips having diagrams and computers having connectivity schematics.

On most attempts to improve software productivity, Brooks (1987) is fairly pessimistic about their overall impact of most, but does have a few good things to say about the potential of others. As marginal contributors Brooks cites:

High level languages: The most a high-level language can do is to furnish all the constructs that the programmer imagines in the abstract program.

Time sharing: As response time approaches zero, at some point it passes the human threshold of noticeability with no additional benefits.

Object-oriented programming: Significant gain can be made only if the unnecessary type-specification underbrush still in the programming language is itself nine-tenths of the work involved in design.

Artificial intelligence: The hard thing about building software is deciding what one wants to say, not saying it.

Expert systems: These require finding articulate, self-analytical experts who know why they do things, and developing efficient techniques for extracting what they know and distilling it into rule bases.

Automatic programming: It does not generalize to the wider world of the ordinary software system.

Graphical programming: It is difficult to extract any global overview

Program verification: It does not save labor

Workstations: Power and memory capacity not that important after a certain point.

Factors with more promise, according to Brooks, are mass markets for buying software, requirements refinement and prototyping and finding a way to mentor more great designers. Of requirements refinement and rapid prototyping, Brooks believes that this area has potential because it attacks the essence, not the accidents, of the software problem through incremental development. Brooks concludes by stating that great designer are vastly more productive than others, producing structures that are faster, smaller, simpler, and cleaner with less effort.

Cox (1996) believes that the software engineering community is too focused on development processes and tools to explain poor productivity. Instead, Cox argues that the problem rests in the way software is bought and sold. He uses the following sequence of logic to justify his opinion:

1. The reason that software is costly, of low quality, and difficult to construct is that we build it rather than assemble it from prebuilt components, the way that every other engineered product is constructed.

2. The reason we build rather than assemble is that there is not a robust market for buying and selling components.

3. The reason there is not a robust market for components is that there is no standard mechanism for pay-per-use of components.

Cox explains that intangible electronic goods such as software applications are quite distinct from tangible goods such as baskets and potatoes. Tangible goods are made of atoms; electronic goods are made of bits. The makers of tangible and intangible goods, asserts Cox, must expend the same capital, labor, and knowledge producing their products. However, intangible goods can be copied in nanoseconds and transported at the speed of light while tangle goods are hard to copy and transport. Because of this difference, the traditional pay-per-copy pricing mechanism has been applied only to tangible goods. But an information product's ease of duplication so thoroughly undercuts the traditional notion of pay-per-copy that the possibility of an abundant supply of pre-fabricated information-age goods is mostly eliminated. Cox thinks that a software productivity revolution will occur if there is a shift from a pay-to-own paradigm for software to a pay-per-use scheme. He refers to this as a superdistribution model that should treat ease-of-replication as an asset rather than a liability, actively encouraging free distribution of information-age goods via any distribution mechanism imaginable.

IT governance turns the attention of software productivity to problems within the organization. IT governance is defined as a systematic, sustained set of behaviors directed toward ensuring that IT continuously meets stakeholder expectations for performance, conformance, and relating responsibly (Pultorak, 2004). Performance is measure of IT's contribution to the creation of value, profit and wealth as well as efficiency and effectiveness. Conformance ensures compliance with financial, legal and regulatory requirements. Relating responsibility with key stakeholders is necessary because of the extent and immediacy of IT impact.

Organizations with mature IT governance generate up for forty percent greater returns on their IT investments than organizations with weaker partnerships (Weill and Ross, 2004). According to Weill and Ross, mature relationships between IT and business exist because of IT governance, specifying the decision right and accountability framework to encourage desirable behavior in the use of IT. Mature enterprises proactively seek value from IT by: