Modern society relies on 'dangerously fragile' software

Are we ready for a civilization that's based on pervasive computing and increasingly complex "systems of systems," all dependent upon software written by unlicensed people calling themselves programmers? In this Computerworld interview, software guru Grady Booch, chief scientist at IBM Rational, talks about the enduring difficulties of software development and the complexity of today's systems:

Software [development] has been and will remain fundamentally hard.

Most of the interesting systems today are no longer just systems by themselves, but they tend to be systems of systems. It is the set of them working in harmony. We don’t have a lot of good processes or analysis tools to really understand how those things behave. Many systems look dangerously fragile. The bad news is they are fragile. This is another force that will lead us to the next era of how we build software systems.

He touched on similar themes in CIO magazine's "Forward Thinking" special report (1 October 2007):

Software-intensive systems will become increasingly invisible (as they weave into the interstitial spaces of society) and increasingly complex (for there is an essential, inescapable and growing complexity to such systems). Our civilization runs on software, and this reality will become even more true in the coming years. The continuing rise in computing performance and storage will contribute to civilization’s increasing dependency on software....

There will undoubtedly be some technical breakthroughs that we cannot anticipate: quantum computing, robotics, practical AI. [...] However, that being said, one thing we know is true: Developing software-intensive systems has been, is and will remain fundamentally hard.

This software dependency is worrisome given the sorry state of software quality. The Standish Group's latest report says only 35% of software projects can be called successful and 19% are failures. (In between, 46% are "challenged," meaning they had cost or time overruns or didn't fully meet user needs.) The figures are better than they were in 1994, but the picture still isn't good enough. Software is part of the modern infrastructure and often life-critical: 9-1-1 systems, utility controls, hospital systems, "intelligent highways," air-traffic control. I don't think we'd want 19% of the bridges we travel across to be failures -- wait, we have infrastructure problems there, too, but that's another story.

It seems like we need a concerted national effort to bolster our infrastructure -- including the software-intensive kind. Above, Booch says "we don’t have a lot of good processes or analysis tools to really understand how those [complex systems of systems] behave." And he talks about entering "the next era of how we build software systems." Seems like it's time to get started on that next era.

My proposal: The National Academy of Sciences should hold an expert symposium on "Our Fragile Software-Intensive Infrastructure," then issue a call-to-action report, with recommendations, at a congressional hearing. It would be a start.

------------
Related:
The Payoff from Software Quality (book excerpt)
Five reasons why software projects fail
Buggy Software (2005)
Watts Humphrey on Software Quality (2004)
Quality software means more secure software (2004)