Saturday, May 16, 2009

Will complexity overwhelm us? [ John Levy ]

Should we worry about software being too complex?

I’ve been teaching a course to elders about how computers work – both hardware and software. The challenge in such a course is to make the content both true and accessible while keeping the presentation interesting.

Hardware is in some ways easy to teach. Once a student grasps that all computers involve the interaction of a processor with memory (both for instructions and for data), there is only the matter of input/output left to deal with. Well, OK, there’s bootstrapping and operating systems and a lot of other stuff in there, but knowing about the processor and memory is the only crucial step to understanding how it runs.

But what about software? First I explain that nobody writes in machine code; very few write in assembly language; and even procedural languages like C and Pascal are being abandoned for object-oriented languages. So I have to get across, first, that people who create software are thinking abstractly in terms of algorithms and procedures. Then the next level of abstraction is objects and messages. And finally, the tools of the trade include editors, compilers, linkers, loaders and program (and object) libraries being used in Integrated Development Environments (IDEs).

And I haven’t even touched on testing and QA yet. How is a layperson going to come to appreciate the difficulty of producing a properly-functioning program? Yet everyone knows war stories of how failing software caused lots of damage. Is our knowledge of testing methods inadequate, or are we just being poorly managed?

Gerald Weinberg, in his recent book, Perfect Software and other illusions about testing (Dorset House, 2008), does a brilliant job of explaining to non-experts why testing doesn’t – and can’t – produce the eponymous perfect software. I highly recommend reading Chapter 14 of his book, which starts off with a study of failed and successful projects at IBM. All of the projects had “bad luck” strike during the projects. His conclusion is that success was a function of good management, not avoidance of bad luck.

Fifty years of Moore’s Law has given us computing power in our pockets that is greater than the room-filling machines of the 1960s. And we have whole networks of tiny microcontrollers inhabiting our homes and our cars. How are we going to deal with the prevalence of bugs in these devices?

No one says the code has to be perfect. But it has to be structured in a way that keeps it from failing catastrophically. And it has to be adequately tested even as it gets larger and more complex. Are we up to the challenge?

I firmly believe that Agile software development frameworks give us the best path to developing decent software. But they are not enough. We have to manage ourselves and our organizations in ways that permit healthy development behavior. This is what Agile Management is about: giving up certain counter-productive behaviors and establishing new management patterns that encourage constant improvement in the face of increasing complexity.

_________________________
John Levy consults on managing Agile development and is a frequent expert witness in computer & software patent cases. He has 30 years’ experience as a consultant and manager at Quantum, Apple, Tandem and DEC. His book on managing development, Get Out of the Way, is due out in 2009. More info is at http://johnlevyconsulting.com

No comments: