Flutterby™! : Software Gardener 2011-05-08 02:26:35.740256+00

Software Gardener

2011-05-08 02:26:35.740256+00 by Dan Lyke 8 comments

Chris Aitchison: You are NOT a Software Engineer!

You are not a Software Engineer. You do not build skyscrapers. You do not build bridges.

You grow gardens.

You are a Software Gardener.

This is particularly relevant because I am aware of a software project that's going to fail. It's a government funded one, where the agency involved has a very thorough engineering specification document and culture, and the project is totally unsuited to that methodology. What they need is two smart guys who can code something up, try it against a couple of potential customers, iterate fast. What they've got is half-way through two and a half million bucks, they're just starting to talk to users, and no code written.

You've gotta plant those seeds early.

From my dealings with control system we start out with a very abstract specification going into design the design phase moves the specification from an abstract to a buildable product. This is what is bid on to build but we know this is not what ultimately will be built. As design flaws are discovered in the build phase change orders are issued and the specification is again updated with the change orders. When the product is finally built we enter the commissioning phase and more design flaws are found, more change orders are issued and that process continues until we have a product ready for verification. This last phase before releasing the system to run production may or may not turn up more design flaws. If it doesn't as builts are issued along with final SOPs. If not, more change orders and the verification is run again. This ends the project but support usually ends up going back in and tweaking the design a little more with a work order, not a change order, after the line has been in production for about three months.

I've followed this same process in the chemical industry, food industry, auto industry, and numerous other industries I've done work in over the years. So are y'all calling me an engineer or a gardener? Maybe a Gardner?

Yeah, it's all largely metaphor anyway. Just seems that far too much of the software engineering culture treats the development portion as though it were building bridges, when in fact it's far more like designing the bridge

Perhaps people are just fussing over the definitions of 'engineering' then. Because now I'm even more confused. If the complaint is 'large systems are not well-served by overly detailed and overly prescriptive/rigid specifications' then... ok, sure. But plenty of good engineers know that.

Absolutely. Part of today's project is looking at vehicle control systems with fuzzy control systems and hard real-time constraints. That requires some serious discipline. However, far more software projects are involved in evolving the human interface than in having the provably optimal response in all of their behaviors.

And civil engineering has hard constraints too, but unlike software there's this hard implementation stage that occurs after the design. If you offered a civil engineer a tool that would allow them to implement and test their systems during the design phase, like the tools we have for software, they'd go nuts.

Urban planning is a hell of a lot like gardening, and in general the attempts to treat it like engineering have led to horrendous failures. Urban freeways, anyone?

Gah - too many thoughts for a blog comment. but the notion of ecosystems applies just as well to, say, urban planning and civil engineering (with multiple interacting parts, transportation, sewer, water, HVAC, building needs, residential, commercial, changes in population/users, and so on). So I'm not at all sure what the great contrast is between "gardening" in the sense of "oo... my software is an ecosystem" and what other domains of engineering have to deal with when designing, deploying, maintaining, other types of systems at scale.

The need for graceful degradation, fault tolerance, adaptation is not unique to software. (But even in software systems one can imagine cases where graceful degradation is NOT optimal and a hard failover (or failstop) will avert a bigger problem.)

I've been thinking about this more, and I'm coming to the conclusion that people have been treating software development like manufacturing more than engineering. The classic notion of "build design documents for everything", when a perfectly good and very formal language for documentation exists (ie: the implementation language), is inherently adopting the flaws of other engineering models.

This is what Agile and its ilk are trying to address: Code is a perfectly good specification system, regression testing and holistic systems testing are the step towards having the entire systems finite element analysis that mechanical engineers aspire to.

But I think the notion of software gardening is especially useful for systems at scale. Because in a big deployed system, with lots of interoperating servers, continuous deployment and independent access via third party applications, we're dealing with things that are far better described as an ecosystem than as a process.

And, similarly, the failure modes need to be managed in a softer way; attacks of a severity that would bring down a physical structure are, in the software world, common-place, you can't engineer to avoid all of them, but you can lay the framework so that when they do occur you can better adapt.

I'm looking at traffic management and engineering this morning, and I think there's probably a lot that us old school software developers who've had experience in both ops and systems implementation can bring to that discipline because we've recognized some of the flaws in the engineering process. Definitely stuff worth pursuing in this metaphor.

Fits with what some people (including me) call organic programming. Although sometimes mutant gourds over-run the garden and you have to till it under and start over.