Flutterby™! : Software development methodologies 2012-05-22 15:08:38.650204+00

Software development methodologies

2012-05-22 15:08:38.650204+00 by Dan Lyke 6 comments

I've been a little frustrated with the progress of a project at work, and this morning I suddenly had an insight: Yes, all of those Agile/Pair/Waterfall/whatever "software engineering" methodologies are snake oil (IMHO), but they're also a chance to get a step towards any accountable process in place.

The real challenge is breaking the process down into discrete steps and saying "here's what needs to be done next", and if the participants in the process aren't used to that discipline, if they're just used to "yeah, I can have that done in 2 days" (where "that" is a functional item laid out in some document without clear inputs and outputs), everything is a matter of wallowing through a bog, and there's a lot of wasted down time because people are hanging out waiting for "that" to ship.

So, if one were tempted to try to bring some sort of process in to an existing team, what mechanisms would one use... hypothetically... What's the current in-vogue "methodology", especially among Perl geeks.

Well, the PSP people who trained me a few years back could do that, but it would take two weeks of training, so you wouldn't really be ahead of the game. I always think of PSP as "Agile for the BDSM crowd."

And really, I think that's going to be the problem with any outside organization. Yeah, they can probably help, but on this particular job, it'll be either a wash or a net loss because of the time to train everyone (which really is the time it takes for the team to accept stuff they can probably be taught in a day).

Yeah, this question is really about the possibility of finding a consulting organization that can come in and teach a group of programmers about breaking things down into steps and building social structures for accountability for those steps. And they're Perl geeks, which is why the Perl-ish-ness is important.

The long-term estimation isn't nearly as important in this case as figuring out a way to get people making and communicating concrete steps towards their goal, so that we can take a project that's going to take a month and a half and ship it in the 2-3 weeks it should have taken.

I'm not sure about the perl geeks, but one of the things I keep trying to do is related to your comment about breaking things down into discrete steps.

Agile methodologies are an attempt to, as far as I can tell, avoid estimating. That's fine if you can support it and just zero in on an answer for when you'll be done along the way, but there are still some tasks where knowing that it'll take N months to do feature X is needed before you start working.

For projects where knowing when you'll be done is important, I'm very firmly of the belief that if you can't estimate it accurately, you've failed to do one of two things. The first is decompose the problem into bite-size chunks; I need to break any task down into sub-tasks smaller than a week before I can estimate, but I can estimate sub-week tasks accurately, and if I spend the time to decompose the problem ahead of time, I can estimate very accurately. That's your second paragraph, and even agile folks need to be able to do that, though the backlog can add things as you go - it'll just drive your product management nuts.

The second requirement is to accurately account for non-task time. Agile methodologies figure out non-task time empirically. However you do it, if your estimates are continually coming in too low, there's probably something you have to do every day or every week that you're not accounting for.

I find that even experienced people who can decompose tasks accurately run into trouble with the second when they move to a new kind of task. They forget about the time needed to debug a UI because users click in crazy places. They don't think about the time lost trying to get the teleconference set up with the team in India. They brush off the time to adapt to new tools or language features. Or in the stuff I work on, they live in denial of the "Apple Tax" in which code you just spent the past year writing might become obsolete because it uses technology that Apple has been saying is deprecated for years, and now they're finally taking it away for realsies.

Oh, and (obviously) this is something I've dealt with a fair amount in the past, am dealing with right now, and will probably be dealing with in the future, so if there are any more specifics, please ask, here or ... other places. It's something I should probably try to get a bit more systematic about thinking about...

AFAIK, there isn't any one "process" that's any hipper than any other with the Perl set -- at least not the subset of it that I hang out with.

The key word there is "accountability". One thing I've found useful in the past is the notion of the short daily morning scrum -- a standing meeting, both as in "every day, no excuses" and "no chairs, you're standing up". Everybody stands in a circle, you go around and say, "Yesterday I did X, today I'm doing Y, and problem Z is slowing me down or screwing me up". That plus having a PM or a team lead who is willing to go up to somebody afterwards and have the "you haven't done shit for 3 days, what gives?" conversation will take you a long way. (That PM or tech lead better also be following up on any problem Z-type issues that come up, too.)

Once you get that in place and get it working, you can introduce the idea of an iteration or a sprint or whatever -- have a slightly longer meeting on Monday morning where you figure what is going to be accomplished that week, and then have a meeting later in the day on Friday where people demo or walk through what they've done *specifically with reference to what they planned back on Monday*.

That helps you introduce the second most important thing after accountability, and that's feedback. Everybody hates making estimates because everybody sort of sucks at it. The way to deal with that is not to try to avoid making estimates, and it's not to "Engineer Scott" the estimates. It's to make an estimate, fuck it up, learn from it, and repeat that cycle a whole bunch -- just like anything else you're not good at.

Of course, the rub with all of this is sticking with it, because it tends to upset apple carts. Lots of people *like* wallowing in the bog and not making much progress. It's comfortable, and they get to check Facebook a lot, and they have a sense of job security. Tred carefully. 8^)