The Status of Agile Programming Among Perlmonks?

[mrdurtal]

I am finding eyepopslikeamosquito's very detailed exploration of agile programming quite interesting. The six part series concluded just one year ago. Additionally, TStanley has reviewed a book on extreme programming by Chromatic for PM, Extreme Programming Pocket Guide.

However, I notice that a more recent book by Chromatic and James Shore, The Art of Agile Development (http://jamesshore.com/Agile-Book/) has not been reviewed. There are other books about extreme/agile programming and program management that similarly have not been reviewed. In particular, Robert Nagle's Extreme Perl (2004), which apparently had Chromatic as one of the technical reviewers, is left without comment (available at this site: http://www.extremeperl.org/bk/home).

Is this area of programming and management typically not a current or major concern for most Perlmonks? (I have only searched the titles of nodes--the thread, An Interesting Rebuttal of Agile, is now three years old.)

[tobyink]

In a way Agile is just a formalisation of the process of software development already commonly used in the free/open source software world. Release early, release often, and all that malarkey.

In that sense, a large number of CPAN modules (probably nearly all of them) follow something approximating agile development methodologies.

[educated_foo]

In that sense, a large number of CPAN modules (probably nearly all of them) follow something approximating agile development methodologies.

FWIW, I have a few CPAN releases and think "Agile" is worthless. One of the best Free projects, Emacs, is absolutely anti-Agile, and I wish the software I wrote were more like it.

[talexb]

To me, Agile is a 'development lifestyle' approach. It prefers *not* to have long design meetings or copious amounts of specifications done ahead of time, rather it prefers to get something lean done quickly, and to go from there. It prefers to have short iterations, in order to confirm (or deny) that the project's going in the right direction; it prefers not to have long chunks of time devoted to a big chunk of code, only to discover That's Not What We Want Anymore. It prefers to have developers chat informally and share information, rather than have weekly (or even daily!!!) meetings where people ramble on about where they're at.

Yes, I've been developing software for over 25 years -- so I've seen what works and what doesn't. The best project I ever worked on was my third job out of school; we went from starting work to having a working electromechanical prototype in 7.5 months. And here's how that happened:

• We were given a well-defined problem to solve;
• We were given sufficient funds to solve the problem; and
• We were left alone to get the job done.
It's really quite amazing how productive we were -- it's my guess it would have taken a larger team years to achieve as much. That's one of the high points in my career.

[mrdurtal]

I think you have identified the things that I find important in many processes that involve humans and that actually get important work accomplished.

It is interesting that you a) had something well specified to work on, b) were given the resources to work on the specification, and c) were given the freedom to do your craft.

Where did the leadership, or the leverage, for this come from? Did the client recognise that this was necessary? Was it the something negotiated by the software development team? Most places do not want to give up control over their resources.

[talexb]

Where did the leadership, or the leverage, for this come from?

I guess it was just the way the project was set up. The political machinations in later jobs amazed and disappointed me.

Did the client recognise that this was necessary?

We were doing development work in Canada in order to get a tax break on the investment. The product development work was being done for a company in California; after about 15 months on the job, we were all offered transfers down there.

Was it the something negotiated by the software development team?

Nope. And *I* was the software development team, in charge of both the 6809-based controller, and the PC-based user interface. Ridiculous, I know -- I was too ill-informed to know that what I was doing was impossible, so I just went ahead and did it anyway.

[sundialsvc4]

I doubt that you could go so far as to say that anything at all is “not a current or major concern for ‘most Perlmonks,’ ” because that is a sweeping generalization and unfit as a reference to this professional community.   By that I mean, “it simply doesn’t fit.”   Here you will find people who earn their daily bread by writing and maintaining computer programs ... and by managing sometimes-large teams of people who do so.   These are, by and large, deeply-seasoned pros, who might have been doing this for more than a quarter-century (and counting), and who regularly use dozens of languages in addition to Perl.   Opinions expressed here are often very sharp and strong, but they are also deep and considered.   And the subject-matter tends to be very focused upon Perl, as it should be.   I rarely see any book-reviews here, and I don’t look for them here.

Ever since the first drop of code went over the first Waterfall, many books have been written (and many videos and seminars have been sold) which promise to contain The Silver Bullet™ but I have yet to see it.   In my experience, they usually amount to some good ideas, taken too far and sold too extremely, but:   “good ideas, nonetheless.”   I no longer get excited about any of them.   But those are merely my personal opinions and prejudices, and nothing more.

[mrdurtal]

Thank you for your consideration.

The response about my generalisation is a fair cop. I could have been more specific. However, I wasn't able to get my head around addressing the whole readership. Your comments about the type of work that Perl programmers engage in and their maturity is exactly to the point.

Over the years, one of the things that I have appreciated about the Perl community is its culture. In a very real sense, Perl seems to attract people who are literate in the broadest sense of the word. They appreciate independence of thought and often think "sideways" about issues. However, being very much an "accidental programmer", I cannot say that I have always fully appreciated the community's humour or mind-set; but, that is a matter of my not always "understanding" the idiom.

Yes, I think that the open source dimension is important and comes before the explicitness of other movements. Or, I like to think that. However, not all consumers believe that.

In my own field, I have tried to use these same principles in terms of research and public health practice. I find that such principles work well with the various communities that we engage. However, funding bodies and bureaucracies have a hard time with the fact that participatory action research creates answers to problems right here and now and that it can incrementally move situation-based solutions into larger areas of concern.

Since this involves much less advanced "heavy planning" and many more iterative processes focused on communication and interaction between various stakeholders, they often do not know what to make of it. I personally doubt you can manage for certainty without imposing a heavy cost on the people you are working with; but, you can manage to move through uncertainty. We have been very successful in achieving end results that satisfy our immediate partners. Its the more abstracted or bureaucratic audience that remains unconvinced.

My hope has been that extreme and agile programming processes might provide some concrete examples of what can be done in another area of practice. If the processes develop out of the craft of experienced practitioners, I think they have more credibility. We talk a great deal about "transfer of knowledge" among the communities Aboriginal and Torres Strait Islander males. There is great respect for people who can get the job done.

For example, I have been applying what I learned as a successful technician leading other young men many years ago to a new situation. I get a hearing among the groups of men that I work with because it has worked so many other times in the past. The process is credible; it takes people seriously.

As I try to convince folks within tertiary institutions of the importance of teaching basic programming skills to university students who have to master the social networking phenomena as a part of their role in public health practice and research, I would like to make connections with alternate project management approaches as well.

I appreciate the comments. Again, thank you.

[mrdurtal]

Ahhhhh...youth, the time that we thought we could do anything! 8^)

But, it does point out the importance of the individual. This does not eliminate the importance of interdependence. Teamwork is important.

Yet, character surely counts for something in all of this.

[sundialsvc4]

A bit of my personal opinion on these methodologies (since you asked):

Software development is a creation process.   “On the first day” there is formless void, and “on the seventh day” you are taking a nap.   Every single one of us starts with an empty source file, every time.   Eventually, we wind up with a set of files that contain a correct piece of computer software.

“Okay... so, how long will that take?”

And here, alas, you have a Hobson’s Choice.   You can either:

• Lie about it.
• Say:   “I don’t know.”

The only way that you can truthfully say how long it will take to develop a piece of software is to know in advance how many unforeseen, vexing, absolutely show-stopper bugs you will encounter, and to know how long it will take you to solve each one.   But if you did know that ... if you could possibly know that ... then you would not have an obstacle at all:   you would merely have a milestone be a millionaire sitting on the deck of your two hundred fifty-foot yacht in the Bahamas, thumbing through your yacht-catalog and thinking about buying another one.  Or three.   Or thirty.

What you can reasonably do, however, is to have a workable process in place:   designing the entire thing as best you can before you begin, doing the work in a planned sequence, and testing as you go along.   You strive to develop some plan so that all of you are not groping around in the dark, and so that if somebody gets stuck everybody knows about it.   Yes, there are established ways to manage creation projects ... but none of them are sexy, glamorous, silver bullets.   Nor are they tremendously linked to the particular business of creating computer software.

A “methodology,” quite necessarily, is to some degree built to be saleable.   Books, seminars, consulting engagements.   The promise that you know how to make a silk purse.   The implication that no one on the company’s existing team knows these things:   that they are all, well, screwing up somehow, and that you just happen to know how.   That notion is unfortunately quite palatable.   But it is based on simplifications that, I find, don’t necessarily measure up to objective scrutiny, at least not to the extent that their zealots proclaim.

I do make it my business to study new methodologies most closely, because, if there is a technique out there with any potential impact on my company’s bottom (line), I want to know about it.   I usually find myself plucking a few pieces of tasty fruit from every new tree, but sifting through a fair amount of hyperbole to do it.   I remain hopeful, but disappointed.

JM2CW.   HTH.

[JavaFan]

Every single one of us starts with an empty source file, every time.

Really? I work with tens of other Perl devs, in a code base that was rooted more than a dozen years ago. Most of my projects involve modifying a bunch of non-empty files.

The only way that you can truthfully say how long it will take to develop a piece of software is to know in advance how many unforeseen, vexing, absolutely show-stopper bugs you will encounter, and to know how long it will take you to solve each one. But if you did know that ... if you could possibly know that ... then you would not have an obstacle at all: you would merely have a milestone be a millionaire sitting on the deck of your two hundred fifty-foot yacht in the Bahamas, thumbing through your yacht-catalog and thinking about buying another one. Or three. Or thirty.

I find this a lot of crap. First of all, if you have some experience, you ought to be able to quickly give a reasonable estimate on how long something is going to take. Someone ask you "how long will it take to do X", and you should be able to at least say "hours", "about a day", "a few days", "about a week", "weeks", "months", "years", and not be too far off. If you can't, given the years of experience you claim to have, you just aren't a good programmer.

Having said that, the IT world has a bad reputation of projects going way over budget, never getting finished, be full of bugs, and being scrapped before ever going live. I think that's a sign of immaturity.