Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Nobody Expects the Agile Imposition (Part VIII): Software Craftsmanship

by eyepopslikeamosquito (Chancellor)
on Jun 11, 2015 at 09:22 UTC ( #1129994=perlmeditation: print w/replies, xml ) Need Help??

Long long ago, at an Agile summit in Utah, a group of software industry veterans, frustrated by what they saw as overly heavyweight software processes, concocted the Manifesto for Agile Software Development:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan
... and so the "Agile Transformation era" began:

The Agile Alliance formed, conferences were held, companies went mad wanting a bit of this, everyone was having standup meetings, burndown charts, product backlogs, Agile coaches all over the place, more Agile coaches than developers ... and Post-its, Post-its, Post-its everywhere. The more Post-its you have the more agile you are.

We spent ten years talking about people, interactions, team building, eliminating waste ... and at some point agile took a detour ... process became more important than technical practices ... everyone went crazy at the Post-it party, three years later they woke up and realised that every two weeks we see the pile of s#?! getting bigger.

-- from Software Craftsmanship talk by Sandro Mancuso (2:00-5:00)

Uncle Bob proposed adding the assertion that we value "Craftsmanship over crap" to the manifesto

-- Uncle Bob, craftsmen and the Agile Manifesto

Around 2008, some of the original Agile Manifesto-istas, led by Robert C Martin (aka "Uncle Bob"), felt that Agile had gone off the rails, with too much emphasis on process rather than technical practices and code quality. This group felt that Agile projects, every two weeks, relentlessly, steadily, iteratively, were producing more and more crap code. Adding to technical debt. Slowing us down. Though Agile gave good feedback on many things, code quality was not one of them; it is not visible on burndown charts or the Scrum board.

How does this happen? Well, there seems to be an unwritten law of the Daily Standup that every morning you have to move at least one Post-it note. To comply with this "law", I sometimes pick up the Post-it note I was working on yesterday and move it a few millimeters to the right on the Scrum board while explaining why it is not really done yet. I always feel bad when I do this though; I would much rather proudly proclaim that it is done, so as to publicly show-off how productive I am.

Sadly, I've often noticed folks succumb to this psychological pressure to proclaim something done when it is not ... resulting in an every increasing pile of poo, as illustrated by Sandro's next (pair programming) anecdote:

"but what about these names, they don't make sense ... and there's lots of duplication ... and ..." yeah, well as it's almost done now let's just check it in and add that to the technical debt backlog. This was a brand new feature in an agile team! The guy (Sandro was pairing with) was writing brand new code already thinking to add stuff to the technical debt backlog!

-- from Software Craftsmanship talk by Sandro Mancuso (5:30-6:20)

How to fix this lamentable state of affairs? The new Software Craftsmanship group chose to add four new Software Craftsmanship principles to the original Agile manifesto, creating a Manifesto for Software Craftsmanship:

  • Not only working software but also well-crafted software. Code we can refactor confidently and without fear.
  • Not only responding to change but also steadily adding value. Not just bug-fixing, but improving code structure, relentlessly keeping it clean.
  • Not only individuals and interactions, but also a community of professionals. Mentoring, sharing, user groups, passion, professionalism.
  • Not only customer collaboration, but also productive partnerships. Not a "manager giving orders to a developer" relationship, rather a partnership of equals built on trust.

Well-crafted Software

God forbid that we have one day someone who is a Software Craftsmanship coach ... it is about lead by example, being a mentor ... not about beautiful code, just trying to provide value, not writing crap code for your customers. The lack of craftsmanship can be one of the main causes of failing projects.

-- from Software Craftsmanship talk by Sandro Mancuso (30:00-31:00)

Like Sandro, I disagree with the view that "Software craftsmen just care about beautiful code". Today, for example, it took me quite a while to figure out why some code that "should" fail was in fact reporting "success". I almost fell off my chair when I finally realized that it was silently catching and throwing away all exceptions! No comment explaining why it would do such a bizarre thing. To me, this sort of sloppiness shows a basic lack of respect for your colleagues; lack of care and comments wastes their time. It is unprofessional. Nothing to do with beautiful code.

Steadily Adding Value

After five years, the software is so s#?! that in the beginning it took two days to add a feature a feature of the same size now takes two months ... so they write a brand new one that is as s#?! as the previous one that will also be decommissioned in five years time.

-- from Software Craftsmanship talk by Sandro Mancuso (17:00-18:00)

Now the two teams are in a race. The tiger team must build a new system that does everything that the old system does. Not only that, they have to keep up with the changes that are continuously being made to the old system. Management will not replace the old system until the new system can do everything that the old system does. This race can go on for a very long time. I've seen it take 10 years. And by the time it's done, the original members of the tiger team are long gone, and the current members are demanding that the new system be redesigned because it's such a mess.

-- Robert C Martin in Clean Code (p.5)

The only way I can see to avoid this sort of fiasco is to to have the discipline to always keep the code clean in the first place, relentlessly refactoring as required every time you add a new feature.

Productive Partnerships

As soon as the button was pressed to mute NASA from our meeting, the managers said "we have to make a management decision", said Boisjoly.

The general manager of Thiokol turned to his three senior managers and asked what they wanted to do. Two agreed to go to a launch decision, one refused. So he (the general manager) turns to him and said "take off your engineering hat and put on your management hat" -- and that’s exactly what happened, said Boisjoly. He changed his hat and changed his vote, just thirty minutes after he was the one to give the recommendation not to launch. I didn’t agree with one single statement made on the recommendations given by the managers.

The teleconference resumed and NASA heard that Thiokol had changed their mind and gave a recommendation to launch. NASA did not ask why.

I went home, opened the door and didn’t say a word to my wife, added Boisjoly. She asked me what was wrong and I told her "oh nothing hunny, it was a great day, we just had a meeting to go launch tomorrow and kill the astronauts, but outside of that it was a great day".

-- from Remembering the mistakes of Challenger

As indicated by the appalling "management decision" to launch the Challenger Space Shuttle, the lack of a true partnership, built on trust, between technical folks and management can have truly tragic consequences.

Under pressure we cut corners ... No one wakes up in the morning and says "today I'm going to screw up, today I am going to write the worst code I can possibly write, I'm going to f#?! up with this company" ... I met some people who did that, but normal people don't do that ... everyone is trying to do a good job.

The business asks how long is it going to take? ... The pressure we keep talking about, the pressure we put on ourselves ... we think we don't have time, even when we have the power to go to the business and tell them how long something is going to take!

-- from Software Craftsmanship talk by Sandro Mancuso (7:00-9:00)

A Community of Professionals

Researchers (Bloom (1985), Bryan & Harter (1899), Hayes (1989), Simmon & Chase (1973)) have shown it takes about ten years to develop expertise in any of a wide variety of areas, including chess playing, music composition, telegraph operation, painting, piano playing, swimming, tennis, and research in neuropsychology and topology. The key is deliberative practice: not just doing it again and again, but challenging yourself with a task that is just beyond your current ability, trying it, analyzing your performance while and after doing it, and correcting any mistakes. Then repeat. And repeat again. There appear to be no real shortcuts: even Mozart, who was a musical prodigy at age 4, took 13 more years before he began to produce world-class music.

-- from Peter Norvig's Teach Yourself Programming in Ten Years

As indicated by Norvig above, it takes time and effort to become an expert, a true software craftsman. There appear to be no real shortcuts. With the current trend towards generalists in agile teams, finding the time to become a craftsman can be problematic as you find yourself constantly switching from one domain to another, from one language to another. This also affects code quality in that most code is being written by generalists, i.e. non-experts. This will be the topic of the next installment in this series.

Other Articles in This Series

External References

Perl Monks References

  • Comment on Nobody Expects the Agile Imposition (Part VIII): Software Craftsmanship

Replies are listed 'Best First'.
Re: Nobody Expects the Agile Imposition (Part VIII): Software Craftsmanship
by chacham (Prior) on Jun 11, 2015 at 11:54 UTC

    there seems to be an unwritten law of the Daily Standup that every morning you have to move at least one Post-it note.

    Agile is a tool, but some people make it a religion. It is somewhat disturbing to see how enthused they are about it. At the office, they send out bulk mail listing this or that. I asked to be removed from the list. No response.

    Agile is great for developing a UI, which ought to match what the user wants. Finding what the user wants is not easy, indeed, it is likely unknown to the user too. "I'll know it when i see it." Conversely, Agile is horrendous at developing a database. A decent database requires a data model which is grown around an idea and not patched together like some Frankenstinian concoction.

    What about code? Well, that's somewhere in the middle and is up to personal taste. If it works for you, use it. If it doesn't, don't. Unfortunately, the ardent followers of Agileism (Agilees?) refuse to accept that Agile is anything other than the embodiment of 42 itself.

    You, however, are much more sensible about it. Thank you for the nice writeup!

Re: Nobody Expects the Agile Imposition (Part VIII): Software Craftsmanship
by BrowserUk (Pope) on Jun 11, 2015 at 22:31 UTC

    My feelings on Agile and related practices are clear, so feel free to ignore this; but I do have a question that come out of reading your latest installment:

    On the ground, in practice, what is achieved by the Agile process -- stand ups, sticky notes et, al -- that isn't (not can't be) achieved without it?


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
    In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
    .

      On the ground, in practice, in my experience, there isn't anything that Agile can accomplish that can't be accomplished without Agile. But it can also be a very useful toolset. It can be a burden if you have to fight too hard against coworkers who have too strong of a tendency to practice Agile soaked in too much Kool-Aid (that is, dogmatic attitudes will usually be a small problem but can also be a big pain).

      It can be very useful to be able to get Agile adopted rather than work to get a whole laundry list of process changes adopted. And it can be particularly useful when you have to push back hard against Business/Product people who are aggressively impatient or demanding.

      And I think there is some truth to the impression of Agile proponents that the various aspects of Agile work especially well when combined. And the principles of Agile can be useful for justifying improvements. But it is also my experience that refusing to "stray" from Agile is very harmful.

      I think Agile makes a very good starting point for creating and maintaining a sane software development Process. You can also just study Agile and adopt parts of it or just use parts of it as inspiration to improve how your team works. But, in my experience, the teams that were based on Agile were closer to working well than the ones that weren't.

      - tye        

        Sounds like most 'sets of principles': read, understand, and apply judiciously; they can help you focus on what's important.

        But try to impose them; dogmatically or automatically; and they become the focus. To the detriment of all else.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
        In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked

      Divorce from bureaucracy is something that is made possible. An attempt to de-assembly-line an industry that cannot possibly work to its potential under that idiom; I know that you know this. In most corporate workplaces getting away from illogical or arbitrary and centralized process is pure fantasy and that very power structure is what is likely to scuttle any good practice and dress it the same chains as the same-old, same-old.

      Regarding your other reply: hearing about something obviously doesn’t make it good either. I’ve heard little but “Six Sigma” for 15 years for example.

        Eons ago, we had a presentation by Motorola on Six Sigma. When they were done, we ask them, "What does Motorola stand for?". We got blank stares. For us, it was easy and was also on our business cards, "The finest in family entertainment".

        James

        There's never enough time to do it right, but always enough time to do it over...

      If your current development cycle is based on

      1. gathering ALL requirements for the project first
      2. then complete architecture and design
      3. then write the code
      4. then write the tests
      5. then deploy the code to production
      then the Agile process can help you to achieve shorter completion cycles by focusing on shorter work cycles. The idea is to simply consume smaller chunks of work and produce potentially shippable product increments. This is very similar if not the same to release early release often.

      There are many critical dependencies to making Agile/Lean work. There are many blockers to preventing it from working. If you ever have the privilege to join a true and successful Agile/Lean team then you will see that it works and it works well. Once the "pipeline" is successfully built and you see your checked-in code automatically deployed to test environments, etc. then you realize it is just an uber form of Clean your room.

      I recommend this site http://agilemethodology.org/ to help understand the more involved portions, such as defining and implemented the necessary meetings that make this methodology so successful. The website has wonderful animated videos to illustrate the processes involved. Welcome to the future.

      Update: Well BrowserUk, you can do whatever you want. If you want to learn more you can. If you want to continue to justify why you reject change, go right ahead. Just realize that you are considered a minority now and i for one am just glad that someone with your lack of vision is not on my team. For example, SSADM is specific example of Waterfall development -- which is pretty much the opposite of Agile, but one would not get that impression if they listened only to you. Thankfully we have Google and Wikipedia (and people who actually read what you link to).

      jeffa

      L-LL-L--L-LL-L--L-LL-L--
      -R--R-RR-R--R-RR-R--R-RR
      B--B--B--B--B--B--B--B--
      H---H---H---H---H---H---
      (the triplet paradiddle with high-hat)
      
        Welcome to the future.

        That's what they said about SSADM back in the '80s. I rejected that back then; and I was right. Who's heard of it now?

        Different methodology; same goals. The reduction of the Art of Programming to a paint-by-numbers process with interchangeable machine parts. (Programmers)

        If you can reduce your programmer rolls to lots of Lil'boxes all made out of Ticky-Tacky then they can be managed as numbers in a spreadsheet.

        No stars; commoditised salaries; zero-hours contracts. An out-sourceable resource, purchased on a world-wide spot market at the cheapest price.

        Management has been falling for the hype for decades; and each new generation of programmers are all too ready to sign up with relish.

        It may be your tomorrow; but its not mine. I did my time. Good luck.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
        In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
        Update: Well BrowserUk, you can do whatever you want. If you want to learn more you can. If you want to continue to justify why you reject change, go right ahead. Just realize that you are considered a minority now and i for one am just glad that someone with your lack of vision is not on my team.

        Read, digest, think!

        It's your future. (Mine's already sorted.)


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
        In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked

        jeffa, Your ideas are intriguing to me and I wish to subscribe to your newsletter.

      I doubt there is anything done because it's "Agile" that isn't already being done by effective teams using other methodologies. I think Agile introduces effective processes to groups who haven't already stumbled upon them, or haven't learned them from experienced team members.

      Dum Spiro Spero
        Agile introduces effective processes to groups who haven't already stumbled upon them, or haven't learned them from experienced team members.

        I think that's a very fair summation. I'd go further and say that it does what pretty much all commercially sold methodologies do: it promises to short-circuit the learning process.

        But there are (at least) two problems with that:

        1. The calculator for kids syndrome: dependancy.

          Whilst providing preteens with calculators appears to give them a head start for a few years; once they get to college age; that head start often results in a stall. One which many of them will never recover from.

          Look at a high school math paper from the early 90s and if you look closely, you might notice something strange. (You probably have to come from the pre-calculator era to do so though.)

          The way the questions are constructed -- whether they are simple sums or the wordy 'problems' form of question -- they are intended to be solved by someone wielding a calculator.

          The differences are subtle; but for example; in the pre-calculator era, the numerical quantities in questions not directly aimed at testing basic arithmetic skills; were often carefully chosen so that the arithmetic required was relatively simple, if you were approaching the problem in the correct way.

          Ie. where division was required, denominators and enumerators would usually cancel to simple, single digit reductions; powers and roots would cancel to avoid the need to actual calculate either; signs would balance out; and so on.

          In this way, the test was about knowing or deriving the right formula and plugging the values in in the right places; not about re-testing the basic maths skills to calculate the final number(s).

          With the advent of calculators, with them doing the actual calculations, paper setters no longer had to look for these simplifications; and so the kids never learnt to do the cancellation steps.

          Role forward to college age (and the real world) and start dealing with probabilities or Taylor series expansions and the like; where instead of canceling out common factorial terms, or reducing by canceling power terms above and below; the kids try to calculate the values in full. With the consequence that intermediate terms exceed the range of their calculators; and they are dead in the water.

        2. The unrecognised problem syndrome.

          When you reduce the learning process to a recipe-style set of steps; or que-card process; it fails to equip the learner with the learning-from-your-mistakes experience.

          There is a popular TV reality program in the UK called MasterChef. Amateur chefs trying to produce professional (Michelin Star) standard dishes. At various stages the competitors are given the same recipes, ingredients and time to produce a recipe that the viewers have previously seen prepared by a professional chef. Then they are compared.

          Given the same inputs, there are people that assume that they should all be able to produce the same outputs. They don't. The differences, often very marked, come down to ethereal, almost immeasurable things, like palette, experience and practice. Even an eye for color, artistic flair and a steady hand under pressure.

        These are things that can't be short-circuited. They cannot be taught, they must be learnt. They cannot be instilled; they must be acquired.

        Applied rote, without room for: learning by your mistakes; or master/apprentice mentoring; and by presenting all work in the singular form of user-stories; you may short-circuit a few months or a couple of years off the early learning process to obtain acceptable productivity more quickly; but you also introduce a plateau that can never be surmounted. There is no room in the system for the gifted to rise above that plateau; and no way for the level of that plateau to rise.

        For the company, provided they can maintain a low-level of costs, which they can because there is no basis for individuals to become worth more than the standard rate; and replacing those who feel they want more becomes a simple process because the intake requirements are so low, this is good. Known costs * numbers = fixed overhead. Exactly what the accountant wants.

        For the programmer looking for a career, it is all bad, unless they are one of those that would never progress beyond the most basic level anyway.

        Present the 'products' of these methodologies -- programmers who expect all the work to be presented to them in the form of user-stories; and to work through each work item by a rote process of: if this do that; if this do that; -- with any task that does not comply with their expectations; and they are lost; because all the learning that was short-circuited -- how to gather requirements; how to prioritise those requirements; how to adjust and adapt their tools and working practices to the process of solving those requirements -- leaves them ill-equipped or unequipped to transition to the new reality.

        It creates 'crisis fodder'. Young, willing, enthusiastic, (usually male) programmers that follow rote, slog through, burn long, and hard, and out.

        In wartime, they are called grunts.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
        In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
Re: Nobody Expects the Agile Imposition (Part VIII): Software Craftsmanship
by sundialsvc4 (Abbot) on Jun 11, 2015 at 13:31 UTC

    Thank you for continuing to share these thorough articles with us.

    In my work, I find that communication is the second most-important aspect in any software team.   The first, is seniority.   Don’t give me someone who has only been writing computer programs, in any language at all, for less than ten years, please.   (This pretty-much excludes all L-1 and H-1B Visa holders ... which lack both age and season.)   Give me someone who knows how to understand a technical requirement and to turn that requirement into a set of necessary code-changes, in her head.   Someone who knows how to study an existing piece of production software and to create, not only a change to it, but an effective and thorough test for that change.   Someone who is routinely talking to and among his colleagues, working with them, sharing her thoughts and actively soliciting the thoughts of others.

    Post-It™ Notes don’t work, because they are small and lose their stickiness.   What you need is a permanent work-record:   something that tracks the original request and all discussions pertaining to it, as well as the git/SVN/CVS thread of changes made, and a record of all test runs.   Apple has such a system, which they call “Radar,” and as you walk in certain hallways you’ll see big posters which say, “Put It On The Radar.”

    To my experience, it is most important that everyone is able to know everything that everyone-else knows, including senior management, about every work-in-progress item, every proposed change, every scheduled change, everything.   A really good tracking system can (greatly help...) do that.   Sticky yellow notes cannot.   Yes, it is fine to sit down each morning with the first cups of coffee, herbal tea, or what-have-you ... ;-) ... but the vital communication is taking place all day.

Re: Nobody Expects the Agile Imposition (Part VIII): Software Craftsmanship
by sundialsvc4 (Abbot) on Jun 12, 2015 at 01:24 UTC

    I read a Kindle-only book several years ago, Managing the Mechanism, which made an extremely simple point that lays these simple-minded methodologies quite firmly in the dust:

    Computer software is a mechanism.   It is an automaton, a software machine which will do everything that it does unattended, automatically.   Yet it is a mechanism of fearsome complexity, composed of literally millions of parts, any of which can and do interact with each other.   The team that built it will put it on the field and push the Start button, leaving it to play the entire game ... alone.   There is nothing like unto it in all human experience.

    The book goes on to argue that “project-management principles” simply fail to apply to software projects, as they might for any other type of project, for precisely this reason.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://1129994]
Front-paged by Arunbear
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (2)
As of 2017-10-23 23:16 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My fridge is mostly full of:

















    Results (285 votes). Check out past polls.

    Notices?