Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

[development] Let's get it started quick'n'dirty!

by blazar (Canon)
on Sep 09, 2005 at 08:28 UTC ( #490469=perlmeditation: print w/replies, xml ) Need Help??

This is not a meditation strictly about Perl, or computer programming. Or computers in general. It may easily be adapted to other areas of human activity in which one has to "produce" something, be it for his own entertainment or professional reasons...

In any case I think that due to the eclectic, multiparadigmatic nature of Perl, this discussion applies particularly well to it. Thus, consistently both with this fact and with our being at PM I will implicitly refer exclusively to it.

Well, suppose you have to write a program to perform a certain task. How do you begin? I generally try to plan in advance its general structure and the "size of the problem" - a bad defined quantity, admittedly. But this hardly ever works, and eventually what I do is a quick'n'dirt first "draft" that does only one thing, or a bunch of things, out of the ones it should.

Now you realize that you must provide for further flexibility, and that to add it you can't simply insert a few lines of code every here and there, but first or later you must refactor part of what you already wrote e.g. into subs.

Then you realize that your subs need to maintain a state or that there are plainly too many calling each other with no evident hyerarchical relationship wrt each other. So it all begins to be hard to read and maintain even for yourself, let alone others! And you need to further refactoring into suitable classes and objects.

But you're not done yet: your code after several rewrites still looks like a bunch of patches over the first draft, so that now that you have gained a much better insight on the nature of the problem, it all looks so bad to you... and you feel like you may rewrite it from scratch, avoiding all the gotchas of the first try.

Well, generally I do feel like that. And sometimes I do rewrite the program from scratch, taking into account the experience I gained in the meantime.

Whatever, I get the job done. I do waste quite a few "production cycles", granted. But I think I couldn't do it differently. What about you?

Otoh, whenever you're in a situation like that and you say "Well, at least let's get it started quick'n'dirty and we'll see how this thing will evolve!" someone will object "No, this thing must be thought accurately", "if we design it in advance we'll save precious time", etc.

If, if, if! Maybe it's just me, but if I insisted too much on the design phase, I suspect I would never get out of it. Trying one approach after the other provando e riprovando you can eventually understand which one is the "best" for your application and of course for you.

Just my 2 Eurocents...

  • Comment on [development] Let's get it started quick'n'dirty!

Replies are listed 'Best First'.
Re: [development] Let's get it started quick'n'dirty!
by ady (Deacon) on Sep 09, 2005 at 09:21 UTC
    The Extreme Progaramming methodology tries to strike a balance between creative exploration/intuition and structured planning/design.

    A good intro for Perl people is ExtremePerl, here's a quote of the core values of XP, expressed in Perlish tounge:

    Laziness means you work hard to look for the simplest solution and that you communicate efficiently. You don't want to misunderstand what someone said which might cause you to do more work than you have to.

    Impatience encourages you to do the simplest thing that could possibly work. You use the code to communicate your understanding of the problem to the customer, because you don't like sitting through long, boring meetings.

    Hubrisis courage born from the fear your code will be too complex for others to understand. Hubris makes you strive for positive feedback and to react quickly to negative feedback from your peers, the computer, and the customer.

    Allan

    ===========================================================
    As the eternal tranquility of Truth reveals itself to us, this very place is the Land of Lotuses
    -- Hakuin Ekaku Zenji
      Thank you. I had never heard of the ExtremePerl book and now that I know, at first sight it already seems to be a precious resource (for free)!
Re: [development] Let's get it started quick'n'dirty!
by tomazos (Deacon) on Sep 09, 2005 at 09:17 UTC
    I've got to admit, having worked through UML and the waterfall model and all the prim and proper software engineering practice - there is nothing like a working prototype hacked together in a weekend to get the design phase moving forward.

    -Andrew.


    Andrew Tomazos  |  andrew@tomazos.com  |  www.tomazos.com
      You could also try an iterative development model as an improvement over the basic waterfall model. Its basically a whole bunch of little waterfalls that incorporate their own requirements - design - build - test iterations before a client review that is fed into the next iteration.

      You start by identifying the high risk issues that are likely to cause problems and tackle these areas first. Any severe problems can be dealt with before too much money has been spent on the project. Reporting on successfully testing the difficult parts of the project will build confidence with the project manager and the client because that's what they worry about.

      I second tomazos' view that a prototype (even of the smoke and mirrors variety) is a great way to start the design process. Reviewing the current product (demos, models etc.) with the client at each review stage allows them to better visualise the deliverables and provide better requirements which are fed into the next iteration.

        I respectfully disagree. The theory is sound; the practice is not.

        Prototypes tend to become products without further design; no one wants to revisit the work done, and if the design isn't done right to begin with, it's usually not done right at all.

        Simple little programs that "evolve" tend to grow into sprawling monstrosities who's fundamental design assumptions are often wrong for the scale of complexity the end product is expected to run at.

        For example, on my desk I have a paper that presents a formal proof that certain aspects of concurrency have to be designed into a system in the first place; any attempt to add them as an afterthough is guaranteed to fail in certain important aspects.

        Sometimes, the "reduce the program and then re-expand it as needed" philosophy fails utterly. As the saying goes: "You can't jump a chasm in two short leaps." Sometimes, it's wise to take the fundamental achitectural considerations into account in the first place: if you know you're building in mountains, plan to build a few bridges. If you're working below sea level, don't assume tunneling is a good idea. And so forth. Don't abstract away the important implementation details, because they're what will make your product sink or swim in the real, non-abstract, non-ideal world.

        I don't like prototypes; they're a good indication that the client doesn't really know exactly what they want. That's a red flag for any project, of any sort, right there. Prototypes tend to drag troublesome people out of the woodwork with all sorts of mutually-contradictory (and self-contradictory) opinions on how you should design the system, who will then remain eternally bitter that you asked for their advice and didn't take it.

        Sometimes it's better to just work out what really needs doing, and do that. Here's an all too typical scenario:

        Boss: "Bill in marketing needs a TPS report sent in with blue headings on a daily basis ASAP! How long will it take you to complete?!?" Me: "I'll find out, and give you an estimate by morning."

        I go see Bill... Me: "Bill, why do you want blue headings on the TPS reports?" Bill: "Mary wants the entries will blue headings sent to Accounting for their summary report, and the entries with the black headings sent to Payroll" Me: "Thanks"

        I go see Payroll... Me: "Why do you need the TPS reports separated out?" Payroll: "We don't really use marketing's data; we thought Mary needed this change made! Why don't you ask her?"

        I go to see Mary... Me: "Hi, Mary. I'd like some more info on the TPS report Blue Heading's project" Mary: "Oh, well, I'm not a very technical person. You should talk to Fred... he's handling the matter for me"

        I go see Fred in Accounting.... Me: "Fred, how do you create the summary reports?" Fred: We take the info from marketing, give it to Joe, and he writes up the summary report." Me: "Is that all you do with them?" Fred: "Yes, that's it. Why?" Me: "No reason. Thanks, Fred!"

        I go see Joe... Me: "Joe, what do you do with the info from marketing for summary reports?" Joe: "Marketing data isn't part of the Summary Reports. I just need it to put the total volume from each department into the Department reports, and email it off to Ralph"

        I go to HR... Me: "Who's Ralph?" HR: "Ralph? He quit three years ago..."

        I go to the boss.

        Me: "Job's done." Boss: "That was fast!" Me: "It was easy, once I eliminated all the redundancies..."

Re: [development] Let's get it started quick'n'dirty!
by xdg (Monsignor) on Sep 09, 2005 at 11:29 UTC

    I suggest you might enjoy reading Agile and Iterative Development: A managers guide by Craig Larman. It's a survey of several agile methodologies, but what really makes the book a wonderful read is a great section up front that charts the history of "waterfall" development and its many failures, with tons of statistics and anecdotes.

    For example, waterfall development really got enshrined as a "best" practice by the US Department of Defense and was picked up by other companies and governments. But the US DOD has subsequently tossed that out and decided that development must be iterative and evolutionary, meaning, among other things, that the design cannot be fully known in advance -- but that switch hasn't been picked up as widely.

    I think it's entertaining and thought provoking even for non-managers and gives good ammunition for a more experimental, evolutionary approach to software development.

    -xdg

    Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re: [development] Let's get it started quick'n'dirty!
by g0n (Priest) on Sep 09, 2005 at 10:40 UTC
    That would depend.

    If I'm developing something for a customer, before I start coding I try to define a rigorous spec that a) the customer can understand as defining their requirement, and b) contains enough detail to ensure the requirement is exactly specified. This can be a challenge with completely non-technical customers.

    When coding less formally (writing support utilities when working under contract, writing my own modules etc) I tend to work out a fairly precise but very basic spec beforehand, write the first version; then immediately think of and start adding lots of new functionality.

    --------------------------------------------------------------

    $perlquestion=~s/Can I/How do I/g;

Re: [development] Let's get it started quick'n'dirty!
by wfsp (Abbot) on Sep 09, 2005 at 13:14 UTC
    I have gradually arrived at a quick and dirty "get something up and running" system.

    I break the task down, often something like:

    get_input(); process_input(); output();
    Each sub would have its own script which would read its input from disk, call the sub and then write its output to disk ready for the next sub. If the input/output is large scripts can be written to validate/produce reports.

    The same process maybe repeated with each task depending on complexity or if I'm stuck. In these cases there can be a string of scripts testing quite short subs.

    When there is reasonable confidence in the parts it is a case of fitting everything together.

    This way I can convince myself that I have some working code with only a small bit that doesn't work rather than a lot of code that doesn't work at all. :-)

    It's entirely conincidental that when you're _really_ stuck you have a shortish piece of code, some test input data and, perhaps, an example of the output you need. Very handy if you ask the monks for help!

    I have some arbitary rules of thumb:

    • Everything must fit on a screen. Subs, loops, if/else blocks.
    • Use subs to avoid deep indenting rolling across the screen.
    • No hard coded data - at all. Use a config file.
    • Use a look up table even if most of data is the same or undef.<.li>
    • Don't sweat over arbitary rules of thumb
    Later on down the line you can go back over it and reduce the number of subs.

    Update

    Oh yes, if there is an and an or and a not on the same line the code is just plain wrong. Rewrite it because you'll (me, at least) never ever get it right. :-)

Re: [development] Let's get it started quick'n'dirty!
by mikeraz (Friar) on Sep 09, 2005 at 16:10 UTC
    Whatever, I get the job done. I do waste quite a few "production cycles", granted. But I think I couldn't do it differently. What about you?

    Some people strongly agree with you on that point. "Plan to throw one away, you will anyway" is one of Brook's rules in the classic Mythical Man Month.

    You have independent verification of this truism.

    Be Appropriate && Follow Your Curiosity
Re: [development] Let's get it started quick'n'dirty!
by zentara (Archbishop) on Sep 09, 2005 at 11:42 UTC
    This seems to be related to the recent brilliance or... easy erasure?. I'm the jump in and start type. Most of the time, I don't even know what the coding problems are, until I make a prototype. So how can I sit down beforehand, and design what I need to do?

    Besides, prototypes are good ways of showing the boss what you've accomplished that day. :-)


    I'm not really a human, but I play one on earth. flash japh
Re: [development] Let's get it started quick'n'dirty!
by exussum0 (Vicar) on Sep 09, 2005 at 15:13 UTC
    Yes, and it's called something oh so simple, "measure twice cut once." What you are attempting to do, when you want an interface to be right, or code to be right, is to try something, see if it leads to the right answer. If it does, continue. It's not a perfect analogy, but bear with me.

    What I tend to do w/ code is to prototype something - especially if it's hard either 'cause it's large or it's complex. If it's not something I can fully churn out in a few hours, I do a bit of it, and walk away at a good stopping point. Why? 'cause that is a one time measurement. If I can come back, and see some sense to it, and not want to scrap it, that is my second measurement. More likely, though not completely 100% all the time true, I've done something right. It's like proof reading.

    It's the same thing you do w/ client facing interfaces. You propose it then prototype it to make sure that things are really what they want. In your case, you are your client, so you need to do a little, and come back fresh to make sure you are making sense. Taking breaks helps! :)

    ----
    Give me strength for today.. I will not talk it away.. Just for a moment..
    It will burn through the clouds.. and shine down on me.

Re: [development] Let's get it started quick'n'dirty!
by samizdat (Vicar) on Sep 09, 2005 at 13:09 UTC
    I am with you, blazar. While I can laugh at myself (as in brilliance or... easy erasure?) for my perceived sloppiness, I also define my systems as I go. I find that the recent 'helical evolution' model of psychological creativity (See Nonaka et al, The Knowledge Spiral) has some bearing. My thought process leads me to build my problem space model (source data + transformational program) organically. Data structures lead to effectors, which lead to refinement of data structures, and then communicators and state-managers become necessary, etc.

    That said, in a group development project it is tougher to maintain the organic growth because we programmers aren't often as good at speaking English (or whatever :) as we are Perl or C or Ruby. IMHO, for those of us who have to perform in such environments, <RANT> time spent learning a second language (using your mouth) and a third language (cultural body cues) and a fourth language (the conversation inside your own head) is time better spent than learning more computerese of any flavor.</RANT>

    Just my 0.02USD, your Euro value may vary. :D
Re: [development] Let's get it started quick'n'dirty!
by punch_card_don (Curate) on Sep 10, 2005 at 23:08 UTC
    It may easily be adapted to other areas of human activity in which one has to "produce" something

    In my lifetime I've had to plan, design, and produce everything from machines for manufacturing lines to residential hvac systems to decks to musical scores...oh, and computer programs too.

    And your observation about this question being applicable to many areas of creative endeavor is very true.

    In every case there is an optimal medium to achieve. At some point planning, calculating, goes beyond the accuracy required by the job and you are wasting time. At the same time, diving head first without a minimum of planning leaves you open to pitfalls.

    You can buy very complicated heating ventilation design programs. After you spend a week getting the dimensions and r-values of everything in your house, then another week inputting the information, it will tell you what diameter your heating ducts should be for ideal air flow. In the meantime, an experienced hvac guy measures the length of runs, and sketches the main trunks and branches. Counts a rough number of elbows needed. Then, with that reasonable planning done, goes to the hardware store to buy the ducts, the elbows, and a few boxes of screws and duct tape. He'll return what he has extra. He's done while you're still measuring, and the difference in system performance is insignificant.

    At the other extreme, the guy down the road set to building his new house with no planning (really). All went well until he discovered the new by-laws on septic systems and his half-finished house sat there for a year while he figured out how he would achieve compliance with property-line set-backs. A minimum of planning would have spared him this major pitfall.

    General rule - do a reasonable amount of planning. Of course, what is reasonable is very subjective, and that's where experience and skill comes into any profession. Experience and skill at estimating what is a reasonable amount of planning comes with, you guessed it, experience - that is, mistakes.

    Forget that fear of gravity,
    Get a little savagery in your life.

Re: [development] Let's get it started quick'n'dirty!
by artist (Parson) on Sep 09, 2005 at 15:49 UTC
A reply falls below the community's threshold of quality. You may see it by logging in.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://490469]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (6)
As of 2020-05-29 17:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    If programming languages were movie genres, Perl would be:















    Results (170 votes). Check out past polls.

    Notices?