Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Breaking The Rules

by Limbic~Region (Chancellor)
on May 30, 2006 at 18:08 UTC ( [id://552576]=perlmeditation: print w/replies, xml ) Need Help??

All,
There is an unwritten creed amongst us hackers with regards to the rules of programming. When it is doled out, it is usually in the form of admonishment. This tradition often leads to the uninitiated chanting cargo-culted mantras. There are of course published style guides and books devoted to best practices. It seems that everyone has an opinion on the matter. If we were each to publish a manifesto on the rules of programming, I seriously doubt any single item would be universally accepted.

It is not uncommon to hear arguments such as:

Sam: I am writing a widget to do X and am stuck with foo. Bob: Don't reinvent the wheel, use Y Sam: I know about Y, but I want to learn it for myself Sue: So why not learn how the author of Y did it Bob: Fine, but keep it in your sandbox (no CPAN or production) Sam: I want to share it because Y doesn't do Z Sue: Why not subclass or provide a patch for Y Tom: If you publish, pick a good name and explain when to use your wi +dget over Y
Sam didn't recieve the help he desired with foo because everyone felt Sam was breaking one of the rules. They had different opinions on the matter, but the outcome was the same. The great advice provided by Bob, Sue, and Tom may not have even registered with Sam. It was likely assumed that Sam possessed the knowledge and experience to understand why the counter-arguments were valid reasons for caution. Unfortunately, it is seldom the case that the uninitiated fit that assumption.

There is seemingly no end to the things we are or are not supposed to do:

  • Don't parse (HT|X)ML with regular expressions
  • Don't reinvent the wheel
  • Don't prematurely optimize
  • Always use strictures
  • Always use warnings
  • Always use revision control

For every single rule, there are exceptions. Wars are started over which exceptions are valid or not. So how is the uninitiated supposed to decide for themself? Here are my rules for breaking the rules:

  • Use extreme prudence
  • Experiment to gain personal experience

Prudence amounts to thinking through what might go wrong and taking necessary precautions. You don't have to rely on your just your own knowledge. You can search the web, read the manuals, ask around. You will have to utilize your situation specific context to know how, if at all, to apply this information.

It has been said that there is no substitute for personal experience. I encourage experimentation but warn that, without common sense and prudence, dragons lie ahead. You may still get burned or eaten alive, but if you live to battle another day - apply the knowledge you have gained. That's why they call it experience.

In summary, don't blindly follow the rules or be afraid to break them. Think for yourself and make sound judgements.

Update: It is ultimately your responsibility to understand why the rules, exceptions, and advice exists. Ignoring advice from others is not in the spirit of this meditation and you must realize the ramifications of your experimentation on others. If you decide to reinvent a wheel and use it in production without bothering to learn from the existing wheel or ensure your code passes the applicable tests from that test suite you are not using common sense, prudence, or sound judgement.

Cheers - L~R

Replies are listed 'Best First'.
Re: Breaking The Rules
by GrandFather (Saint) on May 30, 2006 at 19:55 UTC

    Many of the so called rules are actually guides set out to make your life longer and happier. Sure, you can parse HTML, CSV, ... whatever with regexes. You can write your own regex engine if you want. You'll learn a great deal about regexen, but with respect to your primary goal of parsing some HTML, you are liable to spend a lot of time, encounter a lot of unhappieness and possibly not get the job done.

    You don't have to use strictures, but they cost almost nothing and save bulk time. "use strictures" is a guide, but it should have the status of a rule. There are very few times in modern Perl where strictures are "bad", but almost every line of code written benefits from having them turned on.

    All the rules (or guides) are of greatest benefit to those who don't understand them or don't understand why they should be followed. So, if you don't understand a rule, follow it!

    In summary, don't blindly follow the rules. Follow the rules because we say so, but ask us why the rules are as they are. Understanding why is as important as following the rules, but follow the rules until you know deep in your waters why the rule is there.


    DWIM is Perl's answer to Gödel

      I strongly agree with this excellent summary by GrandFather. There is a basic problem with the OP's summary statement, and the update doesn't really correct it:

      In summary, don't blindly follow the rules or be afraid to break them. Think for yourself and make sound judgements.

      The problem is that, without noticeable exception in my experience, any 'hacker' who is tempted to 'break the rules' fails to possess the judgment necessary to evaluate the extent of his or her own wisdom (or lack thereof). Programmers are an arrogant breed, and we are often guilty of overestimating our own intelligence or knowledge base. Saying it in another way, while we may have a pretty good idea of what we know, we are rarely in a position to know the extent of what we do not know. Rules or guidelines are the safety nets of the developing world, which help us avoid pitfalls that we might not even see, or are perhaps dimly aware of. Personally, I suspect that the so-called legitimate reasons for breaking the 'rules' fall into two categories:

      1. They illuminate an ill-conceived 'rule' or a guideline that is insufficiently precise
      2. They illustrate the triumph of the developer's ego over his understanding

      I don't like rules any more than the next guy, and sometimes it irritates me when people spout a generalized version of a rule that doesn't apply to my specific situation. But nearly every major disaster that has befallen me in my programming career has been the result of ignoring one of these best practices.


      No good deed goes unpunished. -- (attributed to) Oscar Wilde
Re: Breaking The Rules
by BrowserUk (Patriarch) on May 30, 2006 at 21:58 UTC

    Here's a few home truths about rules.

    Not all 'rules' were invented by people who knew enough to be defining rules.

    Not all advocates of 'rules' know why they are advocating the rules they advocate.

    Many 'rules' have very limited application; and not all adherents understand the limitations.

    People learn and grow best by making the own mistakes.

    If you always follow the 'rules', you never learn anything.

    Give me a programmer who goes out on a limb occasionally and learns from the experience, and I'll show you someone who will be making the rules a few years from now.

    Those who always seek a CPAN solution, and never look at the source of the solutions they choose, and never evaluate (perform their own benchmarks, tests, or mindwork exercises on those CPAN implementations), are the same people that will always be screaming for yet bigger, faster, more powerful hardware.

    Not everything on CPAN is well written.

    Not every task requires the complexity of the "perfect solution".

    Not every task is worthy of the expense of the "perfect solution".

    Databases are not magic.

    Primarily, A programmer never gains good experience; never improves; never evolves; if they always take the easy route; always use someone elses code; always follow the rules.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      BrowserUK, I want to "quote two things for truth:"

      • Not everything on CPAN is well written.
      • Not every task is worthy of the expense of the "perfect solution".

      Now if only I could follow those same rules with which I agree so strongly, I'd be getting somewhere!

      Thanks to all the experienced guys on here. I'm gradually learning who you guys are.

      _________________________________________________________________________________

      I like computer programming because it's like Legos for the mind.

Re: Breaking The Rules
by xdg (Monsignor) on May 30, 2006 at 20:32 UTC

    Interesting meditation. It reminds me of three of the points from How do you master Perl?:

    • You need a safety net

    • Be bold

    • Reinvent the wheel

    Reinterpreted to this context:

    • Know when and where you can break rules safely

    • Don't be afraid to break rules

    • Rediscover the basis for rules for yourself

    -xdg

    Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re: Breaking The Rules
by jdporter (Paladin) on May 30, 2006 at 19:22 UTC
    It was likely assumed that Sam possessed the knowledge and experience to understand why the counter-arguments were valid reasons for caution. Unfortunately, it is seldom the case that the uninitiated fit that assumption.
    ... make sound judgements. . . . It is ultimately your responsibility to understand why the rules and their exceptions exist.

    Those are contradictory.

    Your advice has a chicken-and-egg problem wrt wisdom.

    We're building the house of the future together.
      jdporter,
      Life is full of circular dependecies ;-)

      In this particular case, there is a certain amount that can be understood from asking questions. Sam's responsibility should have been to ask follow up questions to each piece of advice. Sam should have weighed those answers with his problem specific objective. This isn't complete understanding but it is to the extent possible for Sam without real personal experience.

      If Sam decides to move forward in spite of everything he has learned, he needs to keep that advice in perspective as he undertakes the project. This is infinite recursion as that knowledge is applied to the next situation.

      Cheers - L~R

Re: Breaking The Rules
by BrowserUk (Patriarch) on May 30, 2006 at 18:36 UTC

    Limbic~Region++ As the monk quips occasionally go ahead... be a heretic.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Breaking The Rules
by chromatic (Archbishop) on May 30, 2006 at 18:31 UTC
    In summary, don't blindly follow the rules or be afraid to break them.

    Unless you're writing an e-mail address parser or a form data parser or a network server or dealing with external, tainted data or plan to use your program more than once or have to deal with a well-established file format, or....

      chromatic,
      The rest of that two line paragraph you omitted reads: Think for yourself and make sound judgements.

      Let me make it clear that I am not suggesting these rules don't exist for good reason or that following them is not a good idea. My point was that the exceptions are not universal and that careful experimentation results in experience to know what to follow and when. Being careful requires the coupling of common sense and prudence.

      Unless you're writing an e-mail address parser...

      In my meditation, I state that while looking for the things that could go wrong (prudence), you will need to utilize your situation specific context to know how, if at all, to apply this information. So what are my existing wheels for parsing email addresses:

      All but the last deal with RFC-822 and/or RFC-2822. These would be useless if the purpose I was parsing was to find addresses attempting to expliot an open-relay. This is because addressesses such as foo@bar.com%blah.net are valid by the RFC but result in the vulnerable MTA responsible for the blah.net MX record in stripping off the %blah.net and redirecting the message for foo@bar.com.

      Ok, so I use Data::Validate::Email which supports RFC 822 addresses but has "real-world" flavor as well. This still we do me no good if my objective is to find addresses that fit a specific pattern. It is a case of the general purpose tool not working for the specific problem space.

      I have updated the original meditation with a closing paragraph and provided a reply that I believe helps explain my position.

      Cheers - L~R

Re: Breaking The Rules
by dragonchild (Archbishop) on May 31, 2006 at 01:21 UTC
    Or, more succinctly:

    CGI.pm doesn't use strictures because Lincoln understands why strictures should be used. Until you know what Lincoln knows, you probably should use strictures as a matter of faith.


    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

      That's pretty generous. I think CGI doesn't use strictures because it's (working and tested but seriously) crufty code intended to replace worse code, written long before the invention of several good Perl programming practices, and never really refactored into better shape.

      A lot of core modules are scary. I just fixed AutoLoader so as not to be halfway broken, for example.

Re: Breaking The Rules
by mikasue (Friar) on May 30, 2006 at 18:52 UTC
    Limbic~Region,

    This is very encouraging to me as a beginner of Perl. I was focusing on not breaking the rules of Perl instead of just using Perl to do what I have to do. I know about "there's more than one way to do it" but I still didn't want to do it the wrong way.

    Thanks for writing this. I feel I have permission to use Perl anyway I see fit now :-)

      mikasue,
      Thanks for writing this. I feel I have permission to use Perl anyway I see fit now :-)

      Either I have failed to properly express my intent of this meditation or you have failed to grasp the spirit of it. This isn't a license to ignore what you don't like and do what you want. Let me enumerate the main themes again.

      • 1. First understand why the rule, the exceptions, and the advice exist
      • 2. Review that in context with your problem specific objective
      • 3. Using prudence, explore all the things that could go wrong from continuing to pursue this approach
      • 4. Safe guard yourself against all those known pitfalls
      • 5. Use common sense and learn from your mistakes
      • 6. Remember that your actions may impact more than yourself

      Cheers - L~R

        I will definately follow these 6 rules :-) I want to learn Perl the right way. But I don't want my mistakes to hender me from learning it. This meditation says to me it's ok if you break the rules as long as you learn why. Have I got it now?

      Please be careful how you take the advice from the OP -- it is not an excuse to simply do it the wrong way, but rather an excuse to do it the wrong way for yourself so you can better learn how to do it the right way for others.

      jeffa

      L-LL-L--L-LL-L--L-LL-L--
      -R--R-RR-R--R-RR-R--R-RR
      B--B--B--B--B--B--B--B--
      H---H---H---H---H---H---
      (the triplet paradiddle with high-hat)
      
        I understand! I wasn't saying I was going to break any rules. I was saying that I wasn't going to stop using Perl to do some programming task now because I'm scared of breaking the rules. As a beginner i'm going to use Perl and if I do it wrong -- learn from it. This mediation help me understand that I shouldn't be soo concerned about doing it wrong that I don't use the language to do it.
Re: Breaking The Rules
by Unanimous Monk (Sexton) on May 30, 2006 at 22:17 UTC
    This may be slightly off topic, but in the spirit of investigating the "Always use warnings" mantra, is there really any benifit to writing
    use warnings; my $var=1; print $var if defined $var;
    over
    use warnings; my $var=1; print $var unless $var eq undef;
    other than to avoid the warnings? I'm still reluctant to "use warnings", simply because it seems to nitpick my syntax more often than provide a legitimate warning.

      I would write:

      use warnings; my $var=1; print $var;

      Why the test? If $var is undef something much more serious is going on and I want to know about it.

      Yes, I know this is a trivial example and you imply that there may be legitimate situations in which $var is undef. However I'm suggesting that if $var is undef when you are about to print its contents, then there is a flaw in the code. It may be as trivial as not having provided a default value, but providing the default value is important - it says to the reader that you have thought about the special case of the variable contents haven't been changed.

      BTW, your second version is needlessly convoluted and will generate a warning (two if $var is undefined). KISS


      DWIM is Perl's answer to Gödel
        However I'm suggesting that if $var is undef when you are about to print its contents, then there is a flaw in the code.

        Its seldom a 'flaw in the code' if you need to check whether something has been defined. It could be a flaw from, say, user supplied input and your code needs to differentiate between data that was missing entirely (undef) and an empty string or 0.

        while (<FILE1>) { $var = GetVar($_); }; if ($var eq undef) { die "File provided doesn't include var!\n"; }
        You can't rely on warnings to tell you var didn't exist in the file, and you have no control over whether var gets defined.

        Also, there are benefits to "define when needed" if you have a complex set of processing which may or may not require a particular variable in numerous places and its impossible to predict which loop is going to need the variable first, if at all. Rather than needlessly calculating it before starting, just check for undef wherever you use it.

        There are countless legitimate reasons why you'd want to check whether a variable has been defined, and its not a rare occurrence.


        BTW, your second version is needlessly convoluted and will generate a warning (two if $var is undefined). KISS

        That's the point of my example; the fact that it throws a warning (or two), and its not needlessly convoluted (although the opinion of convolutedness apparently varies). In fact, it provides more symmetry if your doing something like

        if ($var eq undef) { ... } elsif ($var eq '') { ... } elsif ($var eq 'foo') { ... };
      Your two ways of testing undef don't actually do the same thing. if defined $var will execute your statement if $var is defined. unless $var eq undef will execute your statement if $var is undef, the empty string, or something else that will stringify to the empty string. Try the following code snippet to see for yourself:
      my $var = undef; print "Undefined!\n" if $var eq undef; $var = ''; print "Undefined again!\n" if $var eq undef;
      This happens because the "eq" operator stringifies both of its arguments, and undef stringifies to the empty string. The warning doesn't exist just to critique your syntax... it's warning you of a danger that your code might not be doing exactly what you think it is.
        Hmmm, your right. That’s not how I thought it was working, but it makes sense that it works that way.

        Learn something new everyday! (I apologies if this is what GrandFather was trying to point out, and I was too obtuse to realize it) :)

      The thing about use warnings; is that it helps you to avoid subtle, sneaky errors that you would otherwise have a very difficult time tracking down. I'll gladly put up with a few spurious warnings, versus spending many hours of head-banging debug time trying to find where I slipped up. For me, use warnings; has paid for itself many times over.

      As is so often the case, YMMV. I've never encountered anything like the situation you mention. Perhaps I'm not sufficiently advanced; perhaps it's just coding styles. If I ran into problems more often, I might agree with you and stop using warnings. To date, though, this nit-picky little pragma has saved me much aggravation and gnashing of teeth, muttering evil phrases, and all that fun stuff we're so good at.

      is there really any benifit to writing...(snip)...other than to avoid the warnings?

      Yes, the former is more clear in its intent, the latter is a longer and more obfuscated way of writing print $var unless $var eq '' (update: and even that may as well be just print $var ...printing an empty string shouldn't make a difference anyway...though I assume this is just a simple example and you could have, e.g., print "blah blah $foo blah" unless defined $foo)

Re: Breaking The Rules
by spiritway (Vicar) on Jun 02, 2006 at 04:06 UTC

    You've raised a very interesting point - a couple, actually. What rules do we follow, and (probably as important) Why?

    I like to think of rules (and following them) as one stage in growth in a language. When I'm brand-new to a language, I tend to be scrupulous about following the rules, even though I often have no clue why a rule is in place, what it's supposed to cause/prevent, what might happen if I didn't follow it, etc. As I learn more, I am in a better place to evaluate which rules, if any, I'll follow. It's something like growing up in real life. Parents can't always explain the reason for a rule - so it's, "Because I told you so." Later on we figure out exactly why they wouldn't let us stick our fingers into the light sockets - often by breaking the rule. Once.

    In most other languages, I can go ahead and shoot myself in the foot, no problem. My foot; my choice. In Perl, there is a unique factor that makes me think twice. Perl has CPAN. Suddenly, it's not just my foot we're talking about, but everyone else's who might use a module I write. While I'm still free to blow away my various appendages, there is a strong expectation that I not get overly creative when writing a module that will be shared. This expectation isn't enforced; I can upload crap (as many have done) and CPAN will accept it. But doing so will not endear me to the Perl community. Before I'd knowingly break a rule, I'd run it by the fine Monks here to find out what sort of problems it might cause; or whether there's a way to get it done without breaking that rule.

    On my own, of course, all bets are off, and I'll do whatever I feel like... as, I suspect, most programmers do. Programmers do not appear to be the kind of folks who follow rules just for the sake of following rules. Seems to me they're always saying things like, "Gee, I wonder what would happen if..."

Re: Breaking The Rules
by fletcher_the_dog (Friar) on May 31, 2006 at 18:04 UTC
    My rule of thumb is never break a rule unless you understand why that rule exists. "Understanding" why a rule exists is not saying "It exists because someones an idiot and is just trying to control me". My second rule of thumb is break rules to see why they exist and to see if the arguments are legitimate. For example "Don't parse (HT|X)ML with regular expressions. Try it to understand what it limitations are. Then you can decide whether for you particular needs those are limitations you care about.
Re: Breaking The Rules
by Anonymous Monk on May 31, 2006 at 21:15 UTC
    A rule is usually a summary of a lot of experience; it explains the common cases, not the edge cases.

    For example, "don't parse HTML with regexps" is a good idea in general: you can't do a good job of parsing arbitrary HTML with a regular expression.

    On the other hand, if you wrote all the HTML yourself, and know exactly what applying a given regexp will do to your HTML, then go ahead and use the regexp if it does exactly what you want.

    If you didn't write the HTML, on the other hand, you probably can't reliably predict what a given regexp will do to the meaning of a page. You might need to carefully parse the HTML, determine carefully what sections are which, and only then make your substitution. This is the safer assumption: and so the rule prevents you from making bad assumptions without knowing what you're doing.

    Understanding the summary is a good idea; understanding the full history of why that summary was written is even better. You'll learn why the rule was made, who thought it was a good idea, in what context and to what extent to apply it, what the risks are for not following the rule, and when it doesn't make sense to apply the rule at all.

    For example, hurricane insurance is often a good idea. But this rule works much better for people living in New Orleans than it does for, say, people living in the Nevada desert.

Re: Breaking The Rules
by zentara (Archbishop) on May 31, 2006 at 11:18 UTC
    I always follow the "rulez". :-)

    I'm not really a human, but I play one on earth. flash japh
Re: Breaking The Rules
by zakame (Pilgrim) on Jun 03, 2006 at 14:30 UTC

    I find rules quite exceptional. I also find exceptions to rules quite breakneck ;)

    Programmers make mistakes; of course, they're only human (at least, humanly speaking.) But the true test of a good programmer lies not on how he or she can code the fastest, cleanest, and most appealing code, but rather on his/her understanding of what s/he really wants to do, and how to do it using whatever conceptual apparatus available.

    I remember the quote from Brooks, borrowed from Shakespeare:

    "I can summon spirits from the vasty deep."

    "Well, so can I, and so can any man; but will they come for you when you do call for them?"

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://552576]
Approved by ww
Front-paged by prasadbabu
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chilling in the Monastery: (5)
As of 2024-03-28 13:03 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found