Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re: Re: Re: Re: A Perl aptitude test

by tilly (Archbishop)
on May 05, 2003 at 17:45 UTC ( #255684=note: print w/ replies, xml ) Need Help??


in reply to Re: Re: Re: A Perl aptitude test
in thread A Perl aptitude test

It varies by codebase, project, and most importantly, who you expect to work with and then maintain it later.

The problem is that TIMTOWTDI is a maintainance headache. Anyone who wants to abuse it forces you to have all maintainance work done by someone who has more expertise than the regular programmer. Not the same, because someone who is merely the same may just know a different set of tricks, but better so that they are likely to know all of the tricks that the first show-off used.

The answer is to decide which ways to do it you will settle for based on other factors, and to stick to them so that someone else can pick up the code and be brought up to speed in a reasonable amount of time. Having people who know lots more ways to do it is fine and good and may be a good thing (knowing your choices in advance makes it easier to pick a good set of choices to stick to), but actually using them all all of the time is a bad thing to do unless you are trying to be hard to understand.


Comment on Re: Re: Re: Re: A Perl aptitude test
Re: Re: Re: Re: Re: A Perl aptitude test
by BrowserUk (Pope) on May 05, 2003 at 23:37 UTC

    Sorry, this got a bit wordy. I trimmed it twice, but came close to failing to explain my pov.

    It varies by codebase, project, and most importantly, who you expect to work with and then maintain it later.

    I understand where your coming from, and in any given environment it may make sense to deprecate specific subsets of language constructs through a local coding standard. In the general sense, especially if the granularity is at the project level within a given organisation or codebase, then I have distinct reservations. My major concern is that it then becomes increasingly difficult for any given programmer to move between projects as he will constantly have to review the local coding standards in force on that project before setting to work. And applying a single set of restrictions across the whole codebase, especially in organisations that code projects into many different project arenas (eg. database, financial, scientific, embedded etc.) is likely to limit one or more areas to using less than optimal solutions in order to comply with the coding standards.

    The problem is that TIMTOWTDI is a maintainance headache. Anyone who wants to abuse it forces you to have all maintainance work done by someone who has more expertise than the regular programmer. Not the same, because someone who is merely the same may just know a different set of tricks, but better so that they are likely to know all of the tricks that the first show-off used.

    This identifies (for me. YMMV) the crux of the problem. Many, many organisations whilst agreeing that maintanance is a sizable proportion of the cost of software development, still tend to relegate the process of maintainance to a 'second string' section of the development team as a whole. They often pay large sums for their development 'stars' and considerably less for the maintenance team. This second string status doesn't limit itself to just salaries either. It is also evident in the training and personal development budgets and even access to conferences, symposia, reference materials and beyond.

    They then proceed to hamstring their stars, by imposing coding standards that stiffle innovation, creativity and in-depth knowledge -- that has usually come at the cost of considerable effort and dedication -- to only using that subset of their knowledge that's deemed as understandable by the maintenance team.

    Beyond just the inequalities observed, this process of delineation between job functions seems self defeating as it precludes making full use of the inherent skills of the one group of individuals and condemns the other to never progressing beyond the limits set for the 'average joe' programmer.

    My observations and experiences suggest to me that the problem lies in the whole concept of seperate development and maintenance teams. In fact, the whole practice of building project teams as mono-functional, kick-off to release entities is flawed (IMO). The best software organisation I was ever involved with took a radically different approach.

    Breifly, when a project was in the pre-approval stage, a single project leader was designated. This person, more manager than analyst or coder was charged with putting together the project plan. S/he then employed the skills of one or more members of the analysts group, as & when required, to assist gathering and analysing user reqs, generating ideas, writing and checking proposals.

    Once the project was approved, they would produce a project plan in terms of Job-roles rather than named individuals. The requirements for analyst time would be 'requisitioned' from the analysts group and the same with the coding group. The two groups having considerable overlap with people moving from one to the other as daily/weekly/monthly requirements arose.

    The Job-roles "owned" the code in perpetuity, and when effort was required, whether green field development or maintenance, the "next available coder" would be assigned to the task.

    This would be influenced by the scale and nature of the specific task and the experience required, but generally, the "next available coder" tended to be a less senior member -- by definition, they usually had less on their plates. They would do the initial assessment, code the solution if they could, but frequently refer it back to a more senior member if they got stuck. Regardless of who did the coding, it was always peer reviewed with at least two other coders. The upshot of this is that all projects were planned in terms of job-roles and all work was carried out in those terms.

    There were no stars, and everyone took there fair share of the glory work and the humdrum. This caused the least strong team members to be constantly encouraged to raise their game, without any of the infighting or demarkations that I've seen elsewhere. They also had, collectively, the highest standards of coders and analysts, along with the highest rates of productivity I have ever seen. The architect of the system had a compicated football analogy, something about strength-in-depth and a 22-man first team. Football isn't my thing so I won't try to re-create it.

    The really interesting part was that I never saw the situation were anyone was reluctant to ask for assistance, no matter how senior they were. Nor did I ever see such a request refused.

    It was a true revelation to work in such an environment, and probably the hardest thing coming into it was simply believing that it could be true. I have also never seen any coding group given such a high degree of freedom or produce such a high level of product. It's just a shame that sometime after I moved on, the company was taken over and the new management had more traditional ideas. The company went down hill rapidly thereafter and bit the dust, along so many others, in the tech sector shake out. A real shame.

    As with most things, our ideas are the product of our experiences and influences, and each persons set is different. The next time I'm charged with setting up or heading up a team or project, insofar as I can influence it, this is the model I will try and adopt.


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
      Wrong rant.

      My comments are based on a situation where a small team that included me both developed and did maintainance on an evolving codebase over time. I was one of the maintainance coders (and was the fallback if things got out of hand), and don't think that lack of ability on the part of the maintainers was an issue.

      Rather my comments have to do with a recognition of the fact that good coding is all about making code clear and understandable. It isn't about showing off how crazy a line of punctuation you can spew to get things done in 2 less lines. If you have someone who has that priority wrong, and thinks that their demonstrations of cleverness are a good thing, they either need to learn better, or if uneducable, be fired.

      As for the good environment that you praise, just think about how someone who was cocky and determined to lay claim to stardom would have fit in. Not well, huh? Well those are the people who I am pointing out as being problems. By contrast the simple awareness of what will and won't be understood based on the feedback you describe is enough to achieve the goal that I am after. Not a binding of good programmers with sufficient rules to make them by force understandable to relative idiots, but just an awareness of how the team works, and what kind of code will be clear to the rest of the group.

        Given your agreement off-line, this seems as good a place as any to continue our discussion.

        I guess, as someone who makes a habit (and a career:) of, what I would term, fully understanding the systems, languages and tools I have worked with, and utilising them to their fullest, to effect solutions, you touched a nerve with various phrases.

        • ...develops an understanding of obscure constructs by constantly experimenting...
        • ...the tricks that the first show-off used...
        • ...but actually using them all all of the time is a bad thing to do unless you are trying to be hard to understand....
        • ...showing off how crazy a line of punctuation you can spew...
        • ...someone who was cocky and determined to lay claim to stardom...

        I have no idea whether I even appeared on your radar set when you made these statements, but if even if I didn't, the implication that I would fit into one or more of these disparaged groups has been alluded to from time to time here, and I guess if the cap fits, I tend to wear it.

        What I see as the difference between your stated position, as I interpret it and my own, is summed up for me -- though not necessarially by you, even though they are your words -- in the sentence.

        By contrast the simple awareness of what will and won't be understood based on the feedback you describe is enough to achieve the goal that I am after...

        I read this to mean that if, during peer reviews, certain constructs or techniques used by one or more of the team members were shown to be misinterpreted, or confusing to other team members, then those constructs and techniques would become candidates for the proscribed list.

        The approach taken at the organisation in question was to investigate such occurances and, provided that the code was without side-effects or technical flaws, and especially if it had some clear benefits -- including, but not limited to performance. The technique then became the subject of an "Advanced Techniques paper", was usually presented to the team when written and added to the project, language or tool documentation set as appropriate. The emphasis was on educating the less advanced, rather than limiting the more advanced.

        The reason that there were no 'stars' in the group was simply that the overall standard was so high, that anyone coming into the group, including those who came with a sense of their own stardom, rapidly found that they were, at best, one among many. The effect was salutary. All but a very few that joined whilst I was involved, went through a short period of floundering as they adjusted to the standards in the environment. Then a little wariness as they became accustomed to the peer review process and the way in which help was sought and given so freely, before adjusting to it and revelling in it. The few that did not make the adjustment, usually because they came from backgrounds where you protected your skillset as a way of life, generally moved on of their own volition.

        I now realise that giving the specific example has tended to obscure rather than clarify the crux I was trying to get at :(.

        The assertion I interpreted you to have made was that using the (as I would have it) full expressive power of perl to code algorithms concisely becomes a liability when the notation used to acheive it, falls into that area you variously term as "obscure constructs", "tricks", "showing off", "{as} crazy a line of punctuation you can spew".

        I have several problems with that.

        1. Where do you draw the line?

          When I first encountered perl, despite a strong programming background in other languages, I found even the simplest perlisms, confusing, frustrating and obscure. Whether is was the usual perl beginners traps of $a[1] referring to @a not $a, or the confusion over when $_ was implicitly assigned in a while loop and when it was not; the subtleties of the various forms of magic, like <> versus <STDIN>, the autovivification of intermediate terms in a complex data structures even when checking for definedness, and a host of other similar things. I rapidly learnt to appreciate the reasoning behind these perlish ideas and learnt to read them and use them to produce leaner, and I would say clearer, code.

          Even as my knowledge of perl improves, I continue to see it used (especially here at PM) in ways that I hadn't thought of, that justify the nomenclature of VHLL over and over. And with each new way of expressing the same basic algorithm, comes some advantage (at least in some circumstances) over the previous. Sometimes the improvement comes by way of performance; Sometimes because it allowes the expression of the underlying algorithm in a more concise form that more closely matches the way I think of the algorithm.

        2. All other things being equal, less code means less errors.

          Discounting the obfu and golf strategies designed to either deliberately obscure or reduce the arbitrary metric of keystrokes, neither of which I pursue to any great extent although golf can be fun. I believe that writing concise code is good, not because it is sometimes more efficient, nor because of the 'laziness' ideal, nor even for the 'once and once only' factor, though that too can be beneficial not least when it comes to maintainance -- we've all fallen foul of modifying a peice of code in one place whilst failing to modify a similar piece elsewhere.

          I beleive concise code that achieves the required algorithm is good, because every metrics analysis I have ever seen indicates that the more code there is, the more errors there are. I never seen anything that indicated otherwise.

        3. Less is more when it comes to understandability and maintainance.

          A not entirely secondary reason, is that my own experience suggests that it is always easier to get a feel for the overall algorithm if it can be viewed in one go, than if you have to scroll up and down to see the different parts. This goes right back to my first ever commercial programming course for FORTRAN77 at DEC, where the instructor suggested that any routine over 2/3 (24 line) screenfuls should be broken into two subroutines. At the time, coming from an assembler and BASIC background, that was an anathema to me. Down the years, I've come to appreciate this advice more and more.

          Now in Fortran, the costs of splitting a routine into a couple of subroutines were minimal but in perl, they are considerably higher, but the overhead is more than offset by the expressive power of perl to do a lot with a few well written lines.

          The way I've come to think of it, that may or may not appeal to you, the mathematican, is that any programming language is simple a notation for expressing the algorithm in question. In the same way that advanced math usually relies upon a shorthand notation to express the ideas and steps of a theorem, so programming relies on the expressions of the language used.

          It would be considerably easier for me to follow mathematical ideas if all the formulea and proofs were written in terms of +-/*^ sin/cos/tan etc. with all the intermediate steps spelt out explicitly. Whilst many, if not all of them could be written this way, they would become so tortuous long and repetative, that mathemeticians have evolved notations that encapsulate many of the intermediate terms and steps very concisely. If you understand the notation, then the reduction in clutter makes understanding the overall equations and proofs much easier to comprehend -- or so I'm told, my maths stop way short of this:).

          This latter point is made better and more authoratively by Stephen Wolfram here when he notes.

          In early mathematical notation--say the 1600s--quite a few ordinary words were mixed in with symbols. But increasingly in fields like mathematics and physics, no words have been included in notation and variables have been named with just one or perhaps two letters. In some areas of engineering and social science, where the use of mathematics is fairly recent and typically not too abstract, ordinary words are much more common as names of variables. This follows modern conventions in programming. And it works quite well when formulas are very simple. But if they get complicated it typically throws off the visual balance of the formulas, and makes their overall structure hard to see.

        Whilst there is undoubtably a certain "show off" factor in devising more concise forms for expressing various algorithms in perl. Provided that the aim is reduction of clutter, in the form of removing redundant intermediate terms and steps; that efficiency isn't sacrificed; and deliberate obfuscation isn't the goal; then the result of the freindly competition is often a new idiom. Whilst it may look obscure the first time it is encountered, as with so many now-commonplace idioms, it rapidly becomes familiar through seeing it and using it.

        In my 10 months of perl and visiting PM, I have seen several new idioms come into use, and learnt many more though reading the work of merlyn, Abigail, Aristotle, Juerd, sauoq and many others. I've also witnessed the process wherebye they are used (and sometimes evolved) in the course of one thread, and then start to crop up in the questions and answers of other threads. This does not usually take very long to happen when the idiom is useful. The truely obscure ones simply disappear.

        I appreciate that the audience at PM is, by definition, an exceptional environment populated for the most part by enthusiasts, but with rare exception, every 'good' programer I have encountered over the last 30 years has not just programmed for a living, but was an enthusiast.


        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://255684]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (5)
As of 2014-12-22 00:03 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (109 votes), past polls