Beefy Boxes and Bandwidth Generously Provided by pair Networks Joe
Problems? Is your data what you think it is?
 
PerlMonks  

Program size and effeciency.

by murugu (Curate)
on Jul 08, 2004 at 07:09 UTC ( #372689=perlquestion: print w/ replies, xml ) Need Help??
murugu has asked for the wisdom of the Perl Monks concerning the following question:

Wonderful Monks,

Last week I have joined in a new company as Perl Programmer. The concern is working on converting plain Text file to valid XML/SGML files.

They have given me a older project to take care of and to clear bugs in them.

The code is of about 11,000 lines which was written before two years.

The code is not done properly and the code is not effecient. Simple things have been done in very complicated way. Infact, they have reinvented the wheel in many parts of the code.

I have asked my boss inorder to rewrite the code in effecient way. But he is saying that PERL is not for effeciency, It is only for getting things to be done.

What i want to know is, How important is program size related to the effeciency of the project?. Does it meant that what My boss saying is true?

Please give ur valuable suggestions.

please forgive me for my english.

As usual, Million thanks in advance.

Comment on Program size and effeciency.
Re: Program size and effeciency.
by davido (Archbishop) on Jul 08, 2004 at 07:19 UTC

    Your boss has to weigh the cost of a few more seconds of processor time versus the cost of paying you to rewrite the script, and then to debug what you wrote (a rewrite is going to introduce new, unexpected bugs).

    Perl is about getting things done. While you may be itching to fix the code you've been charged with, it may not be cost effective to do so.

    One of the biggest mistakes a programmer can make is to go and make a bunch of significant changes to something that's been working fine for a long time. Change may be warranted, but take the time to understand before jumping in to big changes.


    Dave

      Definitely. Paul Graham is a well-reknowned software guru who has written a lot about this idea. I've read one of his essays (available on his website) in which he talks about the early days of his online e-commerce venture, and how using a very high-level language (LISP in his case) translated to a great deal of productivity.

      He's a LISP hacker, but once you forgive him for that he really has some interesting stuff to say. :)

      It sounds like one of the concepts that you are attempting to convey to your boss is that very functional Perl code can very quickly grow into an unmaintanable mess. If that's the case, then I think that you could point out to your boss that it's very difficult to make Perl get things done when your programmers cannont understand the codebase with which they are interacting. Your company may gain value by refactoring and redesiging (or at least properly documenting) your Perl codebase. In making a case to your boss, there are a number of software engineering books on refactoring, design and documentation.

Re: Program size and effeciency.
by Happy-the-monk (Monsignor) on Jul 08, 2004 at 08:01 UTC
Re: Program size and effeciency.
by Mercio (Scribe) on Jul 08, 2004 at 08:07 UTC
    If it is running and gets the job done then there really isn't a need to recode it.

    If it's not broken then don't fix it
      Except when you have to maintain it.

      Blackboxes, such as people in certain roles and code, are evil. Sure, roles can become quite valuable, just like some projects, and require some study to understand, but it's hurtful in the end.

      Should one maintain this code, without refactoring anything, and making things "better" (to maintain or in scalability), you'll add months, maybe years of your view of how things should be on top of the prior person's. He may not be a bad coder, but if it doesn't do things "right", i.e. using functions/methods to encapsulate shared logic, CPAN where things are done "right", then you increase the size of the project and wind up with a big ball of mud.

      The same could be said of roles in a company. If someone holds too much information and doesn't share, they become a blackbox and in turn, become dangerous.

      If he does convince them to redo it, he has to be sure to find out how much of the project the code represents, and the code itself, is. Piecemeal vs in one big shot. Anytime a choice exists, it would have to be weighed out in conjunction or in place of other options.

      It's all systems analysis and design.

      Bart: God, Schmod. I want my monkey-man.

Re: Program size and effeciency.
by chime (Friar) on Jul 08, 2004 at 09:08 UTC

    Isn't Perl wonderful
    Legacy code - let me see - yes, written two years ago still works well enough to be left in production. And in the two years that the program has been ticking over and doing its job, the Perl language has constantly improved and become more efficient.

    Software and IT is a new and everchanging industry. I expect that code that I write today will one day become ineffiecent and will need to be rewritten. (hence lots of comments and documentation).

    Also think about why were you hired - your job title is a Perl programmer - Your manager hired you for your Perl experience and your ability to bring on board the wealth of knowledge of the Perl community.
    He must have faith in you to fix the legacy code and to make it more effecient.

    All I can say is just get stuck in.
    Perl has CPAN and lots of code out on the www. Also the Open Source/Sharing of code tradition which are so valuable.

    I have just come across a node called
    "Suggestions for working with poor code"
    It has lots of excellent advice

    Finally congrats on the new job !!!
    :o)

Re: Program size and effeciency.
by adrianh (Chancellor) on Jul 08, 2004 at 09:15 UTC

    I have asked my boss inorder to rewrite the code in effecient way. But he is saying that PERL is not for effeciency, It is only for getting things to be done.

    What i want to know is, How important is program size related to the effeciency of the project?. Does it meant that what My boss saying is true?

    It some ways your boss is completely correct. The point of a Perl program, or a program in any other language, is to get the job done. A nice neat program that doesn't do the task it was written for is of no use to anybody.

    At the moment you have a working program, and your boss has finite resources. What's the better decision for your boss:

    • Spending resources on tidying up something that already works?
    • Spending resources on a problem that's not been solved yet?

    The decision is a fairly easy one to make :-)

    Would it be better to have written a more structured program in the first place - yes. It makes later maintenance much easier.

    However, since it's now a done deal I wouldn't touch it until you need to change what it does. At that point you can then clean it up incrementally as you make changes. There is no point spending resources now to tidy something up that may never need to be changed.

    However, if your boss is implying that Perl programs are naturally more inclined towards quick and dirty solutions rather than efficient well structured programs then he is incorrect. You can write efficient well structured Perl with very little effort.

Re: Program size and effeciency.
by qq (Hermit) on Jul 08, 2004 at 10:04 UTC

    They have given me a older project to take care of and to clear bugs in them.

    Start from the bugs. Clean up as you go, and only so much as you need to.

    Resist the temptation to start from scratch, because you'll get blamed for whatever new bugs you introduce. Remember that you'll find a lot of seemingly non-sensical lines of code - but each was put in for a reason, and some may be crucial. Until you are very familiar with the problem at hand don't assume you know all the wrinkles.

    qq

      I agree with this node. What I would additionally recommend you do is to start writing tests. That way, if you change one thing in the old code and it causes a bunch of stuff to fail, you can then consider inserting your favourite perl module as a "quick test", and if the perl code passes the test, (or if you can tinker with the orignal code to make it work, without causing other things to fail), you could be golden. Good luck!


      ----
      Zak - the office
      Or you'll get blamed for having to take a long time to perform the smallest change. I remember once dealing with such a system.. took me a week to change the price of something. People got all excited when I finally got it. Scary stuff.

      One time, I did get permission to rewrite a piece of that system... you were right, people did get annoyed when new bugs came up. BUT, after the bugs settled, it was the fastest and easier piece of the puzzle to modify. Some time passed, and they were doing efficiency analysis on the system to see where they could improve on it. Turns out the part that got rewritten was skipped since it was the fastest part of the entire thing. :)

      Yes, tooting my own horn, but thought I'd share the positive experience of rewriting, and the negative of not rewriting. There /are/ times when you shouldn't. Just not never.

      Bart: God, Schmod. I want my monkey-man.

        I've has an similar experience to the one you describe here. I need to make a simple change to a critical piece of code in the system I am responsible for after the coworker who wrote it had left the company. He had given me a walk through indicating what needed to be done, and essentially suggested it was a trivial change. Unfortunately time didnt permit me to do the change with him, and once he had gone it turned out that what he had explained was wrong. I spent a week debugging the code, tracing through the code, etc. And got nowhere. Changes I made seemingly did nothing. As I knew there were changes coming along that would be much more drammatic I decided that given a trivial change was turning it a huge effort the future changes would be signifigantly more difficult, and frankly not worth the time. End result I wrote a total replacment, in only a little more time than it had taken to decide the old code was unmaintainable.

        That replacment provided a strong foundation for the much more drammatic enhancements that came down the pike. The time invested in rewriting and restructuring the code paid off big time as the new code base is much easier to extend, and has been pushed far beyond any of the changes I had thought were coming. Had I persevered with the original code base and taken another week to get that trivial change in place the stuff that came after would be impossible.

        I think what the OP needs to do is present a structural argument as to whether the code base should be refactored. If the argument is convincing his boss will give him permission.


        ---
        demerphq

          First they ignore you, then they laugh at you, then they fight you, then you win.
          -- Gandhi


Re: Program size and effeciency.
by Jeppe (Monk) on Jul 08, 2004 at 12:29 UTC
    Well - consider how much time you will spend on clearing out the bugs in the legacy code, and consider how much time it will take you to produce tested, correct, efficient code from scratch. If you can reduce the 11,000 lines to 1,000 lines of lean, efficient, maintainable and correct code, it might be worth it - if writing those 1,000 lines takes roughly the same amount of time as fixing the flaws in the legacy code.

    However, you could also argue that you should only rewrite those portion of the legacy code that is truly horrible. Is everything in the legacy code really so bad?

    Anyhow - start by carefully considering how much time you'll need fixing the legacy code and how much time a re-implementation will take.

    On a more general note, Perl can be very efficient, or very inefficient - both in terms of time to develop a solution and in terms of runtime speed. Usually, you need quite a bit of experience before you learn to write efficient, maintainable code in Perl!

Re: Program size and effeciency.
by samtregar (Abbot) on Jul 08, 2004 at 19:52 UTC
    Drop whatever you're doing, go to your local bookstore and buy a copy of Perl Medic by Peter Scott. You can thank me later. This book is precisely what you need to read to deal with your new job.

    -sam

Re: Program size and effeciency.
by qq (Hermit) on Jul 08, 2004 at 21:57 UTC

    PERL is not for effeciency, It is only for getting things to be done

    Perl is for getting things done efficiently.

    ...but duplicating a working program (if it is working) isn't efficient.

    The relation of program size to efficiency is a bit obscure, because each time you use Some::XML::Module you're pulling in god knows how many lines of code. Of course, these are LOC that you don't have to write or maintain. In some sort of bogus way the ratio of LOC that you write to LOC that you leverage via use could be considered a measure of efficiency. Sort of.

    qq

      I understand the boss' reasoning, I just don't like his excuse. It's hard for us to walk away from a challenge like that.

      It's a toss up really. Maybe if the code had used some modules then the bugs in the modules would have been maintained by someone else and you wouldn't have to fix those bugs as they're discovered in your reinvented wheel.

      I'll agree that a complete rewrite is probably overkill though. I think you could satisfy your desire to fix it by refining the old code by rewriting it one block/function/method at a time. I'm sure it will prove just a challenging and maybe more rewarding to be able to show a before and after.
Re: Program size and effeciency.
by gopalr (Priest) on Jul 10, 2004 at 09:22 UTC
    Hello Murugesan, H r u ? As u know I was looking after that scrips. Before me 3 peoples looked after that old script. It was getting more than 1 week to observe that old script. Because of more than 160 Journals and more than 130 DTDs' are there in that project. So that I discussed with my boss, he said that create patch for individual journal. Don't modify with core file. As per my boss instruction, I create individual patch file for individual Journal. Is there any clarification in that script, pls. let me know. All the best for ur new assignment. with love Gopal R.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://372689]
Approved by davido
Front-paged by grinder
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (11)
As of 2014-04-23 22:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (556 votes), past polls