Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

blaming perl for not using a build policy

by trwww (Priest)
on Aug 26, 2008 at 00:43 UTC ( #706808=perlmeditation: print w/ replies, xml ) Need Help??

Hello,

Over at "Redhat perl. What a tragedy." there is a perl programmer complaining how a bug in the RedHat provided perl made their code run slower than expected.

I say it is his fault he had this problem because he did not use a build policy. I think the system's perl belongs to the system and you should not be doing stuff like installing non packaged perl modules and running your production application with it.

True, RedHat has a bug in their app. But I think the developer owns more of the responsibility than the blog post leads people to believe. What do you think?

trwww

Comment on blaming perl for not using a build policy
Re: blaming perl for not using a build policy
by rhesa (Vicar) on Aug 26, 2008 at 01:02 UTC
    I thought it was common knowledge that the RH build was unsuitable for serious work. Guess he didn't get the memo. Did you notice the incredibly large @INC? Very annoying, that one. It's also got threading built in, which is also known as a drain on performance.

    I've been building perl from source for as long as I've been using RH derived distros, going back as far as (I think) Fedora 3. It runs the socks off the system perl.

      I was using RH at my last job, but had never heard of this. (Admittedly, I didn't notice any performance issues). Maybe this should be a FAQ though, or documented somewhere in the core.
Re: blaming perl for not using a build policy
by kyle (Abbot) on Aug 26, 2008 at 01:56 UTC

    I think it's reasonable to expect the system perl to be sane and use it for one's own work. I'd never heard of this problem before today, but I suppose I might have if I were using that distribution (I've used Debian GNU/Linux since about 1995).

Re: blaming perl for not using a build policy
by ysth (Canon) on Aug 26, 2008 at 05:27 UTC
    I don't understand advocating using a Policy.sh file as if it were some kind of magic wand. What matters is the configuration you end up with, whether it's achieved by Policy.sh, Configure switches, or hand-crafted config.sh.

    I personally would always go for the Configure switches, since that produces a self-documenting perl where perl -V:config_args tells exactly how it was built.

Re: blaming perl for not using a build policy
by syphilis (Canon) on Aug 26, 2008 at 10:51 UTC
    I say it is his fault

    The artistic license under which perl is distributed states quite clearly "You may otherwise modify your copy of this Package ..." (and goes on to specify some conditions that apply if you *do* modify the source).

    In view of that, why on earth would you then use a perl that has been built by someone else for the purpose of helping to run an operating system - and, furthermore, expect it to serve your *own* purposes !! He gets no sympathy from me, unless RedHat have failed to adhere to the conditions of the license under which perl is distributed.

    Cheers,
    Rob

      why on earth would you then use a perl that has been built by someone else for the purpose of helping to run an operating system

      Does this argument not also apply to any number of other tools on the operating system? Why use the system's bash? Why use the system's libc? I install a distribution specifically so that I don't have to compile all my own software from scratch. I don't think it's off the wall to assume that what's good enough for my distribution's purposes is good enough for mine.

      I also think it's normal that an OS's version of some software is not suitable for what I'm doing, especially if what I'm doing is particularly sensitive to performance or particular features. That, however, should be the exception, not the rule. For more mundane work, I expect the OS's tools to be acceptable, not, as in this case, a thousand times slower than necessary.

        I don't think it's off the wall to assume that what's good enough for my distribution's purposes is good enough for mine

        Except that my take on this is that, whereas "bash" is being provided primarily for *you* to use, "perl" is there primarily because the system needs it.

        And you *are* quite free to "assume that what's good enough for my distribution's purposes is good enough for mine". It's your right to complain about the fact that your assumption is in error that I'm not so sure about :-)

        Why use the system's libc?

        I don't know ... probably for convenience ... which presumably is the same reason that you'd use the system's perl. But, having accepted the system's libc, I don't think you can then go and complain about it (unless it has been jiggered with in a way that contravenes the license under which it is distributed). Same goes for the system's perl.

        Cheers,
        Rob
Re: blaming perl for not using a build policy
by rhesa (Vicar) on Aug 26, 2008 at 11:29 UTC
      For those who do not follow the link, the upshot is that rather than go with the standard released version, Red Hat decided to go and add random patches that were floating around. He discovered this because a Red Hat user ran into resulting bugs in the wild. The core Perl sources were patched in November, 2007, but Red Hat doesn't understand or care enough to issue a system update.

      He's also really not happy that Red Hat tracks bugs in its bug tracking system but doesn't see fit to share information with the originating communities. So, for instance, when this bug in Red Hat's Perl (not in any version of Perl the Perl folks have released, but it is in Red Hat's version) had the bug reported they closed the bug report saying, "upstream patch". Which doesn't acknowledge that it was a patch they shouldn't have been integrating in the first place, and doesn't tell the upstream folks that there is a bug in an upstream patch that maybe, just maybe, they'd like to fix.

      That said, I can understand Red Hat being cautious about replacing the Perl they ship. Lots of things depend on Perl, lots of things are compiled against Perl, and so even an innocuous looking patch has a possibility of creating its own bugs. There is a certain sysadmin mindset that looks at that kind of situation and says, "It is better to leave the problems we know about rather than create new problems." Of course when you combine that mindset with an, "Ooh, shiny!" attitude towards including unstable stuff and you get one of the reasons why I personally prefer Debian-derived distributions.

Re: blaming perl for not using a build policy
by sasdrtx (Friar) on Aug 26, 2008 at 12:00 UTC
    Building one's own Perl installation may be the best practice, and maybe all hard-code monks do that. But for those of us of the unwashed masses, that's too much trouble.

    Most programmers don't want to deal with system issues; that's one of the reasons for getting a Linux distro you like; so you don't have to compile everything yourself.

    If your code is going to be distributed widely, then you have little control over what Perl is going to be running it anyway.

    I think the article's author has a good point... Redhat is undoubtedly doing some damage to Perl because they are distributing a bad version of Perl. I agree that one should expect their distribution to contain a non-broken and performant copy of Perl.


    sas
      But for those of us of the unwashed masses, that's too much trouble

      It's ok to use the vendor-supplied perl, but (imho) if you do that, it's unreasonable of you to then complain about its shortcomings.

      Cheers,
      Rob

        It all comes down to why we've ended up with this version of Perl being distributed by RH. If they had legitmate reasons for compiling it in that way, then I guess it's fair enough. If it's an error on their part, then it's something they should fix immediately.

        That said, even if they did have specific reasons for breaking Perl for everyone else, then maybe they should've shipped with 2 versions of Perl, one used by their code, and one for general use.

        I can't really accept that a barrier to using Perl should be that you have to compile it yourself. Are we trying to drive people away deliberately? How successful would Linux be if everyone had to compile their own kernel (even for commercial use)?

Re: blaming perl for not using a build policy
by samizdat (Vicar) on Aug 26, 2008 at 13:29 UTC
    There is a long history in the commercial 'NIX world of compilers and other tools being abominable. It's no different than any other commercial software endeavor, and popularity is no guarantee. A good example that come to mind is the C compiler that shipped with HP-UX right up until the end of the twentieth century, which was so bad as to be mostly unusable. Another is that Solaris until recently shipped with a very early Perl (5.003, IIRC). (I have a friend at SNL who is STILL dealing with some of these old monstrosities. He's not likely to be allowed to retire.)

    The very nature of the true freeware community is that SOMEBODY's going to want something to work on at three in the morning, and that's why GNU 'ls' has forty-'leven options and most of them work the way they're intended to. The very nature of he commercial world, and that includes Red Hat, is that you freeze it and get it out the door. To say the commercial world should be otherwise is like saying that rocks should fly.

    I wholeheartedly agree that anybody doing production work of any kind for commercial purposes MUST be prepared to investigate, understand, and validate their tools, all the way down to the raw os. And yes, I've also been known to run acid tests on even generic PC motherboards. You'd be surprised what crap gets out there into the field, especially when you're trying to bleed the edge. You have to be prepared to take a trip out to the back of the sausage factory, and you can't just spill your cookies after you do so.

    Even when you stand on the shoulders of giants, you must be prepared to find out which way they're headed. YMMV isn't just a saying, it's an expectation that comes with the territory. It's your responsibility to use wealth wisely, be it FOSS or payware or something in-between.

    Don Wilde
    "There's more than one level to any answer."
      A good example that come to mind is the C compiler that shipped with HP-UX right up until the end of the twentieth century, which was so bad as to be mostly unusable.

      Yeah, and their broken vi. Solaris has issues, too; that's right. But those UNIX vendors have a tight license policy, and while they all have roots in good ol' Berkeley, their software is closed source.

      RedHat, on the other hand, and more Fedora, build their distributions on open source - to the extent that they exclude stuff ruled by software patents, like mpeg encoding - for anyone to look at, test, and provide a fix at three o'clock in the morning.

      The only commercial aspect of their's is testing, packaging, distribution and commercial support for what they call "Enterprise" (well, certification of their distros too, for closed source stuff like oracle and SAP). They should, for the sake of their reason to be, get that right - and heed the open source community to get it right. The perl fixes are out there for some time now.

        Yup, absolutely. And, now that FreeBSD's got a handle on truly scalable multiprocessor, Slowlaris has nowhere to hide. Linux won't be far behind, they all look over each others' shoulders.

        Don Wilde
        "There's more than one level to any answer."
      The very nature of the true freeware community is that SOMEBODY's going to want something to work on at three in the morning, [...] The very nature of he commercial world, and that includes Red Hat, is that you freeze it and get it out the door. To say the commercial world should be otherwise is like saying that rocks should fly. [...] It's your responsibility to use wealth wisely, be it FOSS or payware or something in-between.

      (Additional emphasis by me.)

      I personally believe that you're making a big confusion between freeware, which is not free software and open source software which is a development model and needs not be freeware nor free software. You're also making confusion between commercial software and (implicitly) proprietary software, whereas the former can actually be free software, just as much as freeware can be at the same time proprietary, and often is!

      I'm also all with you in supporting gratis, free and OS software, and somehow deprecating commercial, proprietary software, but yours is far too drastic a generalization: some would argue that a developer, or a software house in the second category, depending on people paying for the quality of their products, will have someone having "to work on your problem at three in the morning." OTOH you certainly know that there are tons of freeware, free software and open source projects which are not developed any more because of lack of interest from the original developers or some other reason.

      --
      If you can't understand the incipit, then please check the IPB Campaign.
        I accept your distinction, blazar. :)

        Don Wilde
        "There's more than one level to any answer."
      There is a long history in the commercial 'NIX world of compilers and other tools being abominable

      Partially true. For instance, many vendors ship a free (C) compiler (or make a free compiler available for download), which tend to not be very good. However, give me a commercial, paid-for, C compiler anytime. It tends to build faster binaries than gcc (which doesn't mean gcc is bad, not at all, but gcc targets a wide range of platforms, and its developers usually don't have the same intimitate knowledge of the OS as the commercial vendor does). I've build perl with both commercial compilers and gcc on the same platform, and most of the time, perl build with a commercial compiler was faster in my tests (although never more than 25%).

      Another is that Solaris until recently shipped with a very early Perl (5.003, IIRC).

      Not my experience at all. AFAIR, Solaris went from a "no perl" policy to "whatever is newest when the Solaris code freeze happens". Alan Burlison was always pretty keen to have the newest Perl included, although I don't think he's involved in that process anymore.

        Solaris Perl versions:
        • Solaris 8: Perl 5.005_03
        • Solaris 9: Perl 5.6.1
        • Solaris 10: Perl 5.8.4

        Note: I happened upon this thread and thought I would provide the extra info for anyone searching on this later on.

        Elda Taluta; Sarks Sark; Ark Arks

Re: blaming perl for not using a build policy
by Tanktalus (Canon) on Aug 26, 2008 at 13:42 UTC

    My two cents: Using the system (?:perl|gcc|libc|rpm|deb|python|wx|jvm|libaio|libpcre|ruby|gnumake|cmake|libogg|mesa|bind|postfix|hal|xinetd|gettext|kernel|kernel source|cron) (just as a random sampling) and expecting it to work should be a given. There is no excuse for this stuff to be crap - just because "users" don't use them directly, but the system does, doesn't mean that they should be crap. This throws the entire distro into a tailspin: you'll report a performance problem to the upstream maintainer of some software that relies on one of these things, and they'll have no hope of reproducing and fixing because the problem is actually with your system rather than the other application. The fact that YOU are the "upstream" for some apps should have no bearing on this.

    This is a different stand than I take on trying to use a *different* version of perl (or whatever - I'm not repeating the list) on the system than what comes with the distro. For that, I suggest putting the new version in a new path and not overwriting the old version at all, because the distro likely hasn't been tested with the new version, and especially with some core things like libc, perl, bash, rpm, deb, etc., you can really end up with an unusable system afterwards if you screw it up (either by building wrong, or by replacing with a binary-incompatible version or whatever).

    If the version on the system is the correct level for you, why should you need to compile a private version? That just really makes everything more difficult for what should be no real reason. Down this path lies madness: instead of relying on the system libc, every application comes with its own... just because the system is unreliable? Fix the system, not the app.

Re: blaming perl for not using a build policy
by shmem (Canon) on Aug 26, 2008 at 13:52 UTC

    Installing a distro, I expect all things in it well integrated and optimized for this combo of packages and tools. Where that is not the case, I call it a bug. More if this system carries the attribute "Enterprise". A system is not there for its own sake, and there's no such thing as a "system-only perl". The system is there for applications to run, not the other way round.

    It is a shame RedHat lets this bug sitting around. But then, they clearly favor python over perl since anaconda. It is irrelevant whether they just don't care or are trying to discredit perl - the result is the same.

    building your own perl is not an option on enterprise systems which are tied to the RHN update process; and it is there this bug is most likely to cause the most waste. Everybody subscribed to RHEL should complain at RedHat.

Re: blaming perl for not using a build policy
by Mr. Muskrat (Abbot) on Aug 26, 2008 at 16:56 UTC

    The responsibility lies with Red Hat. We pay them for support. Support goes beyond answering phone calls and email. Support includes fixing known bugs (especially those that already have patches available).

    We use Red Hat Enterprise Linux at $work because the PHBs get a warm fuzzy feeling deep down inside from knowing that Red Hat is just a phone call away should something go horribly wrong. We use the version of Perl that ships with Red Hat because, once again, the PHBs love that warm fuzzy feeling. If we build Perl ourselves then Red Hat won't support it and the PHBs lose that warm fuzzy feeling.

    None of us knew about this problem because we simply have not noticed any performance degradation. Does that mean that are simply going to ignore it? Nope. We are not currently affected by the bug but that does not mean that we will never be affected by it. I will be calling our rep and demanding that they resolve this issue. We pay too much money to Red Hat to do any less.

Re: blaming perl for not using a build policy
by perrin (Chancellor) on Aug 26, 2008 at 17:32 UTC
    I fully agree with building your own perl, but what build policy options would fix this problem? I don't think you could tweak RH's build policy and fix it.

      ...hat build policy options would fix this problem?

      Right, who knows? Who wants to spend the time worrying about it? So what I do is create a user, build a perl in that user's home directory, and update the user's path. Because I'm a web developer, I do the same with apache. Then the application runs from that user's context instead of a system context.

      Sure, I have to rely on what the distro provides for most things (networking, a sane compiler, and an easy to use update system). So I start with a distro that I trust for the things I need to trust it with. But for my application, I build the critical components myself.

      I don't know how many times I've seen where people deploy thier app using the system provided stuff for critical components (like perl), and then their app busts when they do an update. If the critical core components of the app would have been bundled with the app, the problem would have been completely avoided.

      trwww

        I don't know how many times I've seen where people deploy thier app using the system provided stuff for critical components (like perl), and then their app busts when they do an update. If the critical core components of the app would have been bundled with the app, the problem would have been completely avoided.

        It may avoid one problem, but it creates another. If I am compiling my own Apache then I am also burdening myself with deploying a new Apache (or other component) every time there is a security hole. We already have tools to do this work for us.

        Whether I update my components manually or let the system do it for me the burden is still on me to actually test that my code still works after the update. Compiling and deploying my own components is more work than letting the system update itself. Testing my own code takes just as much time in both cases.

        Blindly running 'apt-get upgrade' is just as bad as blindly installing a freshly compiled version of an important component. Either way, it should be simple enough to roll back a package to the previous working version.

        Oknow
Re: blaming perl for not using a build policy
by dragonchild (Archbishop) on Aug 27, 2008 at 03:18 UTC
    I started using F/OSS because it is free. Then, I used it because it is good enough (and more) for what I do. Then, I finally understood why I must use it and that is because I can tailor it to what I want.

    The version of perl (or apache or mysql) that comes with the OS should be used solely by the OS. The reason RH still ships Perl 5.8.0 (patched heavily) with their OS is because their scripts work with 5.8.0 and upgrading may break that. Installation and maintenance scripts can take a little longer to run, but they have to run successfully.

    My highly optimized webapp has very different needs than the OS perl. As such, I shouldn't be screwing with the OS perl by installing additional modules or upgrading stuff. I may break it. So, I ALWAYS compile my own Perl, Apache, and (in some cases) MySQL. I ALWAYS use my own gcc to do these things (largely because I want gcc 4 and the massive improvements over gcc 3.x). And, I ALWAYS put it somewhere else, in its own directory structure and usually in its own jail. And you should, too. Until you've seen an OS fall apart because someone upgraded a CPAN module and you had to fix it at 3am on a Sunday, you haven't lived. :-)


    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
      The version of perl (or apache or mysql) that comes with the OS should be used solely by the OS. The reason RH still ships Perl 5.8.0 (patched heavily) with their OS is because their scripts work with 5.8.0 and upgrading may break that.

      This is what quite a lot of people have been claiming in this thread thus far, and you're the last one in order, so I'm replying to you that I may well buy your argument; but then the OS should clearly say so, to the effect that the Perl installation should not be touched in any way, and probably not even be used but possibly as a fallback solution if no other perl is available: its perl executable should not even be in the path, since for installation scripts the precise indication in the shebang line would suffice.

      As far as I'm concerned, I think that the System perl should be robust enough not to warrant any major problem, and offer a reliable working environment. But above all, if system initializations scripts depend on it, then they should be written well enough not to break for something as innocent as a module upgrade. Which is what that holds for quite about any other program: if I write one say to use 5.008; then it should be guaranteed to work with any perl version higher or equal than that one, shouldn't it?

      --
      If you can't understand the incipit, then please check the IPB Campaign.
        This is what quite a lot of people have been claiming in this thread thus far, and you're the last one in order, so I'm replying to you that I may well buy your argument; but then the OS should clearly say so, to the effect that the Perl installation should not be touched in any way

        Which includes not shipping a CPAN.pm and a cpan script. Clearly you never need CPAN for a perl that's never used for anything else but for the the operating system itself.

        IMHO, placing a binary in /usr/bin is a clear indication that it should be acceptable for general use.

        If something is there only for the system, it should go somewhere else (like /sbin or /usr/sbin).
        "but then the OS should clearly say so, to the effect that the Perl installation should not be touched in any way, and probably not even be used but possibly as a fallback solution if no other perl is available"

        ++. Please.
        but then the OS should clearly say so . . .
        I find it really ironic that on a Linux system people have no problem messing with the system's Perl while they would likely not even think about touching it if they were running a commercial version of UNIX such as Solaris. It shouldn't have to be documented because it should be obvious, but for whatever reason the word "Linux" is justification to trump System Administration Best Practices. My guess is that people are used to playing around with Linux as a hobby and thus forget to go back into Best Practices mode when using it at work.

        Sorry for the late post. I stumbled upon this thread today and thought I would respond to a few of the comments in it.

        Elda Taluta; Sarks Sark; Ark Arks

      The version of perl (or apache or mysql) that comes with the OS should be used solely by the OS.

      There's no distinct line between OS- and non-OS-stuff in RedHat; /usr/local/bin is there only because it's supposed to be there; all app stuff is jammed into /usr invariably. Is OpenOffice as shipped by RH meant to be used by the system only? Where does a RedHat system need mysql?

      The reason RH still ships Perl 5.8.0 (patched heavily) with their OS is because their scripts work with 5.8.0 and upgrading may break that.

      Mind to provide anything to prove that? If "their scripts" only work with a patched perl and might break with a perl release change, they are poorly written in the first place. But then, their OS related scripts are written in python. It is possible to de-select perl upon install, but it will be installed anyways because of bogus package dependencies.

      Let's have a look. Where does RH's systems rely on perl? As of a RHEL-4 update 7 default install (which ships 5.8.5, btw)

      [shmem@rhel4 ~]$ for pkg in `rpm -q --whatrequires perl | grep -v perl +`; do rpm -ql $pkg | xargs perl -nle 'exit if -d $ARGV[0];/\bperl\b/ +and print $ARGV and exit' ; done /usr/bin/genkey /usr/bin/foomatic-compiledb /usr/share/foomatic/db/source/printer/NEC-PinWriter_P6_plus.xml /usr/bin/decode-dimms.pl /usr/bin/fileshareset /usr/share/mimelnk/application/x-perl-module.desktop /usr/X11R6/bin/xscreensaver-getimage-file [shmem@rhel4 ~]$

      the only script "(C)RedHat" is /usr/bin/genkey of crypto-utils which is fairly old and doesn't rely on any esoteric perl configure settings or patches, and

      [shmem@rhel4 ~]$ rpm -q --whatrequires crypto-utils no package requires crypto-utils [shmem@rhel4 ~]$ su - Password: [root@rhel4 ~]$ rpm -e crypto-utils [root@rhel4 ~]$

      not even that one is needed by the system. I have not checked, but I suspect the dependency on perl of any later RH OS to be even less.

Re: blaming perl for not using a build policy
by JavaFan (Canon) on Aug 27, 2008 at 14:45 UTC
    I think the system's perl belongs to the system and you should not be doing stuff like installing non packaged perl modules and running your production application with it.

    I disagree. If an OS depends on a certain version/build of perl (or python, or whatever), then they shouldn't put it in /usr/bin/perl.

    Solaris got this one right (assuming they haven't changed their policy, it's been a while since I last used Solaris). They too have quite a number of scripts that may break if you replace the default perl. So they put perl somewhere else (say /some/path/to/perl) and start their programs that use perl with #!/some/path/to/perl. /usr/bin/perl is then a link to /some/path/to/perl. Now, if you want to put your own perl in /usr/bin/perl, you can do it. The systems scripts will not refer to it, and they'll keep using the default perl that came with the system.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://706808]
Approved by GrandFather
Front-paged by Arunbear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (8)
As of 2014-12-22 01:29 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (110 votes), past polls