Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

5.6->5.8 Analysis

by Anonymous Monk
on Jul 11, 2003 at 13:32 UTC ( #273351=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Has anyone seen a script/module that will scan one of your scripts looking for things that might be incompatible in changing from 5.6 to 5.8?

I've got a script that seems to have a memory leak on my 5.8 system but works fine on the 5.6 systems I've been using, and I can't figure out what's causing the problems. I assume it has to do with the changes to IO, since it operates on some largish (~500MB) files, but I'd like to see if there's anything else I'm missing.

Basically, it's looping through a flat file of records, regexing out some data, and loading it into Oracle.

Replies are listed 'Best First'.
Re: 5.6->5.8 Analysis
by Abigail-II (Bishop) on Jul 11, 2003 at 14:09 UTC
    There have been some pretty large changes in the internals of perl's IO handling when going from 5.6 to 5.8. But seen from the language, there are no backwards compatability issues. So, even if your wished script would exist, it still wouldn't report something.

    It's not inconcievable a memory leak was introduced in 5.8. Please create a small program that doesn't leak on 5.6, but does leak on 5.8, and report it using perlbug as soon as you can. Jarkko released release candidate 2 for 5.8.1 today, but will be away for the next two weeks, meaning that there is sometime left for someone to write a patch to be included in 5.8.1. Unless the bug is already known and fixed of course. But don't hesitate, spend some time isolating the bug from your code now, otherwise you might have to wait another year or so before there's a perl that doesn't leak memory.

    Abigail

Re: 5.6->5.8 Analysis
by grinder (Bishop) on Jul 11, 2003 at 14:31 UTC

    There's no magic program that will load another script and point out problems. The only approach will be to instrument you program and get it to report what it's doing.

    Without knowing anything specific about your problem, I'd tend to point the finger at DBD::Oracle (if indeed that's what you're using) since it's the largest module you're using and has a healthy portion of XS code in it. Then again, a lot of people have pounded on it for years so the bugs should have been shaken out of it by now. Are you using version 1.14?

    Are you using complex regexps (with the /x modifier and (?{...}) or some of the hairier extensions)?

    Have you tried using Devel::Leak or Devel::LeakTrace? (you're going to need a perl compiled with -DDEBUGGING in this case).

    Can you post any code? Is this 5.8.0 or 5.8.1-RC1?

    _____________________________________________
    Come to YAPC::Europe 2003 in Paris, 23-25 July 2003.

      Sorry, forgot to log on before posting my question.

      Perl is v5.8.0.
      DBD::Oracle::VERSION = '1.14'

      I'm mainly looking for something that will just scan the calls and print out a report, just so I have a list of candidates to narrow down my search as to what's broken, like:
      'open' behavior has changed, IO internals have been modified
      etc.

      pseudocode:

      open(INF,"<$filename") || die "Couldn't open in file\n"; my $id = 0; my $data = ""; ... my $statement = $dbh->prepare("insert into data_table values(:1,:2)"); while(<INF>) { if (s/^ID:[\s]+//g) { $id = lc($_); } elsif (s/^Data:[\s]+//g) { $data = lc($_); } #repeat for several labels $statement->execute($id,$data); $id = 0; $data = ""; }

      There's several pieces of data that get pulled out and and compared to the data that exists in the table for a given ID, but this should give an idea of what it's doing.

      As far as Devel::Leak, no haven't used it before. Will investigate.

      I'll try and break down the code to demonstrate the leak, but I have to figure out for certain what's causing it first.

        I seriously doubt that this would cause a memory leak, but that code sample suggests that $id and $data aren't needed outside the while loop. I always try to put variables in the innermost scope that they are needed.

        --PotPieMan

Re: 5.6->5.8 Analysis
by l2kashe (Deacon) on Jul 11, 2003 at 14:00 UTC
    If you can I would post some code. I have not found in my personal experience anything like you are talking about between 5.6 and $VERSION >= 5.6. I work on a few gig of mail data a day, and I dont see any leaks. It could be that you aren't localizing variables, or possibly using local versus my, and the variables are never going out of scope.

    Just some thoughts, but again if you can, post code so we can test. If you can't post the original content, post some code that reproduces the leak.

    MMMMM... Chocolaty Perl Goodness.....

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://273351]
Front-paged by broquaint
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (2)
As of 2021-05-09 05:23 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Perl 7 will be out ...





    Results (100 votes). Check out past polls.

    Notices?