Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Re^2: Parallel::ForkManager, DBI using memory

by 2ge (Scribe)
on Nov 08, 2004 at 13:36 UTC ( [id://406046]=note: print w/replies, xml ) Need Help??


in reply to Re: Parallel::ForkManager, DBI using memory
in thread Parallel::ForkManager, DBI using memory

Thanks for answer BioSysadmin,

I read some documentation about this, and yes - every child process needs own db handler. Next - ofcourse, I tried commenting out DBI stuff, it works good, so it is bug in DBI, some memory leak, or what ? I don't believe that.

And thats not point, if I run 5 processes at once, or 50, or just 2. Always, when my thread ends it takes some memory, so at the finish of script perl process will consume equal memory. I hope someone will give me answer to this interesting question.

Brano
  • Comment on Re^2: Parallel::ForkManager, DBI using memory

Replies are listed 'Best First'.
Re^3: Parallel::ForkManager, DBI using memory
by biosysadmin (Deacon) on Nov 08, 2004 at 15:32 UTC
    Interesting. Another thing you might try is manually undef'ing your database handles at the end of your loop, like this:
    foreach my $x (0..$#links) { $pm->start and next; my $dbh = DBI->connect("DBI:mysql:database=customers;host=loca +lhost;port=3306", "root", "") or die "Can't connect: ", $DBI::errstr; $dbh->{RaiseError} = 1; print "$links[$x]\n"; # do something (read, update, insert from db) $dbh->disconnect; undef($dbh); $pm->finish; }
    Best of luck. :)
      Hi biosys!

      thanks for next suggestion, I tryied that, unfortunately it doesn't help, also you have one little error in your posted script, undef($dbh) should be before $pm->finish. Any more ideas ? :)
      I really don't know how to solve this, I have around 15.000+ iterations, so I will always run out of memory by using this :(

      --
      Brano

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://406046]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others browsing the Monastery: (2)
As of 2024-04-26 04:13 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found