Perl out of memory

Hi - i've been using this perl script for quite some time to pull data from a directory, I have limited perl knowledge and recently the directory has grown very big. It is big enough that I now get a 'out of memory!' error when running the script in active state perl on my windows xp desktop. I've posted the code below...does anyone know how to fix this?

if ( $mesg->code ) {
   # Handle error codes here
    print ("There was a bind error\n");
      }

$mesg = $ldap->search( # perform a search
                        base   => "ldap base here",
                        filter => "ldap filter here",
                        attrs => ['ldap attributes here']
                      );
 
$mesg->code && die $mesg->error;

foreach $entry ($mesg->entries) {
        $entry->dump;
        } 

$mesg = $ldap->unbind;   # take down session
[download]

Comment on Perl out of memory Download Code
Re: Perl out of memory by GrandFather (Cardinal) on Feb 29, 2012 at 19:31 UTC
What makes you think the problem is in the code fragment you showed? Can you add about 10 lines of code to make that a runnable script that shows the problem? How big is "very big"? True laziness is hard work	[reply]
Re^2: Perl out of memory by Anonymous Monk on Feb 29, 2012 at 20:33 UTC
Here are the first lines `use Net::LDAP; $ldap = Net::LDAP->new('ip address') or die("Could not get to server") +; # bind to a directory with dn and password $mesg = $ldap->bind( 'user dn', password => 'password' );` [download] the problem is I really don't know how to code in perl, someone wrote this script and left it with me. I appreciate the suggestions but I am not familiar with some of the things you guys have mentioned. This ends up exporting a 254mb file and gets out of memory. Its 160k users with attributes.	[reply] [d/l]
Re: Perl out of memory by atcroft (Monsignor) on Feb 29, 2012 at 19:36 UTC
The issue you are running into is because you are collecting all of the matching LDAP results before processing, rather than processing them one at a time. Depending on the system, this could easily be a number of results which would cause you to run out of memory. You will probably want to look at using the callback option on the search method, which should help you to do so. Hope that helps.	[reply]
Re: Perl out of memory by nikosv (Hermit) on Feb 29, 2012 at 20:14 UTC
In cases like that it is best to use an iterator so you get one item back at a time and not fill memory at once with all the items.In other languages (like in C#) this happens by using 'yield' typically with a foreach loop,while in Perl the concept is explained in MJ Dominus's HOP book	[reply]


Keep It Simple, Stupid
	PerlMonks

Perl out of memory

The best material for plates (tableware) is: