Way to predict Array Memory Size for returned LDAP Searches?

[ghenry]

Dear Master Monks,

In Catalyst, LDAP User Auto_complete and arrays with grep I mentioned the amount of users in a Directory at around 500,000 - 1,000,000. This is not uncommon and I have seen up to 1.5 - 2 Million.

So my question is, if during an LDAP search for the auto_complete in the above node, the directory returns say 250,000 users that begin with the letter "g", however unlikely, and the usernames are say a max of 6 letters long, is it possible to determine the rough size of the Array the holds these results?

Kind of 6 x 1 byte = 1 username, x 500,000? Plus what ever Net::LDAP uses to store the results?

2-3MB to be safe?

It's just for a rough idea.

Not sure if I am even on the right track.

Many thanks,
Gavin.

[g0n]

To try and cover the overhead from the Net::LDAP::Entry objects I mocked this up:

use strict;
use warnings;
use Net::LDAP;
use Net::LDAP::Entry;

my @entrylist;

for (1..500000)
{
    my $entry = Net::LDAP::Entry->new();
    $entry->dn("cn=name$_,ou=test,dc=test,dc=uk");
    $entry->add(cn=>"name$_");
    push @entrylist,$entry;
    print "$_\n";
}
while (<STDIN>){}

which results in the process using 12MB on my (wintel) box. You might want to try it on linux.

[codeacrobat]

Unfortunately stringlength and bytecount are not the same. You can use Devel::Size for estimations on datastructure sizes.
The example shows "abcdef" take 31 bytes of memory.

$ perl -MDevel::Size+'total_size' -e 'print total_size "abcdef"'
31

[wazoox]

Well I've tried this :

for ( 0..500000) {
push @aa, "abcdef"
}
print "waiting";
$t=<STDIN>

And the perl interpreter gobbles up 22MB. This:

print "waiting";
$t=<STDIN>

uses only 1.2MB. So I'd say the hash uses roughly 4MB for 100000 entries (40KB per line).

[ghenry]

:-)

The easiest way is to mock up a test!

Common sense, but glad you pointed it out.

Thanks,
Gavin.