Re: Hash Search is VERY slow

[johngg]

Further to what choroba said it looks from your very small sample that you might have the same IP/URL combination cropping up many times, with the result that the arrays will get quite large. It might be better to go for an HoH structure counting those combinations.

use strict;
use warnings;

use Data::Dumper;

open my $csvFH, q{<}, \ <<__EOD__ or die $!;
dsd,sdfjk,192.168.102.120,,,"autodiscover-s.outlook.com/",,,
wwfefwe,gfr,192.168.101.208,,,"logmeinrescue.com/",,,
xxd,erer,192.168.100.23,,,"a.n.other.net/",,,
sir,ferutr,192.168.102.120,,,"outlook.office365.com/",,,
thj,,192.168.101.208,,,"logmeinrescue.com/",,'
__EOD__

my %ipURL;
while ( <$csvFH> )
{
    my( $ip, $url ) = ( split m{,} )[ 2, 5 ];
    $ipURL{ $ip }->{ $url } ++;
}

close $csvFH or die $!;

print Data::Dumper->Dumpxs( [ \ %ipURL ], [ qw{ *ipURL } ] );

The output.

%ipURL = (
           '192.168.102.120' => {
                                  '"autodiscover-s.outlook.com/"' => 1
+,
                                  '"outlook.office365.com/"' => 1
                                },
           '192.168.100.23' => {
                                 '"a.n.other.net/"' => 1
                               },
           '192.168.101.208' => {
                                  '"logmeinrescue.com/"' => 2
                                }
         );

I hope this is helpful.