Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

How to parse a file with emails and IP's and output a sorted list with a total count

by mitzyc (Initiate)
on Jul 28, 2016 at 22:39 UTC ( #1168769=perlquestion: print w/replies, xml ) Need Help??

mitzyc has asked for the wisdom of the Perl Monks concerning the following question:

I really need help. I've been trying to code this myself for over 3 hours now, and nothing I do works.

I have a vpopmail file like this:

Jul 28 13:42:27 mail vpopmail[47985]: vchkpw-smtp: (PLAIN) login success joe@example.com:192.168.250.251
Jul 28 13:42:28 mail vpopmail[47992]: vchkpw-smtp: (PLAIN) login success joe@example.com:192.168.5.23
Jul 28 13:42:29 mail vpopmail[47994]: vchkpw-smtp: (PLAIN) login success sally@example.com:192.168.10.28
Jul 28 13:42:27 mail vpopmail[47985]: vchkpw-smtp: (PLAIN) login success fred@example.com:192.168.8.8
Jul 28 13:42:28 mail vpopmail[47992]: vchkpw-smtp: (PLAIN) login success joe@example.com:192.168.5.23
Jul 28 13:42:29 mail vpopmail[47994]: vchkpw-smtp: (PLAIN) login success harry@example.com:192.168.10.5

I'm trying to write a routine that reads in this file, sorts the list, and gives a count:

joe@example.com
=> 192.168.250.251 (1)
=> 192.168.5.23 (2)
joe total: 2 unique IP's

sally@example.com
=> 192.168.10.28 (1)
sally total: 1 unique IP's

I know basic perl. But object oriented programming is something I'm trying to learn, but I just can't seem to get it.

If I had a working example written like this.. I could understand it better, and then be able to finally get how it works when I see the code.

The output format doesn't have to be exact.

I'm hunting to see how many unique IP addresses a person is successfully logging in from and output the counts

I tried playing with XML::Simple, and gave up when I got this:

Can't call method "address" on unblessed reference

#!/usr/bin/perl use XML::Simple; my $xml = new XML::Simple; my $sender = { 'address' => 'joe@example.com', 'ips' => ["12.52","13.53","14.54.55"], }; # add another IP to an existing user $sender->address->'joe@example.com' {; ->ips = "16.70.71"; };

I know I'm screwing up syntax, and all the examples I've been trying to incorporate never seem to work. I'd have a longer code sample if I had it, but I kept erasing my code and trying again, and again, and again.

This led me to Perl Monks. Please help if you can. :)

- Mitzy

  • Comment on How to parse a file with emails and IP's and output a sorted list with a total count
  • Download Code

Replies are listed 'Best First'.
Re: How to parse a file with emails and IP's and output a sorted list with a total count
by Paladin (Vicar) on Jul 28, 2016 at 23:13 UTC
    I'm not sure why you are trying XML::Simple as there doesn't seem to be any XML involved. For data storage, you seem to need a hash of hashes. I'll assume you can get the IP and email address from the file.
    my %users; foreach my $line (<$file>) { my ($email, $ip) = parse_line($line); $users{$email}{$ip}++; }
    Then you'll have a hash that looks something like:
    %users = ( 'joe@example.com' => { '192.168.10.24' => 2, '192.168.10.28' => 1 } );

      Yes! I can get the IP and the email.

      That little bit is enough to get me started. I really didn't know how to load and reference a hash with values.

      ...and no, I guess I didn't need XML at all.

      Thank you very much. This will give me a greater appreciation of how this works and a solid footing to start from.

      Thank you so much.

Re: How to parse a file with emails and IP's and output a sorted list with a total count
by NetWallah (Canon) on Jul 29, 2016 at 00:22 UTC
    Obligatory one-liner:
    >perl -ane "my ($email,$ip)=split(/:/,$F[9]); push @{$h{$email}},$ip}{ +print qq|$_ => |,join(qq|,|,@{$h{$_}}),qq|\n| for sort keys %h" YOUR- +FILE.txt #--- Output--- fred@example.com => 192.168.8.8 harry@example.com => 192.168.10.5 joe@example.com => 192.168.250.251,192.168.5.23,192.168.5.23 sally@example.com => 192.168.10.28

            "Software interprets lawyers as damage, and routes around them" - Larry Wall

Re: How to parse a file with emails and IP's and output a sorted list with a total count
by Mandrake (Chaplain) on Jul 29, 2016 at 02:03 UTC
    You may try something like the below.
    #!/bin/perl -w use strict; my %hash; while (<DATA>) { $hash{$1}{$2}++ if ($_ =~ /.+success\s+(.+)\:(.+)/) ; } for my $user (keys %hash) { print $user."\n"; print "==>".$_."(".$hash{$user}{$_}.")\n" for (keys %{$hash{$user}}) +; print $1." total:".scalar(keys %{$hash{$user}})." Unique IP's \n\n" + if ($user =~ /(.+)\@/); } __DATA__ Jul 28 13:42:27 mail vpopmail[47985]: vchkpw-smtp: (PLAIN) login succe +ss joe@example.com:192.168.250.251 Jul 28 13:42:28 mail vpopmail[47992]: vchkpw-smtp: (PLAIN) login succe +ss joe@example.com:192.168.5.23 Jul 28 13:42:29 mail vpopmail[47994]: vchkpw-smtp: (PLAIN) login succe +ss sally@example.com:192.168.10.28 Jul 28 13:42:27 mail vpopmail[47985]: vchkpw-smtp: (PLAIN) login succe +ss fred@example.com:192.168.8.8 Jul 28 13:42:28 mail vpopmail[47992]: vchkpw-smtp: (PLAIN) login succe +ss joe@example.com:192.168.5.23 Jul 28 13:42:29 mail vpopmail[47994]: vchkpw-smtp: (PLAIN) login succe +ss harry@example.com:192.168.10.5
Re: How to parse a file with emails and IP's and output a sorted list with a total count
by Anonymous Monk on Jul 28, 2016 at 23:20 UTC
    #!/usr/bin/perl -l # http://perlmonks.org/?node_id=1168769 use strict; use warnings; my %ips; while(<DATA>) { /.* (\S+):(\S+)$/ or next; $ips{$1}{$2}++; } for my $who ( sort keys %ips ) { print $who; my @ips; for my $ip ( @ips = sort keys %{ $ips{$who} } ) { print "=> $ip ($ips{$who}{$ip})"; } print $who =~ s/\@.*//r, " total: ", scalar @ips, " unique ip's\n"; } __DATA__ Jul 28 13:42:27 mail vpopmail[47985]: vchkpw-smtp: (PLAIN) login succe +ss joe@example.com:192.168.250.251 Jul 28 13:42:28 mail vpopmail[47992]: vchkpw-smtp: (PLAIN) login succe +ss joe@example.com:192.168.5.23 Jul 28 13:42:29 mail vpopmail[47994]: vchkpw-smtp: (PLAIN) login succe +ss sally@example.com:192.168.10.28 Jul 28 13:42:27 mail vpopmail[47985]: vchkpw-smtp: (PLAIN) login succe +ss fred@example.com:192.168.8.8 Jul 28 13:42:28 mail vpopmail[47992]: vchkpw-smtp: (PLAIN) login succe +ss joe@example.com:192.168.5.23 Jul 28 13:42:29 mail vpopmail[47994]: vchkpw-smtp: (PLAIN) login succe +ss harry@example.com:192.168.10.5

      Bareword found where operator expected at line 24, near "s/\@.*//r"

      print $who =~ s/\@.*//r, " total: ", scalar @ips, " unique ip's\n";

      I took out the single quote, and still same thing. What's the bare word it is complaining about. I think it doesn't like $who in that case..

        You probably have an older perl that does not understand the /r

        Here's an alternate...

        #!/usr/bin/perl -l # http://perlmonks.org/?node_id=1168769 use strict; use warnings; my %ips; while(<DATA>) { /.* (\S+):(\S+)$/ or next; $ips{$1}{$2}++; } for my $who ( sort keys %ips ) { print $who; my @ips; for my $ip ( @ips = sort keys %{ $ips{$who} } ) { print "=> $ip ($ips{$who}{$ip})"; } print $who =~ /(.*?)\@/, " total: ", scalar @ips, " unique ip's\n"; } __DATA__ Jul 28 13:42:27 mail vpopmail[47985]: vchkpw-smtp: (PLAIN) login succe +ss joe@example.com:192.168.250.251 Jul 28 13:42:28 mail vpopmail[47992]: vchkpw-smtp: (PLAIN) login succe +ss joe@example.com:192.168.5.23 Jul 28 13:42:29 mail vpopmail[47994]: vchkpw-smtp: (PLAIN) login succe +ss sally@example.com:192.168.10.28 Jul 28 13:42:27 mail vpopmail[47985]: vchkpw-smtp: (PLAIN) login succe +ss fred@example.com:192.168.8.8 Jul 28 13:42:28 mail vpopmail[47992]: vchkpw-smtp: (PLAIN) login succe +ss joe@example.com:192.168.5.23 Jul 28 13:42:29 mail vpopmail[47994]: vchkpw-smtp: (PLAIN) login succe +ss harry@example.com:192.168.10.5

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1168769]
Approved by Paladin
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (2)
As of 2019-12-07 03:37 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Strict and warnings: which comes first?



    Results (160 votes). Check out past polls.

    Notices?