Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW

Re: Perl+PostgreSQL+GeoIP = Awesome!

by erix (Prior)
on Nov 21, 2018 at 14:52 UTC ( #1226131=note: print w/replies, xml ) Need Help??

in reply to Perl+PostgreSQL+GeoIP = Awesome!

NICE indeed. But TIMTOWTDI and here's one more (not better, just shorter and how I would probably have done it)

(bash + perl, Pg connects via PG* envvars)

wget --timestamping +/ wget --timestamping +/GeoIPv6.csv.gz echo " create table geoip ( netblock cidr not null, country_code text not null, country_name text not null, constraint geoip_pk primary key (netblock) );"| psql \ && ( unzip -p ; gunzip -c GeoIPv6.csv.gz ) \ | perl -MNet::CIDR -ne ' chomp; my @arr = split( /\"\,\ ?\"/, $_ ); my @cidr = Net::CIDR::range2cidr(sanitize($arr[0]) . "-" . sani +tize($arr[1])); for my $subcidr (@cidr) { print $subcidr, "\t", sanitize($arr[4]), "\t", sanitize($arr +[5]), "\n"; } sub sanitize { my ($val) = @_; $val =~ s/^\"//; $val =~ s/\"$//; return $val; } ' | psql -c "copy geoip from stdin with (format csv, header false, del +imiter E'\t');"

(I get 326604 rows (in 12 sec))

Replies are listed 'Best First'.
Re^2: Perl+PostgreSQL+GeoIP = Awesome!
by cavac (Curate) on Nov 23, 2018 at 10:00 UTC

    As just noted in the update/edit in my thread starter, i just found out that we need to switch to the new GeoLite2 databases, because the ones i have used here are no longer supported. I'll post a new version in CUFP after i get it working and fully tested.

    Otherwise, nice use of the psql COPY command ;-)

    perl -e 'use MIME::Base64; print decode_base64("4pmsIE5ldmVyIGdvbm5hIGdpdmUgeW91IHVwCiAgTmV2ZXIgZ29ubmEgbGV0IHlvdSBkb3duLi4uIOKZqwo=");'

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1226131]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (10)
As of 2021-12-02 11:30 GMT
Find Nodes?
    Voting Booth?
    R or B?

    Results (18 votes). Check out past polls.