Tired of "Perl is dead" FUD ?

Yeah, me too. But I'm a capitalist, so I like to see what people are writing checks for. As I'm fed up with the various unsubstantiated claims, I've whipped up a little graphic that will hopefully cheer you up. I'll try to keep it updated regularly.

If nothing else, it'll give us all something to watch as we're overtaken by our Ruby and Python overlords.

Update:
Hmm, based on my weblogs, this seems to have generated a little buzz. Anyway, after a night to sleep on it, I've made some minor changes to better indicate my motivation. I'm not just trying to pimp Perl. I really want to know what the situation is, without all the FUD and religious debate. After all, the chart isn't all good news for Perl: the upticks in Ruby and Python in the past 10 months are important to watch when plotting a career path or making project decisions. It'll be interesting to see the direction of the trend (thnx to perrin for the indeed.com link below).

I've been asked to provide the code, so here it is; be advised the DICE screen scrape is pretty brute force, and highly subject to any CGI or format changes DICE might apply in future; and the FTP code has been altered:

Update 2: posted revised code that produces a better looking chart.

Update 3: yet another code revision:

added bar for "titles only" search to indicate more focused job postings
added link to DICE

use Socket;
use Net::FTP;
use LWP::Simple;
use DBI;

use strict;
use warnings;

my %queries = (
'Perl',
[
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=perl&Ntx=mode+
+matchall&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUNTR
+Y=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.78715899%2C-84.39164034
+&TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50&x=3
+3&y=14',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=perl+and+%28ph
+p+or+python+or+ruby%29&Ntx=mode+matchboolean&x=50&y=8&AREA_CODES=&AC_
+COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT
+_PROV=0&METRO_AREA=33.78715899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORT
+SPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=perl&Ntx=mode+
+matchall&SEARCH_TITLE_ONLY=1&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RADIU
+S=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.787
+15899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=10
+&NUM_PER_PAGE=30&x=40&y=8',
],

'PHP',
[
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=php&Ntx=mode+m
+atchall&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUNTRY
+=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.78715899%2C-84.39164034&
+TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50&x=33
+&y=14',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=php+and+%28per
+l+or+python+or+ruby%29&Ntx=mode+matchboolean&x=50&y=8&AREA_CODES=&AC_
+COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT
+_PROV=0&METRO_AREA=33.78715899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORT
+SPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=php&Ntx=mode+m
+atchall&SEARCH_TITLE_ONLY=1&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RADIUS
+=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.7871
+5899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=10&
+NUM_PER_PAGE=30&x=40&y=8',
],

'Python',
[
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=python&Ntx=mod
+e+matchall&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUN
+TRY=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.78715899%2C-84.391640
+34&TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50&x
+=33&y=14',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=python+and+%28
+php+or+perl+or+ruby%29&Ntx=mode+matchboolean&x=50&y=8&AREA_CODES=&AC_
+COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT
+_PROV=0&METRO_AREA=33.78715899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORT
+SPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=python&Ntx=mod
+e+matchall&SEARCH_TITLE_ONLY=1&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RAD
+IUS=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.7
+8715899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=
+10&NUM_PER_PAGE=30&x=40&y=8',
],

'Ruby',
[
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=ruby&Ntx=mode+
+matchall&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUNTR
+Y=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.78715899%2C-84.39164034
+&TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50&x=3
+3&y=14',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=ruby+and+%28ph
+p+or+python+or+perl%29&Ntx=mode+matchboolean&x=50&y=8&AREA_CODES=&AC_
+COUNTRY=1525&WHERE=&RADIUS=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT
+_PROV=0&METRO_AREA=33.78715899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORT
+SPEC=0&FRMT=0&DAYSBACK=10&NUM_PER_PAGE=50',
'http://seeker.dice.com/jobsearch/servlet/JobSearch?LOCATION_OPTION=2&
+N=0&Hf=0&Ntk=JobSearchRanking&op=300&values=&FREE_TEXT=ruby&Ntx=mode+
+matchall&SEARCH_TITLE_ONLY=1&AREA_CODES=&AC_COUNTRY=1525&WHERE=&RADIU
+S=64.37376&ZC_COUNTRY=1525&COUNTRY=1525&STAT_PROV=0&METRO_AREA=33.787
+15899%2C-84.39164034&TRAVEL=0&TAXTERM=0&SORTSPEC=0&FRMT=0&DAYSBACK=10
+&NUM_PER_PAGE=30&x=40&y=8',
],
);

my %baseline = (qw/
Perl 4150
PHP 1073
Python 665
Ruby 232 
/);

my @ltime = split /\s+/, scalar localtime();
my $ltime = join('-', $ltime[-1], $ltime[1], $ltime[2]);

my %results = ();
my ($k, $url);

my $localin = 0;
my $localout = 0;
foreach (@ARGV) {
    $localin = 1, next if ($_ eq '-f');
    $localout = 1 if ($_ eq '-n');
}

if ($localin) {
    open INF, "langjobs.csv" or die $!;
    my @jobs = <INF>;
    close INF;
    chomp $jobs[-1];
    pop @jobs and
    chomp $jobs[-1]
        while (@jobs && ($jobs[-1] eq ''));
    ($ltime, $results{Perl}[0], $results{Perl}[1], $results{Perl}[2],
        $results{PHP}[0], $results{PHP}[1], $results{PHP}[2], 
        $results{Python}[0], $results{Python}[1], $results{Python}[2],
+ 
        $results{Ruby}[0], $results{Ruby}[1], $results{Ruby}[2]) = 
        split ';', $jobs[-1]; 
}
else {
    $results{$k} = [ 
        fetchCount("$k only", $url->[0]), 
        fetchCount("$k plus", $url->[1]),
        fetchCount("$k titles", $url->[2]),
        ]
        while (($k, $url) = each %queries);
#
#    now append the results to the CSV file
#
    open INF, ">>langjobs.csv" or die $!;
    print INF join(';', $ltime, @{$results{Perl}}, @{$results{PHP}}, @
+{$results{Python}}, @{$results{Ruby}}), "\n";
    close INF;
}

my $dbh = DBI->connect('dbi:Chart:', undef, undef) 
    or die "Can't connect for charting: " . $DBI::errstr;

$dbh->do('create chart currjobs (language varchar(60), total integer)'
+);
$dbh->do('create chart basejobs (language varchar(60), total integer)'
+);
$dbh->do('create chart titles_only (language varchar(60), total intege
+r)');
$dbh->do('create chart mixedjobs (language varchar(60), total integer)
+');
my $sth = $dbh->prepare('insert into currjobs values(?, ?)') or die $d
+bh->errstr;
$sth->execute($_, $results{$_}[0])
    foreach (sort keys %results);

$sth = $dbh->prepare('insert into basejobs values(?, ?)') or die $dbh-
+>errstr;
$sth->execute($_, $baseline{$_})
    foreach (sort keys %results);

$sth = $dbh->prepare('insert into titles_only values(?, ?)') or die $d
+bh->errstr;
$sth->execute($_, $results{$_}[2])
    foreach (sort keys %results);

$sth = $dbh->prepare('insert into mixedjobs values(?, ?)') or die $dbh
+->errstr;
$sth->execute($_, $results{$_}[1])
    foreach (sort keys %results);

$sth = $dbh->prepare("select image, imagemap from 
    (select barchart from currjobs 
    where COLORS IN ('gold', 'lblue', 'lgreen', 'lred')
    AND SHOWVALUES=1) current,
    (select barchart from basejobs 
    where COLORS IN ('gray', 'gray', 'gray', 'gray')
    AND SHOWVALUES=1) baseline,
    (select barchart from titles_only
    where COLORS IN ('lorange', 'blue', 'green', 'red')
    AND SHOWVALUES=1) titles_only,
    (select barchart from mixedjobs 
    where COLORS IN ('orange', 'dblue', 'dgreen',    'dred')
    AND SHOWVALUES=1) mixed
where WIDTH=550
AND HEIGHT=450
AND X_AXIS=' '
AND Y_AXIS='Total Jobs'
AND TITLE='DICE Results For $ltime (10 Days/No restrict/all locs)'
AND FORMAT='PNG'
AND SHOWGRID=1
AND MAPNAME='langjobs'
AND MAPTYPE='HTML'
AND X_ORIENT='HORIZONTAL'
AND KEEPORIGIN=1
")
or die $dbh->errstr;
$sth->execute;
my $row = $sth->fetchrow_arrayref;

open HTMLF , ">langjobs.html" or die $!;
print HTMLF <<"EOHTML";
<html>
<head><title>Dynamic Language Jobs Barometer for $ltime</title></head>
<body>
<table border=0>
<tr><td align=left>
<i>The rumors of my death have been greatly exaggerated.</i></td></tr>
<tr><td align=right>- Mark Twain</td></tr>
</table>
<p>
<h2>Dynamic Languages Jobs Barometer</h2>
<h3>Total Jobs For Dynamic Languages on <a href='http://www.dice.com'>
+DICE</a> for $ltime</h3>
Each language has 
<ul>
<li>a light bar for its overall total, 
<li>a gray bar for a baseline value from Nov. 15, 2006
<li>a medium bar for titles-only search results
<li>a dark bar for listings which overlap any of the other languages.
</ul>
<p>
<img src='langjobs.png' usemap='#langjobs' title='Barchart of job list
+ings by language' border=0>
<p>

A comparative trendline is available at
<a href='http://www.indeed.com/jobtrends?q=perl%2Cpython%2Cphp%2Cruby&
+l='>indeed.com</a>
<i>(thanks to <a href='http://www.perlmonks.com/?node_id=104919'>perri
+n</a> for the link)</i>.
<p>
<p>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<a href='http://www.presicie
+nt.com'>[ Presicient Home ]</a>

<!-- BEGIN AREAMAP -->
$row->[1]

</body>
</html>

EOHTML
close HTMLF;

open IMAGEF, ">langjobs.png" or die $!;
binmode IMAGEF;
print IMAGEF $row->[0];
close IMAGEF;

unless ($localout) {
#
#    now ftp it up
#
my $ftp = Net::FTP->new('ftp.somedomain.com', Passive => 1)
    or die "Cannot connect: $@";

$ftp->login('userid','password')
    or die "Cannot login ", $ftp->message;

print "Connected, sending HTML\n";
$ftp->ascii();
$ftp->put('langjobs.html');
print "Sending image\n";
$ftp->binary();
$ftp->put('langjobs.png');
$ftp->quit;
}

sub fetchCount {
    my ($k, $url) = @_;
    sleep 3;
    my $result = get $url;
    warn "** No result for $k" and return 0
        unless $result;

    $result=~tr/\n/ /;
    $result=~s/\s+/ /gs;
    my ($count) = ($result=~/\b1\s*-\s*\d+\s+of\s+(\d+)\s+jobs/);
    warn "Can't find count for $k\n" and return 0
        unless defined $count;
    print "***$k: $count\n";
    return $count;
}
[download]

Perl Contrarian & SQL fanboy

Comment on Tired of "Perl is dead" FUD ? Download Code

Replies are listed 'Best First'.

Re: Tired of "Perl is dead" FUD ?
by perrin (Chancellor) on Sep 14, 2007 at 02:34 UTC