Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"

Storing Experience for Posterity

by GotToBTru (Prior)
on Jun 12, 2014 at 14:34 UTC ( #1089694=CUFP: print w/replies, xml ) Need Help??

I put the following together to scrape experience, level and writeups off my profile page and store it in a text file on my computer, to record my progress through the Monastery. A scheduled task runs this once a day.

use strict; use warnings; use LWP::Simple; use URI::URL; my $date=`ECHO %DATE:~10,4%%DATE:~4,2%%DATE:~7,2%`; # YYYYMMDD my $url = url(''); my $content = get($url); $content =~ s/\cJ//g; $content =~ s/\cM//g; my ($experience, $level, $posts) = ($content =~ /Experience:\D+(\d+).+ Level:.+([A-Z][a-z]+\s+\(\d+\)).+ Writeups:.+>(\d+)</x); open my $ofh, '>>','perl_xp.dat'; printf $ofh "%d,%d,%d,%s\n",$date,$experience,$posts,$level; close($ofh);

There is probably a way to do this in Javascript that could be included in the Free Nodelet, but that's beyond my skill level.


Improved version here.

1 Peter 4:10

Replies are listed 'Best First'.
Re: Storing Experience for Posterity
by choroba (Chancellor) on Jun 12, 2014 at 14:37 UTC
Re: Storing Experience for Posterity
by oiskuu (Hermit) on Jun 12, 2014 at 15:50 UTC

    By the way, there's a perfect back-end for this sort of data: rrdtool. You might want to explore that option.

Re: Storing Experience for Posterity
by morgon (Curate) on Jun 13, 2014 at 08:17 UTC
    I would prefer to generate the timestamp in Perl rather than using backticks. One possibilty (among many):

    my $date = POSIX::strftime "%Y%m%d", localtime;
    And what you totally disregard is error-handling.

    Your get could fail (e.g. no internet-connection), your parsing of the page could fail (the website changed it's layout) and your open could fail (some permission problem maybe).

    You should handle these conditions and report appropriate errors, rather than fail silently and maybe loose the history of your meteoric raise...


      • Added error message if web page get fails
      • Use strftime instead of backtics and Windows date symbols
      • Test to see if all 3 values were found before logging
      • Using Tie::File for easy read-write access to file
      • Replace latest line with new data if Level has not changed
      • Added error message if regex could not find all 3 values
      • Replace 2 s/// with 1 tr///
      • Add math and reporting of anticipated level up
      use strict; use warnings; use LWP::Simple; use URI::URL; use POSIX qw/strftime/; use Tie::File; use DateTime; my $date= strftime "%Y%m%d", localtime; # YYYYMMDD my ($url, $content, $experience, $level, $posts); $url = url(''); $content = get($url) or die "Get failure"; $content =~ tr/\cJ\cM//d; ($experience, $level, $posts) = ($content =~ /Experience:\D+(\d+).+ Level:.+([A-Z][a-z]+\s+\(\d+\)).+ Writeups:.+>(\d+)</x); die unless ($experience && $level && $posts); my ($start, $startdt, $startexp, $today); tie my @log, 'Tie::File', 'perl_xp.csv'; ($start, $startexp) = (split ',', $log[-2])[0,1]; if ($level eq (split ',',$log[-1])[3] || '') { pop @log; } push @log, join ',',$date,$experience,$posts,$level; untie @log; my ($year,$month,$day) = $start =~ m/(\d{4})(\d\d)(\d\d)/; $startdt = DateTime->new(year => $year, month=> $month, day => $day); ($year,$month,$day) = $date =~ m/(\d{4})(\d\d)(\d\d)/; $today = DateTime->new(year => $year, month=> $month, day => $day); my $timeflies = $startdt->delta_days($today)->{days}; my $exprate = ($experience-$startexp)/$timeflies; printf "%d days, %d posts, %d points, %.2f points per day\n",$timeflie +s,$posts,$experience,$exprate; my $nextlevel = 3000; my $leveldays = ($nextlevel - $experience)/$exprate; my $leveldate = $today->add(days => $leveldays); printf "Level up in %d days, on %s\n", $leveldays, $leveldate->mdy();
      1 Peter 4:10
Re: Storing Experience for Posterity
by ww (Archbishop) on Jun 12, 2014 at 15:23 UTC

    There are numerous homepages which utilize js to play pranks; there are numerous threads which detail ways to incorporate js into your profile here (or to link to an online but offsite server). I think search terms such as "nodelet" + "js" might be helpful.

    And if that's a false trail, What are Nodelets? may offer some clues.

    check Ln42!

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: CUFP [id://1089694]
Approved by igelkott
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (5)
As of 2017-08-19 21:08 GMT
Find Nodes?
    Voting Booth?
    Who is your favorite scientist and why?

    Results (312 votes). Check out past polls.