Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Storing Experience for Posterity

by GotToBTru (Chaplain)
on Jun 12, 2014 at 14:34 UTC ( #1089694=CUFP: print w/ replies, xml ) Need Help??

I put the following together to scrape experience, level and writeups off my profile page and store it in a text file on my computer, to record my progress through the Monastery. A scheduled task runs this once a day.

use strict; use warnings; use LWP::Simple; use URI::URL; my $date=`ECHO %DATE:~10,4%%DATE:~4,2%%DATE:~7,2%`; # YYYYMMDD my $url = url('http://perlmonks.org/?node_id=844862'); my $content = get($url); $content =~ s/\cJ//g; $content =~ s/\cM//g; my ($experience, $level, $posts) = ($content =~ /Experience:\D+(\d+).+ Level:.+([A-Z][a-z]+\s+\(\d+\)).+ Writeups:.+>(\d+)</x); open my $ofh, '>>','perl_xp.dat'; printf $ofh "%d,%d,%d,%s\n",$date,$experience,$posts,$level; close($ofh);

There is probably a way to do this in Javascript that could be included in the Free Nodelet, but that's beyond my skill level.

Update

Improved version here.

1 Peter 4:10

Comment on Storing Experience for Posterity
Download Code
Re: Storing Experience for Posterity
by choroba (Abbot) on Jun 12, 2014 at 14:37 UTC
Re: Storing Experience for Posterity
by ww (Bishop) on Jun 12, 2014 at 15:23 UTC

    There are numerous homepages which utilize js to play pranks; there are numerous threads which detail ways to incorporate js into your profile here (or to link to an online but offsite server). I think search terms such as "nodelet" + "js" might be helpful.

    And if that's a false trail, What are Nodelets? may offer some clues.


    check Ln42!

Re: Storing Experience for Posterity
by oiskuu (Pilgrim) on Jun 12, 2014 at 15:50 UTC

    By the way, there's a perfect back-end for this sort of data: rrdtool. You might want to explore that option.

Re: Storing Experience for Posterity
by morgon (Deacon) on Jun 13, 2014 at 08:17 UTC
    I would prefer to generate the timestamp in Perl rather than using backticks. One possibilty (among many):

    my $date = POSIX::strftime "%Y%m%d", localtime;
    And what you totally disregard is error-handling.

    Your get could fail (e.g. no internet-connection), your parsing of the page could fail (the website changed it's layout) and your open could fail (some permission problem maybe).

    You should handle these conditions and report appropriate errors, rather than fail silently and maybe loose the history of your meteoric raise...

      Changes:

      • Added error message if web page get fails
      • Use strftime instead of backtics and Windows date symbols
      • Test to see if all 3 values were found before logging
      • Using Tie::File for easy read-write access to file
      • Replace latest line with new data if Level has not changed
      • Added error message if regex could not find all 3 values
      use strict; use warnings; use LWP::Simple; use URI::URL; use POSIX qw/strftime/; use Tie::File; my $date= strftime "%Y%m%d", localtime; # YYYYMMDD my ($url, $content, $experience, $level, $posts); $url = url('http://perlmonks.org/?node_id=844862'); $content = get($url) or die "Get failure"; $content =~ s/\cJ//g; $content =~ s/\cM//g; ($experience, $level, $posts) = ($content =~ /Experience:\D+(\d+).+ Level:.+([A-Z][a-z]+\s+\(\d+\)).+ Writeups:.+>(\d+)</x); if ($experience && $level && $posts) { tie my @log, 'Tie::File', 'perl_xp.csv'; if ($level eq (split ',',$log[-1])[3] || '') { pop @log; } push @log, join ',',$date,$experience,$posts,$level; untie @log; } else { die "Could not find values on web page"; }
      1 Peter 4:10

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: CUFP [id://1089694]
Approved by igelkott
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (6)
As of 2014-08-30 04:12 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (291 votes), past polls