http://www.perlmonks.org?node_id=331803

Is an archive of the daily Selected Best Nodes node kept anywhere?


Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"Think for yourself!" - Abigail
Timing (and a little luck) are everything!

Replies are listed 'Best First'.
Re: Selected Best Nodes Archive
by blokhead (Monsignor) on Feb 26, 2004 at 00:04 UTC
    I thought of doing that myself as well. You could of course set up a cron job to just fetch the Selected Best Nodes to a timestamped file. I've set up the following cron job to archive that page into an SQLite database. It then prints out a reputation-sorted list of what it has already archived. That way, after a while, I'll have my own Top 5000 list.
    #!/usr/bin/perl use strict; use warnings; use HTML::TableExtract; use LWP::Simple; use DBI; my $db_file = "best_nodes"; my $pm_site = "http://perlmonks.org/index.pl?node_id=%d"; my $make_table = ! -f $db_file; my $dbh = DBI->connect("dbi:SQLite:dbname=$db_file", "", "") or die "Can't connect to db: $DBI::errstr"; $dbh->do( qq[ create table nodes ( id int unique, title varchar(255), auth_id int, author varchar(255), rep int ) ]) if $make_table; my $html = get( sprintf $pm_site, 328478 ); my $te = HTML::TableExtract->new( headers => [ qw/Node Author Rep/ ], keep_html => 1 ); $te->parse($html); foreach my $row ($te->rows) { my ($node, $author, $rep) = @$row; my ($id) = $node =~ /\?node_id=(\d+)/; my ($auth_id) = $author =~ /\?node_id=(\d+)/; ($rep) = $rep =~ /(\d+)/; my ($title) = $node =~ m{>(.+?)</a>$}; ($author) = $author =~ m{>(.+?)</a>$}; $dbh->do("delete from nodes where id=?", undef, $id); $dbh->do("insert into nodes values (?,?,?,?,?)", undef, $id, $title, $auth_id, $author, $rep); } my $sth = $dbh->prepare( qq[ select id,title,auth_id,author,rep from nodes order by rep desc ]); $sth->execute; open my $fh, ">bestnodes.html" or die; print $fh "<table>\n"; while (my ($id, $title, $auth_id, $author, $rep) = $sth->fetchrow_arra +y) { $id = sprintf $pm_site, $id; $auth_id = sprintf $pm_site, $auth_id; print $fh qq[ <tr><td><a href="$id">$title</a></td> <td><a href="$auth_id">$author</a></td> <td>$rep</td></tr> ]; } print $fh "</table>\n";
    Incidentally, this is my first experience with HTML::TableExtract, and it's just perfect for this job. Maybe I'll post the best nodes archive on my homepage once it gets big enough.

    blokhead

      Believe it or not I actually already coded XML support into the patches I wrote for the Best/Selected nodes. While the other users XML ticker uses this the other pages don't. (Yes internally other users are technically "picked_nodes") But the support is there and if the gods are amenable and I get the tuits to add the rest of the code then youll be able to get an XML feed of this data instead of scraping the HTML for it.

      I meant to release the final patches for the XML stuff right after the normal HTML changes went live, but I guess I got a distracted with other things. Sorry. :-)


      ---
      demerphq

        First they ignore you, then they laugh at you, then they fight you, then you win.
        -- Gandhi


Re: Selected Best Nodes Archive
by demerphq (Chancellor) on Feb 26, 2004 at 00:55 UTC

    Unfortunately not. And its actually refreshed every 6 hours. :-)


    ---
    demerphq

      First they ignore you, then they laugh at you, then they fight you, then you win.
      -- Gandhi


      Oh well. The time stamp on the post would have only led me to the right one if I happened across it before the page had changed which I'll never know now:) Thanks.


      Examine what is said, not who speaks.
      "Efficiency is intelligent laziness." -David Dunham
      "Think for yourself!" - Abigail
      Timing (and a little luck) are everything!
        Go to tilly, click on the write-ups, sort by "highest reputation first". Even tilly hasn't written more than 100 of the best 500 nodes :)

        rdfield

Re: Selected Best Nodes Archive
by belg4mit (Prior) on Feb 25, 2004 at 21:23 UTC
    What would be the point? They're randomly selected.

    --
    I'm not belgian but I play one on TV.

      Well, yes, but so are lottery numbers. And there are people archiving those. Also, judging on the name, I presume the OP to be living in the UK. Which means that there's a 76.34% chance that the OP is either a bookie, his father is a bookie, or the OP has a gambling problem. So, my well-educated guess is that the OP either wants to bet the outcome of the selective best nodes, or wants to run such a bet himself.

      Abigail
      What are those men in white suits doing? No! No! Go away! Go away! I hate needles!

        On my 18th birthday (legal gambling age), I went into a betting shop and made 2 bets. I had to do something "new" to celibrate that special birthday, but I'd previously celebrated my 16th, in a pub with my girlfriend and the landlord, so drinking was passe.

        The first bet was 25p each-way. It came in first and I tripled my stake.

        The second, I bet my winning, 50p on-the-nose. It came in second. I was even.

        That was the last time I bet on a horse. I'm a gambler that never lost any money:)

        I've never yet bought a lottery ticket.

        I've enjoyed fruit machines occasionally, but I bore easily, and with modern ones you don't even have the fun of pulling the handle.


        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "Think for yourself!" - Abigail
        Timing (and a little luck) are everything!

      The "point" was I just wanted to know if it existed.

      The reason is, a few days ago I followed a link (from one of tilly's posts) to the Selected Best Nodes and perused a few of them.

      Today, I encountered a problem that I think one of the nodes I looked at there would answer. The problem is, I cannot remember what the node was called, nor it's author, and searching turned up too many hits for all the keywords I could think of.

      So, I know I could find tilly's post with the link again, and that would give me the day and time. So then, *if* the list was being archived I could have gone to that days list and stood a much better chance of re-locating the post I was looking for.

      As it is, I can't. No matter, I'll find it eventually.


      Examine what is said, not who speaks.
      "Efficiency is intelligent laziness." -David Dunham
      "Think for yourself!" - Abigail
      Timing (and a little luck) are everything!
        I don't suppose you checked your browser's history?

        --
        I'm not belgian but I play one on TV.

      Maybe he wants to ascertain how random your pseudorandom number generator is!

      BTW, I recommend that PerlMonks use lava lamps for random numbers. It's just too darn cool and you can put a webcam on the thing just for kicks. Just make sure you unplug it before you leave the house / server-room. Ok, I wasn't serious :)

      FYI -- I found an odd implementation using digital cameras, long exposures, and CCD Noise based on the lava-lamp work. Check it out.