Struggling with a data feed/data scraping perl program

[jdlev]

Hey guys,

First of all, thanks to everyone who has helped me in the past! This site rocks the casba!! : )

I'm trying to figure out how to scrap data from yahoo's fantasy football site. It is very complex when looking at the page's source code.

Really, all I want to do, is be able to enter a week, and get the projections for that player. Here is the page I am trying to pull the data from. Once I have the data, I can insert into one of my MySQL databases. Can someone point me in the right direction on how to scrap the players first name, last name, and projected points from this web page:

http://football.fantasysports.yahoo.com/f1/628117/players?&sort=PR&sdir=1&status=ALL&pos=O&stat1=S_PW_11&jsenabled=1&jsenabled=1

Thanks a ton in advance!!! :)

[afoken]

Can someone point me in the right direction on how to scrap the players first name, last name, and projected points from this web page

Search the website for a public API. Ripping data from a heap of HTML junk is the worst way of aquiring data.

Alexander

[2teez]

Hi jdlev,
..Can someone point me in the right direction on how to scrap the players first name, last name, and projected points from this web page..
You may consider using these two modules LWP::UserAgent and HTML::TreeBuilder to get and parse through the web page for the information you want.
I couldn't get the page you gave, but use one to illustrate, like thus:

use warnings;
use strict;
use LWP::UserAgent;
use HTML::TreeBuilder 5 -weak;

my @players_name;

my $url =
'http://facebook-football.fantasysports.yahoo.com/f1/745190/players?st
+atus=A&pos=O&cut_type=9&stat1=S_S_2013&myteam=0&sort=PR&sdir=1';

my $ua      = LWP::UserAgent->new;
my $broswer = $ua->get($url);

if ( $broswer->is_success ) {
    my $tree = HTML::TreeBuilder->new;
    $tree->parse_content( $broswer->decoded_content );
    @players_name = $tree->look_down( 'class' => qr/Nowrap name/ );
}
else {
    die $broswer->status_line;
}

print $_->as_text, $/ for @players_name;

The code above will print all the firstname and lastname of the players.
There are other modules in CPAN or METACPAN that can also do so. So, check in and get started, now you have the right direction you wanted :)

[karlgoethebier]

Perhaps you should take a look at this?

Regards, Karl