Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Re: Struggling with a data feed/data scraping perl program

by 2teez (Priest)
on Nov 16, 2013 at 06:27 UTC ( #1062857=note: print w/ replies, xml ) Need Help??


in reply to Struggling with a data feed/data scraping perl program

Hi jdlev,
..Can someone point me in the right direction on how to scrap the players first name, last name, and projected points from this web page..
You may consider using these two modules LWP::UserAgent and HTML::TreeBuilder to get and parse through the web page for the information you want.
I couldn't get the page you gave, but use one to illustrate, like thus:

use warnings; use strict; use LWP::UserAgent; use HTML::TreeBuilder 5 -weak; my @players_name; my $url = 'http://facebook-football.fantasysports.yahoo.com/f1/745190/players?st +atus=A&pos=O&cut_type=9&stat1=S_S_2013&myteam=0&sort=PR&sdir=1'; my $ua = LWP::UserAgent->new; my $broswer = $ua->get($url); if ( $broswer->is_success ) { my $tree = HTML::TreeBuilder->new; $tree->parse_content( $broswer->decoded_content ); @players_name = $tree->look_down( 'class' => qr/Nowrap name/ ); } else { die $broswer->status_line; } print $_->as_text, $/ for @players_name;
The code above will print all the firstname and lastname of the players.
There are other modules in CPAN or METACPAN that can also do so. So, check in and get started, now you have the right direction you wanted :)

If you tell me, I'll forget.
If you show me, I'll remember.
if you involve me, I'll understand.
--- Author unknown to me


Comment on Re: Struggling with a data feed/data scraping perl program
Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1062857]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (9)
As of 2014-09-19 19:53 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (145 votes), past polls