Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

Re: Struggling with a data feed/data scraping perl program

by 2teez (Priest)
on Nov 16, 2013 at 06:27 UTC ( #1062857=note: print w/ replies, xml ) Need Help??

in reply to Struggling with a data feed/data scraping perl program

Hi jdlev,
..Can someone point me in the right direction on how to scrap the players first name, last name, and projected points from this web page..
You may consider using these two modules LWP::UserAgent and HTML::TreeBuilder to get and parse through the web page for the information you want.
I couldn't get the page you gave, but use one to illustrate, like thus:

use warnings; use strict; use LWP::UserAgent; use HTML::TreeBuilder 5 -weak; my @players_name; my $url = ' +atus=A&pos=O&cut_type=9&stat1=S_S_2013&myteam=0&sort=PR&sdir=1'; my $ua = LWP::UserAgent->new; my $broswer = $ua->get($url); if ( $broswer->is_success ) { my $tree = HTML::TreeBuilder->new; $tree->parse_content( $broswer->decoded_content ); @players_name = $tree->look_down( 'class' => qr/Nowrap name/ ); } else { die $broswer->status_line; } print $_->as_text, $/ for @players_name;
The code above will print all the firstname and lastname of the players.
There are other modules in CPAN or METACPAN that can also do so. So, check in and get started, now you have the right direction you wanted :)

If you tell me, I'll forget.
If you show me, I'll remember.
if you involve me, I'll understand.
--- Author unknown to me

Comment on Re: Struggling with a data feed/data scraping perl program
Download Code

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1062857]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (11)
As of 2014-12-25 06:01 GMT
Find Nodes?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?

    Results (159 votes), past polls