Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

web scrapping

by ansh batra (Friar)
on Mar 20, 2013 at 15:20 UTC ( #1024539=perlquestion: print w/ replies, xml ) Need Help??
ansh batra has asked for the wisdom of the Perl Monks concerning the following question:

hi monks
i need to scrap website which have several links to be navigated
now the problem is that those links are ajax binded i.e they dont actually contain any url

<a onclick="new Ajax.Updater('reviews-list', '/rate-and-review/repagin +ate?product_type=home_loans&amp;provider=6', {asynchronous:true, eval +Scripts:true, method:'get', parameters:'page=2'}); return false;" hre +f="#">2</a>
i am using www::mechanize , please tell me how should i proceed

Comment on web scrapping
Download Code
Re: web scrapping
by Corion (Pope) on Mar 20, 2013 at 15:21 UTC
Re: web scrapping
by marto (Bishop) on Mar 20, 2013 at 15:23 UTC

    You read the part of the WWW::Mechnize docuemntation where it tells you that it doesn't support JavaScript, then read the FAQ. You contact the site owner, chances are if they want you to have access to their data they'll provide an API to do so.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1024539]
Approved by davido
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (7)
As of 2015-05-23 00:43 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    In my home, the TV remote control is ...









    Results (465 votes), past polls