Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re: Cookies, JavaScript and User Agent Problems

by planetscape (Canon)
on Oct 25, 2011 at 00:47 UTC ( #933510=note: print w/ replies, xml ) Need Help??


in reply to Cookies, JavaScript and User Agent Problems

IMHO, you need to see what's actually being sent between an actual browser and the site you're scraping. I use:

to find out this kind of information, then code accordingly.

HTH,

planetscape


Comment on Re: Cookies, JavaScript and User Agent Problems

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://933510]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (12)
As of 2015-07-04 16:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (60 votes), past polls