Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re: Re: Re: Mimic referring URL in LWP?

by pjf (Curate)
on Jun 21, 2002 at 04:00 UTC ( #176221=note: print w/ replies, xml ) Need Help??


in reply to Re: Re: Mimic referring URL in LWP?
in thread Mimic referring URL in LWP?

It seems fairly clear that Google either does not expect or does not want automated scripts to access this particular facility. You may find it worthwhile asking them if they have a more programmer-friendly method of accessing the information that you're after, as that will save a lot of hassle trying to reverse-engineer the whole process.

Judging from the session-expired message, I would judge that google requires you to go through the whole login process to get a valid cookie, as even if these don't expire on the client, they do on their server.

It's very common for sites to stop accepting old cookies, particularly when money's involved. They want to avoid the situation of having cookies stored on a public computer, and a potential third-party accessing the content in question.

If you were dealing with un-encrypted HTTP sessions, then you could use tcpdump/ethereal to log and examine what's happening "under-the-hood". However, since your connection is proceeding via SSL, that's not an easy option.

Paul Fenwick
Perl Training Australia


Comment on Re: Re: Re: Mimic referring URL in LWP?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://176221]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (5)
As of 2014-08-30 14:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (293 votes), past polls