Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re^3: Fetching URL

by Corion (Pope)
on Oct 04, 2011 at 10:04 UTC ( #929491=note: print w/ replies, xml ) Need Help??


in reply to Re^2: Fetching URL
in thread Fetching URL

You will need to send exactly the same information that your browser sends. Find out what the browser sends, for example using the Mozilla Live Headers extension or using Wireshark, and then replicate that data from your script.


Comment on Re^3: Fetching URL
Re^4: Fetching URL
by Shaveta_Chawla (Acolyte) on Oct 04, 2011 at 10:28 UTC
    Actually the Content length, content header and cookies all the values changes with every request.. I am not able to generate the same header information. Can you guide me on this.

      Extract the cookies from the previous request, and generate the content header and content length headers. Also see WWW::Mechanize, which has solved these problems.

        But this is what the problem is... The cookie information also changes with every request. The code on which i am working is like all the pages are dependent on first page. so the cookie information is same for all the request according to my code. but i need to generate different cookie information with every request which should be same as the source is generating.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://929491]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (5)
As of 2014-09-21 12:42 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (168 votes), past polls