in reply to User agent through Privoxy?
Here is the sample code:
With LogLevel set to 'debug', forward proxing is confirmed from error.log of my local server:use LWP::UserAgent; $ua = LWP::UserAgent->new; $ENV{ http_proxy } ='http://localhost:80'; $ua->env_proxy; print $ua->get( 'http://www.google.com' )->as_string;
[Sat Jun 17 17:30:57 2006] [debug] proxy_http.c(630): Content-Type: te +xt/html
|
---|
Replies are listed 'Best First'. | |
---|---|
Re^2: User agent through Privoxy?
by neilwatson (Priest) on Jun 17, 2006 at 23:48 UTC | |
Neil Watson | [reply] [d/l] |
by Anonymous Monk on Aug 30, 2006 at 20:28 UTC | |
I am having the very same problem. I was trying to run WWW::Mechanize through privoxy, which in turn was forwarding everything to Tor so I could run my scripts anonymously. I set everything up on my Linux machine and was able to confirm that it was working when I used Firefox and Privoxy to check my Tor status at... status When I then used my script, I kept getting error message... 500 Chunked must be last Transfer-Encoding 'identity' Here's my program... <tor_test.pl> </tor_test.pl> The result was... 500 Chunked must be last Transfer-Encoding 'identity' Initially, I thought that the problem was due to the fact that the timeout was not set long enough, but after setting the timeout to a range of values from small to very large, I still get the same problem. I also noticed that when I stepped through the code, the timeout did not seem to have any impact on how quickly the 500 was generated (instantaously). So, I edited the privoxy config file to increase logging. <privoxy config> debug 16 </privoxy config> Now I restart privoxy and "tail -f /var/log/privoxy/logfile" <privoxy logfile>
</privoxy logfile> So, you can see that I am getting a response from the remote server and everything is working!! But for some reason WWW::Mechanize doesn't like the response from Privoxy and issues the 500 error rather than accept the results. I grep'd the perl code and found several references... find /usr/lib/perl5 -exec grep -H 'Transfer-Encoding' '{}' \; ...and this seems to be the line where it is choking... /usr/lib/perl5/vendor_perl/5.8.5/Net/HTTP/Methods.pm: die "Chunked must be last Transfer-Encoding '$te'" I haven't gotten any further on this problem, if someone else can suggest something, I'd be very appreciative!! Edited by planetscape - linkified link and changed pre to code tags | [reply] [d/l] [select] |
In Section
Seekers of Perl Wisdom