Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

LWP::UserAgent : setting max_redirect to 0 yields 'Client-Warning: Redirect loop detected

by bliako (Sexton)
on Mar 13, 2018 at 18:38 UTC ( #1210840=perlquestion: print w/replies, xml ) Need Help??
bliako has asked for the wisdom of the Perl Monks concerning the following question:

Sisters and Brothers,

I am trying to tell LWP::UserAgent (UA) not to follow any redirect (for example, a server response of '302 Found').

I would like UA to simply make a request and then just to return the HTTP::Response back to me.

So, I create UA as thus:

my $ua = LWP::UserAgent->new( 'requests_redirectable' => [], 'max_redirect' => 0, );

With the above, I can see in my logs that indeed UA does not follow the redirect but now I get a warning:

Client-Warning: Redirect loop detected (max_redirect = 0)

which is produced in this check in (sub request):

if ($response->redirects >= $self->{max_redirect}) { $response->header("Client-Warning" => "Redirect loop detected (max_redirect = $self->{max_re +direct})" ); return $response; }

The redirect count is zero, the max_redirects is zero but that does not mean a redirect loop has been detected.

Or am I confused (just in this particular issue)?



  • Comment on LWP::UserAgent : setting max_redirect to 0 yields 'Client-Warning: Redirect loop detected
  • Select or Download Code

Replies are listed 'Best First'.
Re: LWP::UserAgent : setting max_redirect to 0 yields 'Client-Warning: Redirect loop detected
by genio (Beadle) on Mar 13, 2018 at 20:43 UTC


    There seems to be a bit of confusion, but that's not your fault. The documentation is probably a bit lacking in this area.

    my $ua = LWP::UserAgent->new(requests_redirectable => []);

    There shouldn't be a need to worry about setting how many redirects to allow if you're already not allowing redirects on those types of requests.

    Can you give that a try and maybe take a crack at helping fix the documentation lack in this area so nobody else falls into the same problem? We're always happy to have documentation patches!

    -- Chase

      Thanks for your reply,

      I can confirm that by just setting ONLY requests_redirectable => [], LWP::UserAgent does not follow redirects and does not warn about a loop detected.

      So, I guess you are asking me to write a documentation patch where to say:

      "If you do not want LWP::UserAgent to follow redirects for a certain request then prepare a new array with only the request names you wish redirects to be followed and insert it via requests_redirectable => $array_of_requests_to_follow_redirect . If you do not want LWP::UserAgent to follow ANY request then set this to an empty array ref."

      That's no problem, I can do that.

      As a side remark (but quite important imo):

      when looking at the code of LWP::UserAgent at the place where it gives the loop warning (just search for 'loop') one can see that when the alleged loop is detected, the warning is inserted in the headers AND the sub returns immediately thus omitting code for running the handlers or normalising (absolutising) the Location url from the returned headers and lots of other stuff.

      So that "warning" is not that innocent and people should care about it. Maybe it should be clarified. I will agree with the "Anonymous Monk" (Mar 13, 2018 at 20:21 UTC) who considers this as a bug because the logic (with my limited understanding of that code) should be to make a request, increment the redirect count (its name probably is confusing because there is no redirect yet but just one - and maybe the only - request) and then check if that count exceeds the max_redirect limit (and not check if it is equal to it too).

      In view of the above, a second documentation patch could go like this:

      "Redirect loop detection is a bit petulant and one should not upset it further by demanding zero max_redirect. Notice that max_redirect counts the first request as a redirect. And so if one wants zero redirects, one should set max_redirect to 1 really. The side-effect of LWP::UserAgent detecting a loop and acting on it, appart from inserting a Client-Warning in the response headers, is that the request() will immediately return albeit half-finished. For example no handlers will be run."

        That's no problem, I can do that GLADLY.
Re: LWP::UserAgent : setting max_redirect to 0 yields 'Client-Warning: Redirect loop detected
by bliako (Sexton) on Mar 14, 2018 at 11:27 UTC

    In order to test LWP::UserAgent behaviour on server response 302 Found, I have created the following scripts and uploaded them to a server. If you want to do a quick test then use the last script below to hit You should get redirected to 200.cgi, and have a cookie set.

    script name: 302.cgi

    # perl CGI to send a 302 Found response to caller plus setting # a cookie with name=XYZ123 and value 1. # the REDIRECT_URL points to the script described further below # hosted at my server. use strict; use warnings; use CGI; use CGI::Cookie; my $REDIRECT_URL = ''; my $cgi = CGI->new; my $cookie = $cgi->cookie( -name=>'COOKIE302', -value=>'1', -expires=>'+4h', -path=>'/' ); print $cgi->redirect( -uri=> $REDIRECT_URL, -cookie => $cookie, -status=>302 ); exit(0); 1; __END__

    script name: 200.cgi

    # perl CGI which expects a cookie with name=XYZ123, if received # then cookie's value is incremented by 1 and returned. # this script can act as the final destination of the 302 Found redire +ct script above. use strict; use warnings; use CGI; use CGI::Cookie; my $cgi = CGI->new; my $cookies = fetch CGI::Cookie; my $cookie = $cookies->{'COOKIE302'}; if( ! defined($cookie) ){ print $cgi->header() . "No cookie received.<br>"; exit(1) } my $cookie_value = $cookie->value; if( ! ($cookie_value =~ /^([0-9]+)$/) ){ print $cgi->header() . "Invalid cookie received, value is '$cookie +_value'.<br>"; exit(1) } $cookie->value($cookie_value+1); print $cgi->header( -charset => "utf-8", -cookie => $cookie, ) . "this is 200 OK, you have been redirected OK, your cookie is valid +, value was '".$cookie_value."' and now is '".$cookie->value."'.<br>" +; exit(0); 1; __END__
    The following script can be run locally at command line to make a request to the 302.cgi and check how it handles redirects with various values for LWP::UserAgent's max_redirect and requests_redirectable.

    script name:

    #!/usr/bin/env perl # create a UserAgent and send a request to 302.cgi to see how # it handles redirects. use LWP::UserAgent; use LWP::ConsoleLogger::Easy qw( debug_ua ); use HTTP::Response; use HTTP::Request::Common; use HTTP::Cookies; use URI; my $URLobj = URI->new(''); my $URLstr = $URLobj->as_string; my $ua = LWP::UserAgent->new( # 'requests_redirectable' => [], ); LWP::ConsoleLogger::Easy::debug_ua($ua, 6); $ua->cookie_jar(HTTP::Cookies->new( 'file' => './cookies.txt', 'autosave' => 1, 'ignore_discard' => 1, )); my %extra_headers = (); my $a_request = HTTP::Request::Common::GET( $URLstr, %extra_headers ); if( ! defined($a_request) ){ print STDERR "$0 : call to ".'HTTP::Reque +st::Common::GET()'." has failed.\n"; exit(1) } my $a_response = $ua->request($a_request); print $a_response->as_string."\n";

    The first two scripts are already hosted on a server so one can use the last script only to send a request lto

    The two CGI scripts do not keep any logs.

    bw b
Re: LWP::UserAgent : setting max_redirect to 0 yields 'Client-Warning: Redirect loop detected
by Anonymous Monk on Mar 13, 2018 at 20:21 UTC
    That actually looks like a bug to me. The test should be "is greater than," not "is greater than or equal." I would open a problem-report to that effect. I think that what you have said should be understood to mean that redirects ought not be followed. But, really, there ought to be a follow_redirects (true/false) option to settle the matter. Meanwhile, I wonder what would happen if you set requests_redirectable to an empty array?

      Not quite. It should be >=, but it should only check if the last response in the chain was a redirect.

      Thanks for your reply,

      I did both at the same time: set requests_redirectable to an empty array AND max_redirect to zero. (see the code in my first post). But I also report in another comment here how your suggestion works. And I agree with you that this is probably a bug. That '>=' should be '>' really.

Re: LWP::UserAgent : setting max_redirect to 0 yields 'Client-Warning: Redirect loop detected
by Anonymous Monk on Mar 13, 2018 at 19:12 UTC
    I dont get it. You dont want redirects. Why care about lwp client warnings about redirects?
      I surmise that the OP's objective is to say: "if you're handed a 302, don't do it. At all."

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1210840]
Approved by marto
Front-paged by Corion
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (4)
As of 2018-03-24 06:50 GMT
Find Nodes?
    Voting Booth?
    When I think of a mole I think of:

    Results (297 votes). Check out past polls.