Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

GD and LWP giving 500 errors

by Schmunzie (Sexton)
on Mar 16, 2017 at 19:55 UTC ( #1184922=perlquestion: print w/replies, xml ) Need Help??

Schmunzie has asked for the wisdom of the Perl Monks concerning the following question:

Holy Monks,

My webhosts (Northgateweb) have changed their upstream providers to Litespeed. Scripts that previously worked using GD or LWP::Simple or LWP::UserAgent are now failing with 500 errors. Uniquely, in my experience, I am getting 500 errors despite using CGI::Carp. When I comment out references to the offending modules, the scripts fail in an elegant way with proper error messages.

I have installed the modules and got the thumbs up from cPanel.

What I'd really like to know is whether anyone else has come across this and what might possibly be the cause - and, of course, how to fix it. The Northgateweb people don't understand Perl at all and tell me they've raised it with Litespeed but all has gone silent.

I can work around GD and LWP::Simple using Image::Size and qx(curl...) respectively but I'm stumped on how to work around sending a POST with LWP::UserAgent. I fear that constructing socks from scratch is beyond my skill level. How else can I do it?

Thanks for practical and/or moral support!

Replies are listed 'Best First'.
Re: GD and LWP giving 500 errors
by huck (Parson) on Mar 16, 2017 at 20:39 UTC

    im not sure why GD is involved, but for the calls using LWP::UserAgent i have a few suggestions. Lets assume you code looks like

    my $ua = new LWP::UserAgent(); my $req = new HTTP::Request (GET => $uri); my $request = $ua->request ($req);
    have you inspected $request->content and $request->status_line for further info like
    500 Can't connect to no.such.server.in.my.lan:80 (Bad hostname)
    That may help you identify what is going on.

    ssl problems can also case a 500. if you change the line to

    my $ua=LWP::UserAgent->new(ssl_opts => { verify_hostname => 0 });
    and it starts working it means you probably have outdated certificates. rather than to continue to skip ssl verification there are modules to update. Mozilla::CA IO::Socket::SSL Net::SSLeay come to mind.

    if those dont seem to identify the problem add this

    Use Data::Dumper; print Dumper ($request);
    That will dump the entire request hash, and inspecting it may provide further insight

    one more thought. There may be some sort of blocking base on the agent. If you change the line to

    my $ua = new LWP::UserAgent(agent =>"myagent");
    and it starts working someone is filtering lwp. another agent you could try is something like
    my $ua = new LWP::UserAgent(agent =>"Mozilla/5.0 (Windows NT 5.1; rv +:51.0) Gecko/20100101 Firefox/51.0");

Re: GD and LWP giving 500 errors
by pryrt (Prior) on Mar 16, 2017 at 22:03 UTC

    Is it just those specific modules you cannot access, or can you not access any module you install via cPanel? Oftentimes with cPanel, you either have to use #!/usr/bin/perlml in your shebang instead of perl, or you can use cPanelUserConfig;, in order to correctly point perl to the user-installed modules. If you cannot access any of the user-installed modules, it may be that when your host changed upstream providers, they might have changed either the shebang requirements or the name of the cPanelUserConfig (I think I remember my webhost changing the name of the use-file at one point).

    On my webhost's cPanel, the "install perl modules" task shows me both the shebang and the use ... methods, and it also shows the installation path for the modules it installs. You might make sure you match what's listed there... and you might even try a simple delme.pl:

    #!/usr/bin/perlml # or whatever your cPanel says use CGI::Carp qw/fatalsToBrowser/; use CGI; # later: add other use BLAH; here print "Content-type: text/plain;\n\n"; print join "\n", @INC; printf "%-20s => %s\n", $_, $INC{$_} for keys %INC; or #!/usr/bin/perl use CGI::Carp qw/fatalsToBrowser/; use CGI; use cPanelUserConfig; # later: add other use BLAH; here print "Content-type: text/plain;\n\n"; print join "\n", @INC; printf "%-20s => %s\n", $_, $INC{$_} for keys %INC;

    ... to find out your @INC and %INC, then slowly add in your other use-statements where indicated.

    Alternately, cPanel should give you a way to look at your log files -- in mine, it's cPanel: statistics: error logs for the host-level logs, or one of the stats packages might give you filtered data.

      All good points. I've stripped the code down to this and it's still 500ing. I am becoming convinced it's blocked by the hosts.
      #!/usr/bin/perlml use CGI::Carp qw(fatalsToBrowser); print "content-type: text/plain\n\nTesting"; use LWP::UserAgent;
      I have installed loads of modules and they all work except for GD and LWP. I can imagine LWP being abused for phishing sites - perhaps that's why it's blocked. But GD??

      I'll try the @INC idea...

      Meanwhile, anyone got a non-LWP way of POSTing?

        more fancy

        use strict; use warnings; print "content-type: text/plain\n\nTesting"; no warnings 'once' ; local $Data::Dumper::Sortkeys=1; local $Data::Dumper::Indent=3; use warnings; addin('Data::Dumper',1); addin('CGI::Carp qw(fatalsToBrowser)',1); addin('GD',1); addin('LWP::UserAgent',1); addin('x::x',1); sub addin_old { my $mod=shift; my $pre=shift; my %inc0=%INC; if ($pre) {print '<pre>'."\n";} eval 'use '.$mod.';' ; print 'eval use ERROR:'.$@."\n" if ($@); my %incnew=(); for my $k (keys %INC) { $incnew{$k}=$INC{$k} unless ($inc0{$k}); } print 'new to %INC from adding use '.$mod."\n"; print Dumper(\%incnew); if ($pre) {print '</pre>'."\n";} } sub addin { my $mod=shift; my $pre=shift; my %inc0=%INC; if ($pre) {print '<pre>'."\n";} eval 'use '.$mod.';' ; print 'eval use ERROR:'.$@."\n" if ($@); my %incnew=(); my $max=0; # this ends up as largest in %INC not largest that is new + , good nuf for my $k (keys %INC) { $max=length($k) if (length($k)>$max);} my $fmt=' %-'.$max.'s => %s'."\n"; print 'new to %INC from adding use '.$mod."\n"; for my $k (sort keys(%INC)) { printf $fmt,$k,$INC{$k} unless ($inc0{$k}); } if ($pre) {print '</pre>'."\n";} }
        result (to term)
        content-type: text/plain Testing<pre> new to %INC from adding use Data::Dumper $VAR1 = { 'Carp.pm' => 'C:/Perl/lib/Carp.pm', 'Data/Dumper.pm' => 'C:/Perl/lib/Data/Dumper.pm', 'Exporter.pm' => 'C:/Perl/lib/Exporter.pm', 'XSLoader.pm' => 'C:/Perl/lib/XSLoader.pm', 'bytes.pm' => 'C:/Perl/lib/bytes.pm', 'constant.pm' => 'C:/Perl/lib/constant.pm', 'overload.pm' => 'C:/Perl/lib/overload.pm', 'overloading.pm' => 'C:/Perl/lib/overloading.pm', 'vars.pm' => 'C:/Perl/lib/vars.pm', 'warnings/register.pm' => 'C:/Perl/lib/warnings/register.pm' }; </pre> <pre> new to %INC from adding use CGI::Carp qw(fatalsToBrowser) $VAR1 = { 'ActivePerl/Config.pm' => 'C:/Perl/lib/ActivePerl/Config.pm' +, 'CGI/Carp.pm' => 'C:/Perl/site/lib/CGI/Carp.pm', 'Config.pm' => 'C:/Perl/lib/Config.pm', 'Config_git.pl' => 'C:/Perl/lib/Config_git.pl', 'Config_heavy.pl' => 'C:/Perl/lib/Config_heavy.pl', 'DynaLoader.pm' => 'C:/Perl/lib/DynaLoader.pm', 'File/Spec.pm' => 'C:/Perl/lib/File/Spec.pm', 'File/Spec/Unix.pm' => 'C:/Perl/lib/File/Spec/Unix.pm', 'File/Spec/Win32.pm' => 'C:/Perl/lib/File/Spec/Win32.pm', 'deprecate.pm' => 'C:/Perl/lib/deprecate.pm', 'if.pm' => 'C:/Perl/lib/if.pm' }; </pre> <pre> new to %INC from adding use GD $VAR1 = { 'AutoLoader.pm' => 'C:/Perl/lib/AutoLoader.pm', 'Fcntl.pm' => 'C:/Perl/lib/Fcntl.pm', 'FileHandle.pm' => 'C:/Perl/lib/FileHandle.pm', 'GD.pm' => 'C:/Perl/lib/GD.pm', 'GD/Image.pm' => 'C:/Perl/lib/GD/Image.pm', 'GD/Polygon.pm' => 'C:/Perl/lib/GD/Polygon.pm', 'IO.pm' => 'C:/Perl/lib/IO.pm', 'IO/File.pm' => 'C:/Perl/lib/IO/File.pm', 'IO/Handle.pm' => 'C:/Perl/lib/IO/Handle.pm', 'IO/Seekable.pm' => 'C:/Perl/lib/IO/Seekable.pm', 'SelectSaver.pm' => 'C:/Perl/lib/SelectSaver.pm', 'Symbol.pm' => 'C:/Perl/lib/Symbol.pm' }; </pre> <pre> new to %INC from adding use LWP::UserAgent $VAR1 = { 'Exporter/Heavy.pm' => 'C:/Perl/lib/Exporter/Heavy.pm', 'HTTP/Date.pm' => 'C:/Perl/lib/HTTP/Date.pm', 'HTTP/Headers.pm' => 'C:/Perl/lib/HTTP/Headers.pm', 'HTTP/Message.pm' => 'C:/Perl/lib/HTTP/Message.pm', 'HTTP/Request.pm' => 'C:/Perl/lib/HTTP/Request.pm', 'HTTP/Response.pm' => 'C:/Perl/lib/HTTP/Response.pm', 'HTTP/Status.pm' => 'C:/Perl/lib/HTTP/Status.pm', 'LWP.pm' => 'C:/Perl/lib/LWP.pm', 'LWP/MemberMixin.pm' => 'C:/Perl/lib/LWP/MemberMixin.pm', 'LWP/Protocol.pm' => 'C:/Perl/lib/LWP/Protocol.pm', 'LWP/UserAgent.pm' => 'C:/Perl/lib/LWP/UserAgent.pm', 'Storable.pm' => 'C:/Perl/lib/Storable.pm', 'Time/Local.pm' => 'C:/Perl/lib/Time/Local.pm', 'URI.pm' => 'C:/Perl/lib/URI.pm', 'URI/Escape.pm' => 'C:/Perl/lib/URI/Escape.pm' }; </pre> <pre> eval use ERROR:Can't locate x/x.pm in @INC (you may need to install th +e x::x module) (@INC contains: C:/Perl/site/lib C:/Perl/lib .) at (ev +al 11) line 1. BEGIN failed--compilation aborted at (eval 11) line 1. new to %INC from adding use x::x $VAR1 = {}; </pre>

        Edit to add:

        interesting there are two modules LWP and GD share

        'Config.pm' => 'C:/Perl/lib/Config.pm', 'Fcntl.pm' => 'C:/Perl/lib/Fcntl.pm',
        adding
        addin('Config',1); addin('Fcntl',1);
        below  addin('Data::Dumper',1); may identify your problem, but both are core

        edit2: just noticed CGI::Carp qw(fatalsToBrowser) brought in Config, and that worked it seems

        Edit 3: fixed addin so Dump::Dumper doesnt have to be the first one added, left old as addin_old but it can be deleted if ya want

        try

        #!/usr/bin/perlml use CGI::Carp qw(fatalsToBrowser); print "content-type: text/plain\n\nTesting"; #use LWP::UserAgent; use Data::Dumper; print '<pre>'; print Dumper(\%ENV); print Dumper(\@INC); print Dumper(\%INC); print '</pre>';
        Notice i commented out the lwp use.

        if that works then try
        #!/usr/bin/perlml use CGI::Carp qw(fatalsToBrowser); use Data::Dumper; print "content-type: text/plain\n\nTesting"; print '<pre>'; eval {require LWP::UserAgent; } ; print 'possible require ERROR:'.$@."\n" if ($@); eval {require x::x; } ; print 'possible require ERROR:'.$@."\n" if ($@); print Dumper(\%ENV); print Dumper(\@INC); print Dumper(\%INC); print '</pre>';
        notice the change to a require in an eval. and x::x to prove it does fail right.

        edit: dayum forgot "use Data::Dumper;"

Re: GD and LWP giving 500 errors
by Schmunzie (Sexton) on Mar 25, 2017 at 15:34 UTC
    Oh my days!

    Now it turns out DBI doesn't work either. I'm never going to qx my way into MySQL. Maybe I just need new hosts.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1184922]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (3)
As of 2019-09-22 18:59 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    The room is dark, and your next move is ...












    Results (273 votes). Check out past polls.

    Notices?