500 error with LWP:UserAgent

[RenardBleu]

Solved !
thanks for every help i had :)

Found http://stackoverflow.com/questions/8026524/how-do-i-force-lwp-to-use-cryptssleay-for-https-requests and solved my issue with

use Net::SSL (); # From Crypt-SSLeay
BEGIN {<br />
  $Net::HTTPS::SSL_SOCKET_CLASS = "Net::SSL"; # Force use of Net::SSL
  $ENV{HTTPS_PROXY} = 'http://10.0.3.1:3128';
}

------------------ Hi
I tried a thousand different ways to connect to the URL https://sis-t.redsys.es:25443/sis/entradaXMLEntidad/ and i seem unable to get something else than a 500 error.

Latest code i tried was

require LWP::UserAgent;
 
my $ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 0});
$ua->agent("Mozilla/8.0");
$ua->timeout(10);

my $req = HTTP::Request->new( GET => "https://sis-t.redsys.es:25443/si
+s/entradaXMLEntidad/");
$req->header( 'Accept' => 'text/html' );

# send request
my $res = $ua->request($req);

# check the outcome
if ( $res->is_success ) {
    print $res->decoded_content;
} else {
    print "Error: " . $res->status_line . "\n";
}

I tried with and withcout $ua->proxy('https', 'http://192.168.0.1:80');
I tried POST and GET (since in the end, ill have to access that URL with POST)

of course I made sure I was able to get this page from a regular browser. I made sure my code always was able to connect to other pages with the form of https://some-domain:some-port/path/to/another/page

If anyone can help me to find what i do wrong, I d be thankfull

[Loops]

Hi there!

Would like to help, but this is not a Perl question. As you said, changing the URL to another site, it-just-works. The problem is knowing what that payment system company expects; unfortunately their documentation seems to be private. You'll need a peek at the documentation or some help from them directly.

Update: Okay, have to apologize I guess, since I decided to try your code here anyway. It turns out with v5.18.4, i get your 500 error, and with v5.20.0 i get web content back saying that an error occurred (in Spanish). With the exact same copy of your code. Odd.

Update2: Wireshark revealed it was an SSL error causing the problem. Something must have changed in the Perl SSL module between v18 and v20. Anyway, if you change the ssl_opt to that given below, the code works in either version:

my $ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 0, SSL_v
+ersion => 'TLSv1' });

You may want to look at the other SSL options as listed in the IO::Socket::SSL docs.

HTH

[RenardBleu]

Hi there,

I tried your modification and it didnt change my case.

I use
perl v5.14.2
LWP::UserAgent v6.06
HTTP::Request v6.00
IO::Socket::SSL 1.955

Adding


use Net::SSLeay;
$Net::SSLeay::trace = 2;


as suggested on stackoverflow : http://stackoverflow.com/questions/26732578/500-error-with-lwpuseragent produced the output


DEBUG: .../IO/Socket/SSL.pm:449: socket not yet connected
DEBUG: .../IO/Socket/SSL.pm:451: socket connected
DEBUG: .../IO/Socket/SSL.pm:469: ssl handshake not started
DEBUG: .../IO/Socket/SSL.pm:504: using SNI with hostname sis-t.redsys.
+es
DEBUG: .../IO/Socket/SSL.pm:527: set socket to non-blocking to enforce
+ timeout=10
DEBUG: .../IO/Socket/SSL.pm:550: ssl handshake in progress
DEBUG: .../IO/Socket/SSL.pm:560: waiting for fd to become ready: SSL w
+ants a read first
DEBUG: .../IO/Socket/SSL.pm:570: handshake failed because socket did n
+ot became ready
Error: 500 Can't connect to sis-t.redsys.es:25443

[Loops]

Well it's a puzzle for sure. I installed Perl and those modules with the same version you have, and it just worked with the updated code. The old code gave the same Net::SSLeay debug output as yours. There is some underlying difference between our setups obviously.

There's a chance that changing the SSL version to this would help:

SSL_version => 'TLSv1:!TLSv11:!TLSv12:!SSLv2:!SSLv3'

The last two components are supposedly redundant. But in any case, that long version string wasn't needed here (with any of 3 machines).

Update: Just as a debugging step, try adding random garbage to the string you pass into SSL_version. You should expect the following error, otherwise something is up in Denmark:

Error: 500 invalid SSL_version specified

[Discipulus]

yes, as stated seems not a problem with your code. Browsing the link you'll have:


La forma en que intenta acceder al sistema de pago no es correcta.

Este mensaje lo puede estar viendo por dos motivos:

1.- Nos envía el formulario con los datos por método GET.
2.- Está usted intentando navegar por esta página; esta pagina no está
+ accesible para entrar directamente desde el navegador de internet.

Para acceder correctamente al TPV Virtual debe enviar a esta URL un fo
+rmulario HTML mediante POST, con los parámetros descritos en la docum
+entación que se obtiene a través del módulo de administración.

Por favor, tenga en cuenta esto para poder realizar pagos por internet
+ con nuestro sistema.

And using my, still not perfect, WebTimeLoad.0.23.pl i got:

#webtimeload23.pl -u https://sis-t.redsys.es:25443/sis/entradaXMLEntid
+ad/ -v 4 -h
====================================================================
https://sis-t.redsys.es:25443/sis/entradaxmlentidad     Tue Nov  4 11:
+17:21 2014
--------------------------------------------------------------------
Response code:            404
Response message:         Not Found
Response server:          UNDEF
Response declared length: 219
Response title:           404 Not Found
--------------------------------------------------------------------
main page content (1):  219.0000 bytes in 61.573974 seconds @ 3.5567 b
+ytes/s)
--------------------------------------------------------------------
        detail of loaded pages (url):
--------------------------------------------------------------------
        https://sis-t.redsys.es:25443/sis/entradaxmlentidad
--------------------------------------------------------------------
no included content found.
no external content found.
no broken links found.
--------------------------------------------------------------------
downloaded 219.0000 bytes (219 bytes) in 61.573974 seconds (3.5567 byt
+es/s)
====================================================================
headers received:
--------------------------------------------------------------------
Connection: close
Date: Tue, 04 Nov 2014 10:17:20 GMT
Content-Length: 219
Content-Type: text/html; charset=iso-8859-1
Client-Date: Tue, 04 Nov 2014 10:17:21 GMT
Client-Peer: 195.76.9.247:25443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /C=US/O=VeriSign, Inc./OU=VeriSign Trust Netwo
+rk/OU=Terms of use at https://www.verisign.com/rpa
 (c)10/CN=VeriSign Class 3 International Server CA - G3
Client-SSL-Cert-Subject: /C=ES/ST=Madrid/L=Madrid/O=REDSYS SERVICIOS D
+E PROCESAMIENTO, S.L./OU=REDSYS SERVICIOS DE PROCE
SAMIENTO, S.L./CN=sis-t.redsys.es
Client-SSL-Cipher: AES128-SHA
Client-SSL-Warning: Peer certificate not verified
Set-Cookie: TS8e1dfd=a50bf1628fedf28fafe3b2ec1d8aca3ea874d6dd7c147bd15
+458a7b0; Path=/
Title: 404 Not Found

====================================================================


HtH
L*

[Khen1950fx]

Stick with POST. GET just gives you an "incorrect" warning; however, using POST will give you an exception. Here's the code that I ran:

#!/usr/bin/perl -l

use strict;
use warnings;
use Encode;
use Encode::Locale;

require LWP::UserAgent;
require HTTP::Request;

my $ua = LWP::UserAgent->new( ssl_opts => { verify_hostname => 0 } );
$ua->agent("Mozilla/8.0");
$ua->timeout(10);
$ua->protocols_allowed( ['https'] );

my $req = HTTP::Request->new( 'POST',
    'https://sis-i.redsys.es:25443/sis/entradaXMLEntidad/' );
$req->header( 'Accept' => 'text/html' );

my $res = $ua->request($req);

if ( $res->is_success ) {
    print $res->decoded_content;
}
else {
    print "Error: " . $res->status_line . "\n";
}

And the result:

<!-- RSisException --><html>
<head>
<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1
+">
<meta content="-1" http-equiv="Expires">
<meta content="Monday, 01-Jan-90 00:00:00 GMT" http-equiv="Expires">
<meta content="no-cache" http-equiv="Pragma">
<meta content="no-cache" http-equiv="Cache-Control">
<script type="text/javascript" src="/dtagent56_n3_5964.js" data-dtconf
+ig="rid=RID_1948368250|rpid=272727326|tp=500,50,0|domain=redsys.es"><
+/script><link type="text/css" href="/sis/estilos/unica/9999.css" rel=
+"StyleSheet">
<link type="text/css" rel="StyleSheet" href="/sis/estilos/.css">
<script src="/sis/javascript/xsl/RSisException.js" type="text/javascri
+pt"></script><script src="/sis/javascript/utilSis.js" type="text/java
+script"></script>
</head>
<body onload="load()">
<div id="divCuerpo" style="overflow:auto;height:90%;">
<table class="contenedor">
<tr>
<td height="52" colspan="2" align="right" class="tituloSuperior"><img 
+height="52" onerror="this.src='/sis/graficos/logotipos/comunes/redsys
+.gif';" src="/sis/graficos/logotipos/comunes/-1.gif" alt="Terminal de
+ pagos virtuales"></td>
</tr>
<tr>
<td colspan="2" class="separador">&nbsp;</td>
</tr>
<tr>
<td colspan="2" class="resultado"><font class="denegacion">
<!--:--></font></td>
</tr>
<tr>
<td colspan="2" class="separador">&nbsp;</td>
</tr>
<tr>
<td colspan="2" class="botones">
<form action="" method="get">
<div style="position:relative;visibility:hidden" id="divImg">
<input type="button" class="botonAccion" onmouseover="hov(this,'botonA
+ccion btnhov')" onmouseout="hov(this,'botonAccion')" value="Imprimir"
+ onClick="javascript:imprimir();" alt="Imprimir"><input type="button"
+ class="botonAccion" onmouseover="hov(this,'botonAccion btnhov')" onm
+ouseout="hov(this,'botonAccion')" value="Cerrar" onClick="window.clos
+e();" alt="Cerrar">
</div>
<div style="position:relative;visibility:visible" id="divImgNoScript">
<NOSCRIPT value="Imprimir">                                
                                            &nbsp;
                                            <input type="submit" class
+="buttonAccion" value="Cerrar" ALT="Pulse el bot&oacute;n para cerrar
+">
</NOSCRIPT>
</div>
</form>
</td>
</tr>
</table>
</div>
<div id="divPie" style="overflow:visible; height:55px;">
<table cellpadding="0" cellspacing="0" border="0" width="100%">
<tr style="background-color:#8e9092;">
<td height="52" style="width:100%">&nbsp;</td>
</tr>
</table>
</div>
</body>
</html>

Make sure that you have these modules installed:

LWP::Protocol::https
Mozilla::CA
Net::HTTPS

[noxxi]

> ... sis-t.redsys.es:25443

This server has some serious issues. If you use the old default cipher set of IO::Socket::SSL, that is "ALL:!LOW", with TLS1.0 or higher the connection simply hangs. This kind of problem is often seen in connection with older F5 load balancers in front of the server, which simply drop ClientHello packets greater than 255 bytes. Since version 1.962 (about a year ago) IO::Socket::SSL will use a smaller (and more secure) cipher set to work around such problems. But I guess you are using an older version of IO::Socket::SSL.

Crypt::SSLeay instead uses the default cipher set of OpenSSL, which makes the packet in this case just 248 bytes and thus small enough. That's why it works with Crypt::SSLeay. But please note, that Crypt::SSLeay does not make any verification of the hostname against the servers certificate and is thus open to man-in-the-middle attacks.

With current versions of IO::Socket::SSL the problem should be fixed too. But, since you are using a proxy you should use the latest versions of LWP::UserAgent and LWP::Protocol::https too, because proper proxy support with IO::Socket::SSL backend was only added to version 6.06 (it might be in 6.04 already if you use Debian or a spinoff like Ubuntu).