LWP::Simple not getting content

[jonnyfolk]

I am using LWP::Simple to grab content from a page, and parse the data for exchange rates.

The url of the page is: https://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml?025a000c2e476c719095c900b9c8b5e4

If I go to the page in my browser I can see the (xml) content. Using the script it is not getting the content. Why might this be? (It used to work!).

#!/usr/bin/perl -w
use strict;
use CGI::Carp qw(fatalsToBrowser warningsToBrowser);
use CGI ':standard';
use LWP::Simple;

my $address = 'https://www.ecb.europa.eu/stats/eurofxref/eurofxref-dai
+ly.xml?025a000c2e476c719095c900b9c8b5e4';

my $content = get("$address");
die "Couldn't get it!" unless defined $content;

print "Content-type:text/html\n\n"; 
print $content;

[hippo]

Why might this be?

Could be almost anything, but my guess is on the certificate issuer not being in your CA list. The current cert was only issued on the 7th of June so that's a recent change. What version of Mozilla::CA are you using? It works fine with mine (20180117).

[jonnyfolk]

Thanks for your reply. I switched over to LWP::UserAgent and successfully used the following code

my $url = 'https://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.x
+ml?025a000c2e476c719095c900b9c8b5e4';

my $ua = LWP::UserAgent->new(
  ssl_opts => { SSL_verify_mode => 'SSL_VERIFY_NONE'},
);

   $ua->agent('Mozilla/4.0 (compatible; MSIE 5.12; Mac_PowerPC)' );

my $response = $ua->get($url);
 
if ($response->is_success) {
    print $response->decoded_content;  # or whatever
}
else {
    die $response->status_line;
}

[Corion]

Maybe the remote server does not like the LWP::Simple useragent header?

My suggestion is to use something like LWP::ConsoleLogger and to compare and adapt what your script sends with what your browser sends until the two match, or you get a satisfactory result.