Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

how to read data from a URL

by armenica (Novice)
on May 15, 2007 at 07:49 UTC ( #615484=perlquestion: print w/replies, xml ) Need Help??

armenica has asked for the wisdom of the Perl Monks concerning the following question:

Hi!

Is there a simple way to open a handle to an URL and read the replied content? I would like to convert the following PHP code to PERL, but can't quite find a function which could do the same:

//Specify your URL $URL = "http://www.exampleurl.com"; //Open a stream in READ mode $handle = fopen ($URL, "r"); //Read the content of the URL and manipulate it. $key = str_replace(' ', '%20',fread($handle, 1000000));

It's the openning of the URL-stream and how to read it I would like to know.

Thanks in advance,

/V

Edit: g0n - code tags and formatting

Replies are listed 'Best First'.
Re: how to read data from a URL
by Tomte (Priest) on May 15, 2007 at 07:55 UTC

    Something like the following should get you going:

    use strict; use warnings; use LWP::Simple; my $url = "http://whatev.er"; my $content = get($url); $content =~ s/ /%20/g;

    regards,
    tomte


    An intellectual is someone whose mind watches itself.
    -- Albert Camus

      # An example using WWW::Mechanize and URI::Escape
      
      use strict;
      
      use WWW::Mechanize;
      # Create a new mechanize object
      my $mech = WWW::Mechanize->new();
      my $url = 'http://www.exampleurl.com/';
      # Associate the mechanize object with a URL
      $mech->get($url);
      # Print the content of the URL
      print $mech->content;
      
      # Load URL::Escape to escape URI's with uri_escape method.
      use URI::Escape;
      my @links = $mech->links;
      foreach my $link (@links) {
          print uri_escape($link->url), "\n";   
      }
      
Re: how to read data from a URL
by Mutant (Priest) on May 15, 2007 at 08:19 UTC
    Hi armenica,

    As mentioned above, LWP::Simple will do what you need. But here are a few more tips:
    • It's actually spelt 'Perl' not 'PERL'. It may seem pedantic, but people in the Perl community can get a bit irritated if you get this wrong.
    • In general, Perl prefers to have less functionality in the 'core', and more in loadable modules, as opposed to PHP, which packs all it's functionality into the core. So for example, in PHP you don't have to do anything special to get a remote URL other than call the function, but in Perl you have a few modules to choose from, one being LWP::Simple. There are pros and cons to both approaches, but it can take a while to get used to this in Perl if you've come from PHP. A good place to find the modules, along with their documentation is search.cpan.org (CPAN is like PEAR in PHP).
    • If you have content that you need to URI encode, try URI::Escape (another example of where Perl puts functionality into a module rather than has it in the core as PHP does).
    • There are a few places to find Perl's documentation, but one is http://perldoc.perl.org/
    Also note that posts on PerlMonks use HTML, so you need to add in linebreaks, etc. It also supports the <code> tags, which make formatting your code much easier. If you wrap your code above in these tags, you'll see what I mean.
Re: how to read data from a URL
by bart (Canon) on May 15, 2007 at 10:02 UTC
      With regards IO::All - I hadn't read the POD until your comment. It sounds like a good idea, so why would you hesitate to recommend it as a unified interface? (other than the fact that Ingy says that he may break backwards compatibility in the future)?

        Using IO::All reopens all the security holes that the three argument form of open fixes. Of course, in the sense of universality, IO::All is just like PHP, which also has these problems of being able to launch a process or access a foreign website via an open with an unguarded filename, so there is no loss by using IO::All, but especially when writing applications that are publicly accessible, it's better to have less magic.

Re: how to read data from a URL
by mantra2006 (Hermit) on May 15, 2007 at 13:03 UTC
      thumbs up - thx for the link!
Re: how to read data from a URL
by AK108 (Friar) on May 15, 2007 at 15:49 UTC
    LWP::Simple is very good for simple requests like this. While there are other LWP modules for more advanced requests, I prefer WWW::Mechanize for submitting forms and doing multiple requests on a site. It's quite powerful.
Re: how to read data from a URL
by faraco (Acolyte) on Mar 12, 2017 at 06:31 UTC
    I just use LWP::Simple, it's pretty common option for what you need. I recommend to have a look at https://perlmaven.com/simple-way-to-fetch-many-web-pages.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://615484]
Approved by Corion
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (3)
As of 2021-08-01 11:24 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My primary motivation for participating at PerlMonks is: (Choices in context)








    Results (5 votes). Check out past polls.

    Notices?