Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

Re: How do I Extract Data From's Website?

by matsi (Novice)
on Aug 15, 2007 at 18:43 UTC ( #632828=note: print w/replies, xml ) Need Help??

in reply to How do I Extract Data From's Website?

Respected Monks suggested several modules aimed at handling Amazon site.
But if you want implement similar stuff yourself or probably you need to solve similar problem for another website, then you should look around LWP modules.
  • Comment on Re: How do I Extract Data From's Website?

Replies are listed 'Best First'.
Re^2: How do I Extract Data From's Website?
by Joost (Canon) on Aug 15, 2007 at 18:50 UTC
    If you're dealing with a web interface (i.e. HTML with forms, links, etc instead of "pure" HTTP) you're probably better off using WWW::Mechanize - it's an LWP subclass with lots of specialized methods to search and "click" through web pages/forms.

    There are even a few "clones" of WWW::Mechanize that use popular browsers at the back-end so you can deal with javascript and other client-side objects not normally supported by WWW::Mechanize.

      Recently, some folks figured out a way to use Mozilla in a mechanize driver without the need for a (visible) X-server.

      Sounds promising.


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://632828]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (4)
As of 2021-05-07 13:43 GMT
Find Nodes?
    Voting Booth?
    Perl 7 will be out ...

    Results (91 votes). Check out past polls.