Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

Re: Parsing HTML

by Corion (Pope)
on Jun 07, 2012 at 10:13 UTC ( #974907=note: print w/replies, xml ) Need Help??

in reply to Parsing HTML

I would use XPath or CSS expressions, and look at HTML::TreeBuilder::XPath to run the expressions against the HTML. Or rather, I would use App::scrape, which puts that approach into a module, or Web::Scraper and Web::Magic.

With XPath expressions, you can specify the elements you want like paths to files in a directory. In your case, it looks like the following XPath expressions would work:

# Each voyage //p[@class="itinerari-info"] # Itinerary within a voyage ./span[1] # Departure date ./span[2] # Ship ./span[3] ...

Depending on whether your target page only lists one such itinerary, you can roll the XPath expressions into one expression, instead of using them relative to the voyage nodes:

# Itinerary //p[@class="itinerari-info"]/span[1] ...

You can test out these queries in Firebug (I think), or with scrape-ff tool in WWW::Mechanize::Firefox, or with the scrape tool in App::scrape. Likely, Mojolicious and the modules mentioned before also contain tools for easy command line testing of XPath expressions against URLs.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://974907]
[erix]: he caught a hare once (must have been a young one...)
virtualsue uses arcane git commands to resurrect a file, gets it back into shape and quits for the day
[GotToBTru]: a few years ago, when we had 3 dogs, the largest of them caught a chipmunk, but did not know what to do with it other than hold it in his mouth
[GotToBTru]: my wife told him to spit it out, so he did
[erix]: yeah, modern dogs are cute tjhat way :)
[erix]: except of course, the RHODESIAN LION HOUND ! :)
[Discipulus]: in roma cars are good mouser.. fortunnately no mouse at 7th floor..
[GotToBTru]: that's a name that should always be accompanied by a trumper fanfare
[GotToBTru]: trumpet, not trumper
[GotToBTru]: good thing too, Discipulus, no roads on the 7th floor either

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (13)
As of 2017-11-17 20:04 GMT
Find Nodes?
    Voting Booth?
    In order to be able to say "I know Perl", you must have:

    Results (272 votes). Check out past polls.