http://www.perlmonks.org?node_id=992021


in reply to Re: need help in scrapping asp site
in thread How to scraper ASP websites

When added to a regex, the x modifier tells the regex engine to ignore whitespace — that is, to omit the spaces, etc., in the regex from the pattern to be matched. So, if you are trying to match something like:

<td class="countryValue"> # ^ note the space

and your regex has an x modifier, you must specify the space(s) to be matched explicitly. For example:

<td \s+ class="countryValue">

That said, when I run your code with this fix applied:

while ($content =~ m! tr \s+ class="bgrow1"> <td> (.*?) + # $1 </td> <td \s+ class="countryValue"> (.*?) + # $2 country </td> <td \s+ class="destnameValue"> (.*?) + # $3 destination </td> <td \s+ class="hotelNameValue"> (.*?) + # $4 </td> <td \s+ class="durationValue"> (.*?) + # $5 trip_length </td> <td \s+ align="RIGHT" \s+ class="priceValue"> <a \s+ target="_blank" \s+ href="(.*?)"> + # $6 url (.*?) + # $7 </a> </td> !gisxm)

the regex still gets no matches, so there is more wrong than just the missing whitespace. (Or, there is more whitespace lurking in the target webpages than I have allowed for.) For further help from the monks, please follow the advice given above by davido, and reduce your problem to a minimal code snippet demonstrating the problem and complete with representative data.

BTW, the variable $airport is accessed in the final print statement, but never initialized. You would have seen this if you had begun the script with

use strict; use warnings;

as Gangabass advised in Re: How to scraper ASP websites.

Athanasius <°(((><contra mundum