Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked

Re^4: Mechanize can't find a link with hidden text.

by bitingduck (Chaplain)
on Mar 04, 2015 at 06:49 UTC ( #1118720=note: print w/replies, xml ) Need Help??

in reply to Re^3: Mechanize can't find a link with hidden text.
in thread Mechanize can't find a link with hidden text.

You can look at the response your agent got and see if it has the web page you expect by doing:

print $agent->content();

I just used a browser and did "view source". Virtually all browsers have that option, but sometimes you have to do something like "enable developer menus" or something like that.

Also, did you fix it, or did it fix itself because you're not at a date boundary? For just following a link in a page you already have that doesn't seem like it would be an issue for that page.

Replies are listed 'Best First'.
Re^5: Mechanize can't find a link with hidden text.
by efedotov (Initiate) on Mar 04, 2015 at 17:40 UTC
    The scraper is supposed to get several of the items, so it went back to previous dates. The problem was the item only got released every Monday, Wednesday, and Friday; so it had the problem when it tried to go back several days and touched a Tuesday page, which didn't exist. I just set it to go back one more day to reach the Monday.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1118720]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (8)
As of 2019-01-18 18:04 GMT
Find Nodes?
    Voting Booth?
    After Perl5, I'm mostly interested in:

    Results (328 votes). Check out past polls.