Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Re^4: Mechanize can't find a link with hidden text.

by bitingduck (Chaplain)
on Mar 04, 2015 at 06:49 UTC ( #1118720=note: print w/replies, xml ) Need Help??


in reply to Re^3: Mechanize can't find a link with hidden text.
in thread Mechanize can't find a link with hidden text.

You can look at the response your agent got and see if it has the web page you expect by doing:

print $agent->content();

I just used a browser and did "view source". Virtually all browsers have that option, but sometimes you have to do something like "enable developer menus" or something like that.

Also, did you fix it, or did it fix itself because you're not at a date boundary? For just following a link in a page you already have that doesn't seem like it would be an issue for that page.

Replies are listed 'Best First'.
Re^5: Mechanize can't find a link with hidden text.
by efedotov (Initiate) on Mar 04, 2015 at 17:40 UTC
    The scraper is supposed to get several of the items, so it went back to previous dates. The problem was the item only got released every Monday, Wednesday, and Friday; so it had the problem when it tried to go back several days and touched a Tuesday page, which didn't exist. I just set it to go back one more day to reach the Monday.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1118720]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (10)
As of 2018-09-18 15:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Eventually, "covfefe" will come to mean:













    Results (159 votes). Check out past polls.

    Notices?
    • (Sep 10, 2018 at 22:53 UTC) Welcome new users!