Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Re: question about LWP and mysterious skipping over of pages

by ikegami (Pope)
on Aug 24, 2004 at 21:18 UTC ( #385511=note: print w/replies, xml ) Need Help??


in reply to question about LWP and mysterious skipping over of pages

Do you see a line "Player\tSport\tSeason\t..." in STATS2 for the skipped loop? If not, that means @sports doesn't get populated as you expect. Either cause the page you downloaded wasn't as you expected it, or because it wasn't downloaded at all. Try checking if $page is defined (and turning on warnings). If that's the problem, you could retry a download that failed.

  • Comment on Re: question about LWP and mysterious skipping over of pages

Replies are listed 'Best First'.
Re^2: question about LWP and mysterious skipping over of pages
by dchandler (Sexton) on Aug 24, 2004 at 21:45 UTC
    I was thinking something along those lines as well. If it were the case that the page wasn't coming in okay, would something like: unless ($page) { redo;} to just repeat the loop be good?
      yup, but you may want to limit the number of times you retry, and maybe pause a second before retrying.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://385511]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (3)
As of 2020-10-27 03:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My favourite web site is:












    Results (256 votes). Check out past polls.

    Notices?