Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re: Re: LWP::Simple Seems to Hang on get

by svsingh (Priest)
on Nov 06, 2003 at 22:47 UTC ( #305180=note: print w/ replies, xml ) Need Help??


in reply to Re: LWP::Simple Seems to Hang on get
in thread LWP::Simple Seems to Hang on get

Sorry, I was unclear. The program has never finished execution and only stops when I break execution. The main page is local, and from that I extract a list of IDs. The while loop takes each ID and then calls each subroutine with the ID. Each subroutine loads a separate page from Yahoo with the ID in the URL.

I updated the code to use the real Yahoo index page instead of my local copy. The local file never changed while I was running the script and contains over 500 rows.

I hope this clarifies things.


Comment on Re: Re: LWP::Simple Seems to Hang on get
Re: LWP::Simple Seems to Hang on get
by Abigail-II (Bishop) on Nov 06, 2003 at 23:08 UTC
    Perhaps you are impatient? The page contains 680 players, and you sleep for 15 seconds for each player. So, even if the fetching and scanning takes no time at all, it's still going to take 2 hours and 50 minutes for your program to finish. I took the sleep out, and your program seems to be running fine.

    Abigail

      I realize that the script will take a while to run and that's why I added the print STDERR statement. Thinking the problem was that maybe those statements weren't printing immediately, I set $|=1. It doesn't seem to be helping. I'm rerunning the code and it's stuck again, this time at the second iteration.

      Anyway, thanks for your help. I'm now using LWP::Simple::getstore to save the files locally. I hope that by running the script against local files, I'll be able to bypass whatever's holding up the get calls.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://305180]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (18)
As of 2014-10-22 14:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    For retirement, I am banking on:










    Results (118 votes), past polls