Re: LWP::Simple Seems to Hang on get

[Abigail-II]

You mean, it ran through the loop 7, 4, 2 and 6 times, and then completed the program, without the content of file:///c:/temp/2003.htm? The amount of time the while loop gets executed only depends on the content of file:///c:/temp/2003.htm, not of what yahoo is returning.

Abigail

[svsingh]

Sorry, I was unclear. The program has never finished execution and only stops when I break execution. The main page is local, and from that I extract a list of IDs. The while loop takes each ID and then calls each subroutine with the ID. Each subroutine loads a separate page from Yahoo with the ID in the URL.

I updated the code to use the real Yahoo index page instead of my local copy. The local file never changed while I was running the script and contains over 500 rows.

I hope this clarifies things.

[Abigail-II]

Perhaps you are impatient? The page contains 680 players, and you sleep for 15 seconds for each player. So, even if the fetching and scanning takes no time at all, it's still going to take 2 hours and 50 minutes for your program to finish. I took the sleep out, and your program seems to be running fine.

Abigail

[svsingh]

I realize that the script will take a while to run and that's why I added the print STDERR statement. Thinking the problem was that maybe those statements weren't printing immediately, I set $|=1. It doesn't seem to be helping. I'm rerunning the code and it's stuck again, this time at the second iteration.

Anyway, thanks for your help. I'm now using LWP::Simple::getstore to save the files locally. I hope that by running the script against local files, I'll be able to bypass whatever's holding up the get calls.