Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re: 2000 cURL Request

by talexb (Canon)
on Aug 24, 2013 at 15:57 UTC ( #1050819=note: print w/ replies, xml ) Need Help??


in reply to 2000 cURL Request

Ideally, the way you should do this is start a relationship with the web site's API, which, when asked nicely, would tell you when a particular user was most recently changed (assuming you're looking for updates to that user).

Once you had that information, you'd be able to get details on just the users that were changed, saving you and the web server lots of wasted cycles.

Alex / talexb / Toronto

"Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds


Comment on Re: 2000 cURL Request

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1050819]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (8)
As of 2015-02-01 09:53 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My top resolution in 2015 is:

















    Results (263 votes), past polls