Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister

Re^4: multi threading DBI

by Laurent_R (Abbot)
on Oct 29, 2013 at 22:43 UTC ( #1060233=note: print w/replies, xml ) Need Help??

in reply to Re^3: multi threading DBI
in thread multi threading DBI

I definitely agree with what you said, marinersk. From the context, I would think that what is slow is collecting the data on the Internet, therefore BrowserUK's solution to multithread the web scraping part is certainly the first solution that came to my mind and I believe most probably the right one. Having said that, sundialsvc4's response also makes an interesting point which is worth taking into consideration (and can very easily be tested with minimal code changes).

The point is that we would really need profiling of the program to figure out what is really slow.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1060233]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (6)
As of 2016-09-29 02:30 GMT
Find Nodes?
    Voting Booth?
    Extraterrestrials haven't visited the Earth yet because:

    Results (545 votes). Check out past polls.