Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re: WWW:Mechanize::Firefox slow into Wmare/Virtualbox VMs

by exilepanda (Pilgrim)
on Jan 30, 2016 at 16:14 UTC ( #1154071=note: print w/replies, xml ) Need Help??


in reply to WWW:Mechanize::Firefox slow into Wmare/Virtualbox VMs

If you got an physical machine in 800pgs/min && your VM image is a perfect copy of your physical one, then the only difference seem here is your VM.

Are you on Windows though? Windows limits you can only have a few live network connection at a time ( 4 as far as I can remember in XP decade, and 20 max if you have a win7 pro, after some reg hack ), thus you may experience broken result which caused those complains. Consider vm-to-windows consumed 1 channel, and windows-to-web consumed 1 channel, and so forth times 2 network loading( not precisely like that, but the concept is sort of like this). If a round trip is not done fast enough, the afterwards queries are all jammed.

Maybe you can check your scraper how many threads, forks are run at a same time

  • Comment on Re: WWW:Mechanize::Firefox slow into Wmare/Virtualbox VMs

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1154071]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (4)
As of 2020-02-22 07:29 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    What numbers are you going to focus on primarily in 2020?










    Results (98 votes). Check out past polls.

    Notices?