perl -e "fork while fork"
And here's a bit of the output:
Don't blame Perl or threads because your C runtime libraries are paying lip service to the realities of concurrency.
If it doesn't take care of things, you'll have to do it.
The addition of my $old = select $fh; $|++; select $old; might sort out the interleaving problem.
And use threads stack_size => 4096; will substantially reduce the memory footprint.
But that's all irrelevant. Like saying a bendy-bus is 25x better than a family car because it can carry 100 people.
The 500 do-almost-nothing threads in your code
Not my code!
And what is the point of running 500 threads?
It takes ~40MB to run 4 threads. That's a whole 2% of the ram of the lowest spec commodity box you'll ever find for sale.
And with 4 threads, it takes: c:\test>junk39 -THREADS=4 >nul
51.556654214859 at C:\test\junk39.pl line 55, <> line 1
51 seconds to pull 501 names.
With 8 threads: c:\test>junk39 -THREADS=8 >nul
49.5222151279449 at C:\test\junk39.pl line 55, <> line 1.
49 seconds. So doubling the number of threads gained almost nothing. The pipe or the remote server is the limiting factor.
I'd love to compare like with like, but having installed the 131 file that make up the POE behemoth: ppm> install 1
Downloading POE-1.352...done
Downloading POE-Test-Loops-1.351...done
Unpacking POE-1.352...done
Unpacking POE-Test-Loops-1.351...done
Generating HTML for POE-1.352...done
Generating HTML for POE-Test-Loops-1.351...done
Updating files in site area...done
131 files installed
I was still missing stuff your script needed: c:\test>mbethke.pl
Can't locate POE/Component/Client/HTTP.pm in @INC (@INC contains: c:/P
+erl64/site/lib c:/Perl64/lib .)
BEGIN failed--compilation aborted at (eval 33) line 1.
could not import qw(Component::Client::HTTP) at C:\test\mbethke.pl lin
+e 4
BEGIN failed--compilation aborted at C:\test\mbethke.pl line 4.
So, then I tried to download that, but one of its dozen or so dependancies was unavailable: 1: POE-Component-Client-HTTP
a HTTP user-agent component
Version: 0.945
Released: 2012-03-10
Author: Rocco Caputo <rcaputo@cpan.org>
Provide: POE::Component::Client::HTTP version 0.945
Provide: POE::Component::Client::HTTP::Request version 0.945
Provide: POE::Component::Client::HTTP::RequestFactory version 0.945
Provide: POE::Filter::HTTPChunk version 0.945
Provide: POE::Filter::HTTPHead version 0.945
Require: HTTP::Headers version 5.81 or better
Require: HTTP::Request version 5.811 or better
Require: HTTP::Request::Common version 5.811 or better
Require: HTTP::Response version 5.813 or better
Require: HTTP::Status version 5.811 or better
Require: Net::HTTP::Methods version 5.812 or better
Require: POE version 1.312 or better
Require: POE::Component::Client::Keepalive version 0.269 or better
Require: Socket::GetAddrInfo version 0.19 or better
Require: Test::More version 0.96 or better
Require: Test::POE::Server::TCP version 1.14 or better
Require: URI version 1.37 or better
Repo: ActiveState Package Repository
CPAN: http://search.cpan.org/dist/POE-Component-Client-HTTP-0.945/
ppm> install 1
ppm install failed: Can't find any package that provides Socket::GetAd
+drInfo for POE-Component-Client-HTTP
Can't find any package that provides Socket::GetAddrInfo for POE-Compo
+nent-Resolver
It might take a bit less memory, but it certainly won't be quicker because the limitation is the pipe and/or remote server.
I guess I could try installing the 270 module mojo behemothe, but it never terminates: c:\test>zwon >nul
Too late to run CHECK block at c:/Perl64/site/lib/EV.pm line 84, <> li
+ne 501.
Terminating on signal SIGINT(2)
One way uses 0.5% of my memory; only needs what came installed with my Perl installation; and works.
The other two require gobs of extra code and either don't run or never finish. The decision is an easy one for me.
All that's left for me to do is free up about 50 MB of space on my harddrive by throwing away all the crap installed to write this post: ppm> uninstall POE
POE: uninstalled
ppm> uninstall mojolicious
Mojolicious: uninstalled
There, all done.
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
|