SOLVED Updated to perl 5.20.2 and all problems vanished.
Hello! Been noticing some strange memory issues with a script I'm working on, and have come up with this to highlight the issues
use strict;
use warnings;
use threads;
use threads::shared;
$|++;
my $DONE :shared = 0;
my $lock :shared;
my $execMethod = $ARGV[0] || 0;
if($execMethod !~ /[12345]/){
print "Must pass an exec method:\n";
print "1 = backticks\n";
print "2 = backticks synchronized\n";
print "3 = open\n";
print "4 = open synchronized\n";
print "5 = system\n";
exit 1;
}
sub execute{
my $cmd = shift;
if($execMethod == 1){
`$cmd`
}elsif($execMethod == 2){
lock $lock;
`$cmd`
}elsif($execMethod == 3){
open(my $fs, "-|", $cmd);
foreach(<$fs>){};
close $fs;
}elsif($execMethod == 4){
lock $lock;
open(my $fs, "-|", $cmd);
foreach(<$fs>){};
close $fs;
}elsif($execMethod == 5){
system($cmd . ">nul");
}
}
sub worker{
while(!$DONE){
execute('echo hello world');
}
}
my @workers = map threads->create( \&worker), (1..30);
print "Press <enter> to terminate\n";
<STDIN>;
$DONE = 1;
$_->join() for @workers;
Script is invoked with <script>.pl executionMethod
Question 1 relates to memory usage. Methods 2, 4, and 5 all exhibit a permanent memory creep. Is there a way to fix this and let this script run forever? If the solution is periodically join threads, this is not feasible. I have tried this a couple years ago and came across some bug but outlined in this node and changing to an approach like this would require far too much testing to ensure no other bugs occur. If there really is no solution, would be good to know :)
Question 2: For methods 1 and 3, why do i need to synchronize them? If I do not, the script just hangs up usually after only a few seconds. Synchronizing this access kind of seems to defeat the purpose of running system commands in threads, is there something I can do differently?
*I have been running this on Windows 2008 R2, using perl 5.18.2
Thanks in advance for all your help
UPDATE 1: Here is the log file from perfmon running test 5 for a little over a minute and eating roughly 150MB.
UPDATE 2:
perl -v output
This is perl 5, version 18, subversion 1 (v5.18.1) built for MSWin32-x
+64-multi-thread
(with 1 registered patch, see perl -V for more detail)
Copyright 1987-2013, Larry Wall
Binary build 1800 [297570] provided by ActiveState http://www.ActiveSt
+ate.com
Built Sep 20 2013 15:07:17
Perl may be copied only under the terms of either the Artistic License
+ or the
GNU General Public License, which may be found in the Perl 5 source ki
+t.
Complete documentation for Perl, including FAQ lists, should be found
+on
this system using "man perl" or "perldoc perl". If you have access to
+ the
Internet, point your browser at http://www.perl.org/, the Perl Home Pa
+ge.
systeminfo
Host Name: VCLOUD291
OS Name: Microsoft Windows Server 2008 R2 Enterprise
OS Version: 6.1.7601 Service Pack 1 Build 7601
OS Manufacturer: Microsoft Corporation
OS Configuration: Standalone Server
OS Build Type: Multiprocessor Free
Registered Owner: Windows User
Registered Organization:
Product ID: 55041-507-5915375-84291
Original Install Date: 4/23/2014, 12:10:03 PM
System Boot Time: 7/31/2015, 1:49:06 PM
System Manufacturer: Microsoft Corporation
System Model: Virtual Machine
System Type: x64-based PC
Processor(s): 1 Processor(s) Installed.
[01]: Intel64 Family 6 Model 46 Stepping 6
+GenuineIntel ~1862 M
hz
BIOS Version: American Megatrends Inc. 090004 , 3/19/2009
Windows Directory: C:\Windows
System Directory: C:\Windows\system32
Boot Device: \Device\HarddiskVolume1
System Locale: en-us;English (United States)
Input Locale: en-us;English (United States)
Time Zone: (UTC-07:00) Arizona
Total Physical Memory: 4,096 MB
Available Physical Memory: 1,635 MB
Virtual Memory: Max Size: 6,143 MB
Virtual Memory: Available: 3,364 MB
Virtual Memory: In Use: 2,779 MB
Page File Location(s): C:\pagefile.sys
Domain: WORKGROUP
Logon Server: \\VCLOUD291
Hotfix(s): 46 Hotfix(s) Installed.