Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

Hello Monks

The script I am writing will be implemented on a windows device. I'm not sure if the reason why it fails on windows but works in mac is due to incompatibility of commands

Overview of the script. I have A LOT of fancy binary 'chp' files that need to be converted to txt files. The only way to convert this file is through a proprietary program

cont. The script takes an array of the directories of the files, chops them up into smaller arrays of 20. For each 20 files it forks into 20 child processes which sends the command for text conversion into system. Then it parses through the file and gets the data I want

#!/usr/bin/perl use strict; use File::Find; use Parallel::ForkManager; my @toParse; # array of chip files foreach(@toParse){ my $encryptKey = $_; #Encrypted for patients my @name = split /\//, $fileManifest{$encryptKey}; #hash contains +the directories my $id = pop(@name); # @name is an array of the directory, last el +ement is the sample ID my $perlOut = join "\/",@name; # directory for perl commands my @safeID = split / /, $id; my $safeDirect = $perlOut."/".$safeID[0]; # hipaa compliant next if($safeID[0 ] =~ /cgo/ig); #----------------------------------------------------------------- # Prepare the commands for conversion + #----------------------------------------------------------------- my $cychpCommand = $fileManifest{$encryptKey}; #directory of the c +ychp file $cychpCommand =~ s/\//\\/g; # Format the directory for windows and + for the affy program $cychpCommand = "\"".$cychpCommand."\""; my $cychpTxtOut = join "\\", @name; #CYCHP TXT OUTPUT DIRECTORY + $cychpTxtOut ="\"".$cychpTxtOut."\""; + my $affyCommand = "apt-chp-to-txt -o ".$cychpTxtOut." ".$cychpComm +and; unless(exists($cychpLog{$encryptKey})) { my $rmCom = $perlOut."/apt-chp-to-txt.log"; push @toConvert1, $affyCommand; # To send commands for conversion push @toConvert2, $rmCom; # To remove the log files $cychpLog{$encryptKey}++; $timestamp = localtime(time); open LOG, ">>".$logFileName; print LOG "$encryptKey\t$safeID[0]\t$safeDirect\t-\t-\t$timestamp +\n"; close(LOG); } } #----------------------------------------------------------------- # Begin forking #----------------------------------------------------------------- my $maxProcs = scalar(@toConvert1); my $pm = new Parallel::ForkManager($maxProcs); foreach(@toConvert1){ my $pid = $pm-> start and next; # FORK Begin system($_); # CONVERSION COMMAND $pm->finish; # End of FORK } $pm->wait_all_children; # Blocks until all children are finished foreach(@toConvert2){ unlink $_; #Delete log files made by the conversion program } foreach(@toParse){ my $txtFile = $_.".txt"; open IN, $txtFile or die "Cannot open [$txtFile]\n";   while(<IN>){ parsing happens here } }

So my question is as follows,

Is my code converting the files, and then waiting until they are all finished? If not then why not? How can I accomplish this on a windows machine. I cannot convert all files at once because the resulting files are too large. I have to convert a few, parse them, and then unlink them later. The error I'm getting is that the proprietary program cannot open the log file, I presume it's deleted because I unlink them right after the conversion takes place. The only way I think this error is occurring is that the script moves on without waiting for the forking to finish


In reply to Is my code forking the way I want it to? Forking and System Calls by pimperator

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others avoiding work at the Monastery: (3)
As of 2024-06-16 14:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?
    erzuuli‥ 🛈The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.