Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses

Process each file one by one.

by vinoth.ree (Monsignor)
on Sep 04, 2009 at 05:25 UTC ( #793364=perlquestion: print w/replies, xml ) Need Help??
vinoth.ree has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks

I have a two files in a directory, I need to read that directory and need to process each file that directory contains.Both the file contains database queries.What I did say that I opened that directory and read the directory and saved all the files names into an array. Then I used foreach loop inside of that and used the system command to process those files,When those files get process at the same time it also update a log file says that query executed successfully. I need to process each file one by one only. When I check the log files both the log files get updated simultaneously, I thought that after a file completes another file will get process, but both the file get processing. how to process those files one by one.

Below is that foreach loop

foreach(@files{ system "perl download ../downloads/files_download/$_ 1>/tmp/$_.log 2>/ +dev/null &\n "; }

Title updated.

Replies are listed 'Best First'.
Re: Process a file one by one.
by graff (Chancellor) on Sep 04, 2009 at 05:49 UTC
    I gather from your "foreach" loop that "download" is the name of some perl script, and that "@files" has already been filled to hold the two file names that you are talking about.

    Well, if the two files are not very big, and if the "download" script is fairly simple, it might look like the two files are being processed "at the same time", because the whole operation on each file takes a very small fraction of a second.

    The foreach loop pretty much dictates that the two files are being done one at a time, in sequence. What evidence do you have that the two log files are being "updated simultaneously" (apart from the modification time on the files)?

    If you want to see a delay between the updates of the two files, try putting a sleep call inside the foreach loop, right before or right after the system call.

    Oh, and do you understand what the "&" is doing at the end of the command line you pass to system()? On a linux/unix box, that causes each command to be run "in the background"; the shell exits immediately, but the perl process that it runs keeps running until it finishes on its own. If you want to make sure the two iterations do not overlap in time, you have to get rid of the "&" at the end.

    (update: one more thing -- you don't need the "\n" at the end of the command-line string you pass to system.)

Re: Process a file one by one.
by ig (Vicar) on Sep 04, 2009 at 05:46 UTC

    Since you haven't shown the value of @files it is difficult to be certain what your program fragment does. You should examine (print them or use the debugger) the values of @files to ensure they are what they should be and what you think they are. If they are not, you will have to correct the code that sets @files.

    If @files has reasonable values (i.e. simple file names, no wildcards, no shell metacharacters, etc.) then your program fragment looks like it will run your download script once for each file. What is it that makes you think that both log files are updated simultaneously? If it is that they have the same modification time, it may be that your program runs to completion within one second. If you want to ensure that the files are being processed at different times and that the log files will have different timestamps, you can add a delay to your loop:

    foreach (@files) { print "processing \"$_\"\n"; system "perl download ../downloads/files_download/$_ 1>/tmp/$_.log + 2>/dev/null &\n "; sleep 5; }

    If that doesn't resolve your problem, then you should look at your "download" program.

      I already checked the @files, its contains only the file names nothing else

        You've said that before and you were mistaken, you need to prove it.
Re: Process each file one by one.
by vitoco (Friar) on Sep 04, 2009 at 13:30 UTC

    All files are being processed simultaneously because you added the "&" at the end of the line. That says to the system shell not to wait for the command to complete, leaving it running in the background.

    Remove the "&" (and the "\n") and try again...

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://793364]
Approved by graff
[ovedpo15]: The only thing I know is that there is a comma at the end and a value after that
[hippo]: Try it and see. Your computer won't explode.
[Corion]: length will tell you the length of a string.
[Corion]: But if you are parsing CSV, again, let me point to Text::CSV_XS, which you should use instead
[hippo]: +1 for Text::CSV_XS
[ovedpo15]: Im not allowed to use additional modules, so I can't use it. looking for a simple regex to do this task though..
[Corion]: Ah, so this is homework? Then most likely, you should review your course material closer. It would greatly surprise me if the course material wouldn't mention the topics that will be used in the homework...
[Corion]: I already showed you the simple regex to match the last part of a string without a comma in it
[Tux]: Corion++ hippo++

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (6)
As of 2018-05-27 10:41 GMT
Find Nodes?
    Voting Booth?