Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

How can I run an folder full of text files through a perl script?

by Dr Manhattan (Beadle)
on Dec 08, 2012 at 09:57 UTC ( #1007878=perlquestion: print w/ replies, xml ) Need Help??
Dr Manhattan has asked for the wisdom of the Perl Monks concerning the following question:

I have a small perl script that opens a text document and modifies a few things inside the text document through various arrays and regexes etc.

I want to know if there is a way to pass a entire folder full of text files through my perl script?

Comment on How can I run an folder full of text files through a perl script?
Re: How can I run an folder full of text files through a perl script?
by BrowserUk (Pope) on Dec 08, 2012 at 10:02 UTC

    The detail depends which OS you are using. For Windows you might do:

    for %f in (*.txt) do @yourscript.pl %f

    Other OSs have similar shell script solutions, though the syntax will vary.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    RIP Neil Armstrong

Re: How can I run an folder full of text files through a perl script?
by GrandFather (Cardinal) on Dec 08, 2012 at 10:09 UTC

    If you mean have a script process all the files in a folder (rather than using a loop to invoke the script for each file in the folder in turn) then you can either (using OS magic - *nix in particular) pass all the files on the command line using a glob (*), or you pass the folder into the script and use either File::Find or opendir/readdir to run through the files and process them.

    True laziness is hard work
Re: How can I run an folder full of text files through a perl script?
by zentara (Archbishop) on Dec 08, 2012 at 10:16 UTC
    Here is a simple File::Find script that might help.
    #!/usr/bin/perl use File::Find; $|++; my $path = '.'; my $cmd = 'file'; # a test command finddepth (\&wanted,$path); sub wanted { return unless (-f and -T); # only process text files system($cmd ,$_) or warn "$!\n"; # or put your processing code here instead of system } __END__

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh
Re: How can I run an folder full of text files through a perl script?
by graff (Chancellor) on Dec 08, 2012 at 17:31 UTC
    This is a very common type of task (applying some process to each item in a list of files), and it can actually be generalized to a broader set of tasks (applying some shell command-line operation to each item in a list of strings, and in many cases, needing a modified version of the input string to serve as an "output string" on each iteration). In an average work week, I might need to do something like this many times a day.

    So, many years ago, I wrote shloop -- execute shell command on a list, and I've been using it every since. In your case, you'd do something like this, which assumes that your system has the unix-style "ls" utility to list the contents of a directory (there's an 'ls.exe' available for MS-Windows systems), and that your "small perl script" simply takes the name of a text file as the sole command-line argument:

    ls *.txt | shloop -e small_perl_script
    Quite often, the situation involves creating a modified file from a source file, either by providing input and output arguments for a process, or by redirecting process output to a new file:
    # supplying input and output args: "cmd inp.arg out.arg" ls *.txt | shloop -e small_perl_script -s '.txt$:.new.txt' # or, using redirection: "cmd inp.arg > out.arg" ls *.txt | shloop -e small_perl_script -r -s '.txt$:.new.txt'
    shloop has grown over the years to include lots of bells and whistles, but one of the early things I wanted to be sure of was that it would work on MS-Windows shells as well as unix/linux ones. (Getting the Windows version of the linux "bash" shell is always an improvement over a "native" MS-DOS style shell, but shloop should work with cmd.exe, or command.exe or whatever you have.)

    HTH.

Re: How can I run an folder full of text files through a perl script?
by karlgoethebier (Priest) on Dec 08, 2012 at 19:28 UTC

    Just for completeness and fun a simple GUI solution for the Mac using AppleScript:

    set theWindowTitle to "PerlMonks" set theSuffix to text returned of (display dialog "Enter a file suffix +:" & return default answer "" with title theWindowTitle) set theFolder to (choose folder with prompt "Enter directory") set thePerlScript to quoted form of (POSIX path of ((choose file with +prompt "Select a script:") as Unicode text)) -- display dialog "Script: " & thePerlScript with title theWindowTitle tell application "Finder" set theFiles to (every file of theFolder whose name extension is t +heSuffix) --as alias list end tell repeat with theFile in theFiles set theFile to quoted form of (POSIX path of (theFile as Unicode t +ext)) display dialog "Found: " & theFile with title theWindowTitle set theResult to do shell script thePerlScript & " " & theFile display dialog "Result: " & theResult with title theWindowTitle -- beep end repeat

    Perl script:

    #!/usr/bin/perl while (<>) {print}

    Files:

    Karls-Mac-mini:find karl$ cat foo.txt bar.txt foo bar

    Regards, Karl

    «The Crux of the Biscuit is the Apostrophe»

      I've written a couple of AppleScripts before but had forgotten how appallingly bad that language is. Clearly my subconscious had blotted out those memories as some kind of coping mechanism. But now it's all coming flooding back...

      perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'

        It was my first one, honestly! Regards, Karl

        «The Crux of the Biscuit is the Apostrophe»

Re: How can I run an folder full of text files through a perl script?
by karlgoethebier (Priest) on Dec 08, 2012 at 20:14 UTC
    Karl-Mac-mini:find karl$ find . -type f -name "*.txt" | xargs ./perl.p +l bar foo

    Update: or find . -type f -name "*.txt" -exec ./perl.pl {} \;

    See find, xargs

    Update: For xargs vs exec see: 1 2 3

    Regards, Karl

    «The Crux of the Biscuit is the Apostrophe»

Re: How can I run an folder full of text files through a perl script?
by bart (Canon) on Dec 08, 2012 at 22:31 UTC
    If all text files are on the same depth, then either you can depend on the shell globbing (except on Windows) to get all files into the @ARGV array, or use perl's own glob. (Unlike on Linux, Windows doesn't expand *.txt on the command line to a list of all files ending in ".txt". Yo uhave to take care of that yourself.)

    Using glob can be done, for example, like this:

    chdir $dir; @ARGV = glob "*.txt";
    and now you can just use while(<>){...}. On each line, $ARGV contains the current file name, and eof can be used to detect when the current file is finished, and the next file will start to be used.

    Or you can just loop through the file list yourself and open each file in turn.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1007878]
Approved by GrandFather
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (6)
As of 2014-12-21 09:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (104 votes), past polls