Beefy Boxes and Bandwidth Generously Provided by pair Networks RobOMonk
Just another Perl shrine
 
PerlMonks  

Shell Commands executed in perl script

by msk_0984 (Friar)
on May 22, 2007 at 10:57 UTC ( #616727=perlquestion: print w/ replies, xml ) Need Help??
msk_0984 has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks,

I am facing a problem in integrating Shell commands with a Perl Script...

I used Shell module in my program.

I am getting error at awk in following code:

#!/usr/bin/perl use Shell; $sh = Shell->new; print $sh->ls("-l ./logs/ |grep -v "^d" |grep -v file1 | awk 'NR>1 { p +rint \$9}'");
here $sh is Shell object...

When executed the same command at CLI it is giving correct result....

Here is the error which is being printed on execution of the program

String found where operator expected at grep_data.cgi line 12, near "d" |grep -v "" Bareword found where operator expected at grep_data.cgi line 12, near "" |grep -v "file1" (Missing operator before file1?) String found where operator expected at grep_data.cgi line 12, near "file1" | awk 'NR>1 { print \$9}'"" syntax error at grep_data.cgi line 12, near "d" |grep -v "" Execution of grep_data.cgi aborted due to compilation errors.

Work Hard Party Harderrr!!
Sushil Kumar

Comment on Shell Commands executed in perl script
Download Code
Re: Shell Commands executed in perl script
by rinceWind (Monsignor) on May 22, 2007 at 11:08 UTC

    You have quotes inside your quotes. Try the following:

    print $sh->ls( qq(-l ./logs/ |grep -v "^d" |grep -v file1 | awk 'NR>1 +{ print \$9}'));

    Or indeed you could use q() as you're not doing any variable substitution inside the string.

    Update: yes I know it would be better for the OP to have coded this in perl rather than shelling out 4 subprocesses. Sometimes you need to start with where the person is. Spotting the quote nesting is something glaringly obvious to me - a kind of mistake anyone can make in coding, and worth pointing out.

    --
    wetware hacker
    (Qualified NLP Practitioner and Hypnotherapist)

      Hey rinceWind

      Really thanx for your prompt reply ..... but i am facing another problem after making the changes, when executed it is giving me the error. Even used q() also but no success

      awk: 'NR
      awk: ^ invalid char ''' in expression

      Work Hard Party Harderrr!!
      Sushil Kumar
      Dude ... Do you really need to use Shell at all ??? Convert this shell to perl code...will probably take less time then debugging this errors one at a time.
Re: Shell Commands executed in perl script
by shmem (Canon) on May 22, 2007 at 11:37 UTC
    rinceWind answered your question well, but
    print $sh->ls("-l ./logs/ |grep -v "^d" |grep -v file1 | awk 'NR>1 { p +rint \$9}'

    Gah..! really? fork 4 processes to get a list of files excluding a pattern?

    { my $dir = './logs'; opendir my $dh, $dir or die "Can't opendir '$dir': $!\n"; my @files = grep { ! -d "$dir/$_" and ! /file1/ } readdir $dh; print "$_\n" for @files; } __END__

    To me, integrating doesn't mean copy & paste, but also throwing out cruft and rewrite. The shell command is poor to start with - useless use of grep... then your $9 in the awk command isn't portable, there are different date formats around, e.g. 2007-05-22.

    --shmem

    _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                  /\_¯/(q    /
    ----------------------------  \__(m.====·.(_("always off the crowd"))."·
    ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
Re: Shell Commands executed in perl script
by betterworld (Deacon) on May 22, 2007 at 11:43 UTC
    Why use awk inside Perl?
    my $i=0; print map { (split)[8]; } grep { ! /^d/ && ! /file1/ && $i++ } `ls -l ./logs/`;
    BTW, my ls -l output does not have 9 fields (only for symlinks, where the ninth field is "->"). I guess you can get rid of the shell call altogether if you use stat() and glob() (or readdir() or IO::Dir).
      It's six of one and a half dozen of the other, I suppose, but I would prefer to dispense with the unwanted first line another way.

      $ ls -l total 4 -rw-r--r-- 1 root root 0 May 22 21:09 accts drwxr-xr-x 2 root root 512 May 22 21:10 dir1 -rw-r--r-- 1 root root 0 May 22 21:09 file1 -rw-r--r-- 1 root root 0 May 22 21:09 file1.log -rw-r--r-- 1 root root 0 May 22 21:09 file13 -rw-r--r-- 1 root root 0 May 22 21:09 fred drwxr-xr-x 2 root root 512 May 22 21:10 staff $ perl -Mstrict -Mwarnings -e ' > open my $lsFH, q{-|}, q{ls -l} > or die $!; > scalar <$lsFH>; > print > map { qq{@{ [ (split)[8] ] }\n} } > grep { m{^[^d]} && ! m{file1} } > <$lsFH>;' accts fred $

      Using a filehandle and a scalar read gets rid of the first line before the loop and avoids the extra test each time in the grep.

      Cheers,

      JohnGG

Re: Shell Commands executed in perl script
by andreas1234567 (Vicar) on May 22, 2007 at 13:33 UTC
    You probably don't need the shell at all:
    $ mkdir logs $ touch logs/file1 $ touch logs/file2 $ touch logs/file3 $ touch logs/file4 $ perl -wl use File::Find; # Print filenames in directory 'logs', but skip directories # and files containing 'file1' find( sub { return if (-d or /file1/); print; }, q{logs}); __END__ file4 file3 file2
    Andreas
    --
Re: Shell Commands executed in perl script
by Moron (Curate) on May 22, 2007 at 15:03 UTC
    And yet another reason not to shell is that grep (albeit a more flexible variant) is available in Perl anyway, e.g.:
    print join ("\n", grep !(( -d $_ ) || /^file1$/ ), glob './logs/*' ) . + "\n";
    n.b. your original code will exclude any file for which file1 is a substring found anywhere in the ls -l line for the file - that's a lot of potential holes ... /^files1$/ operating on what the glob returns closes them all, /file1/ would exclude also any file where file1 is any substring of the filename. That seems almost but not quite as unlikely to DWYM under rigorous testing.
    __________________________________________________________________________________

    ^M Free your mind!

Re: Shell Commands executed in perl script
by jabirahmed (Initiate) on May 22, 2007 at 18:10 UTC
    This is an issue i face long back. So i guess you should put the shell commands in a file like a.sh and invoke that from perl. To analyse the problem further if you could check the ps -ef and figure out how the process is carried.. Do let me know if it works. Try using system or exec and one of them should help ... I guess i used something like open (HANDLE,"ls -altr|grep abc|awk -F '{print NF}'|");
    Jabir Ahmed, Bangalore 9980799008
      I tried with "system", and it seems to do the work.

      citromatik

Re: Shell Commands executed in perl script
by 13warrior (Acolyte) on May 23, 2007 at 09:14 UTC
    IMHO
    Some time ago , I was using perl system command to execute shell commands , something similar to what you are doing, grep and awk, i found out that I can do that much more efficiently and faster using perl itself -regex rather than spawning subprocesses .
    Sharad
      my question is plain and simple. Is it possible to execute awk command inside perl script???
Re: Shell Commands executed in perl script
by g_kon (Initiate) on Jan 04, 2008 at 17:50 UTC
    For some obscure reason, Shell.pm quotes apostrophes (') and backslashes (\) on UNIX, and spaces and quotes (") on Windows. If you set $Shell::raw to true no quoting whatsoever is done, so the only remaining thing to do is to negate the special meaning of characters ',{,},$ inside the awk statement. To be consistent here is an example of what you have to do:
    use strict; use Shell; my $sh = Shell->new; $Shell::raw='TRUE'; print $sh->ps("aux | grep firefox | awk \'\{print \$2\}\'");
      Man... you saved me: $sh = Shell->new; $Shell::raw='TRUE'; my $EXISTS_= $sh->java("-classpath bpjt-v2.1-3.jar:pmpclient.jar com.mycompany.clients.FilterWithAttributes $CNUM_ID manager"); my $CNUM_MNGR = $sh->echo("\"$EXISTS_\" | grep mng: | gawk -F \",\" \'\{ print \$1 \}\' | gawk -F \"=\" \'\{ print \$2 \}\'"); Thanks, a lot!

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://616727]
Approved by leocharre
Front-paged by derby
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (10)
As of 2014-04-18 13:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (468 votes), past polls