Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

System command using array and pipe

by civil777 (Initiate)
on Jun 04, 2011 at 02:02 UTC ( #908096=perlquestion: print w/replies, xml ) Need Help??
civil777 has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to execute a system command. The problem is that 1) there is a space in the directory name for the directory path, and 2) there is a pipe and a redirection.

I have read that the solution to the first problem is to pass the arguments as an array to system(). But when I try to pass the arguments as an array, I run into the second problem which is that the pipe and redirect do not work when they are passed to system() in the array, apparently because Perl forks a new process and does not have access to shell commands like pipe and redirect. What's the solution?


my $backup_dir = "/home/username/Ubuntu One"; # dump the database and gzip it my @mysqldump = ("mysqldump","--add-drop-table","-uroot","-ppassword", +"mydatabase","|","gzip","-9c",">","$backup_dir/mydatabase.sql.gz"); system(@mysqldump);


mysqldump: unknown option '-9'

Replies are listed 'Best First'.
Re: System command using array and pipe
by ikegami (Pope) on Jun 04, 2011 at 03:17 UTC

    I am trying to execute a system command. The problem is that 1) there is a space in the directory name for the directory path, and 2) there is a pipe and a redirection.

    (2) means you are trying to execute a shell command, but you aren't running a shell. When using the single-argument form of system, Perl will invoke the command in a shell if it thinks necessary. But you aren't using the single-argument form of system.

    system("mysqldump","--add-drop-table","-uroot","-ppassword", "mydataba +se","|","gzip","-9c",">","$backup_dir/mydatabase.sql.gz");

    is the same as

    mysqldump '-add-drop-table' '-uroot '-ppassword' 'mydatabase' '|' 'gzi +p' '-9c' '>' "$backup_dir/mydatabase.sql.gz"

    at the prompt.

    You have a couple of options.

    • Perl is the shell, so have Perl do the redirection.

      use IPC::Open3 qw( open3 ); { local *FROM_MYSQLDUMP; open(local *INPUT, '<', '/dev/null') or die $!; open(local *OUTPUT, '>', "$backup_dir/mydatabase.sql.gz") or die $! +; my $mysqldump_pid = open3('<&INPUT', \*FROM_MYSQLDUMP, '>&STDERR', mysqldump => ( '--add-drop-table', '-u'.'root', '-p'.'password', 'mydatabase', ), ); my $gzip_pid = eval { open3('<&FROM_MYSQLDUMP', '>&OUTPUT', '>&STDERR', gzip => ( '-9c' ), ) }; if (!$gzip_pid) { my $e = $@; kill(KILL => $mysqldump_pid); waitpid($mysqldump_pid, 0); die($e); } waitpid($mysqldump_pid, 0); waitpid($gzip_pid, 0); }
    • You could craft a shell command and have the shell do the piping.

      sub text_to_shell_lit(_) { return $_[0] if $_[0] =~ /^[a-zA-Z0-9_\-]+\z/; my $s = $_[0]; $s =~ s/'/'\\''/g; return "'$s'"; } my $mysqldump_cmd = join ' ', map text_to_shell_lit, mysqldump => ( '--add-drop-table', '-u'.'root', '-p'.'password', 'mydatabase', ); my $gzip_cmd = join ' ', map text_to_shell_lit, gzip => ( '-9c' ); my $output_file_lit = text_to_shell_lit("$backup_dir/mydatabase.sql.gz +"); system("$mysqldump_cmd | $gzip_cmd > $output_file_lit");

    You could surely simplify at the cost of overhead.

    • IPC::Run and IPC::Run3 allows you to have a callback. You could use the callback as a pipe.

    • Again with IPC::Run and IPC::Run3, you could get the entire response from one process into memory and pass it to the next in the chain.

    • You could also replace gzip with a Perl module, leaving you with just external process. Then, all you'd need is backticks.

      Bravo! Thank you very much. Your first solution worked like a charm.

      I did have some trouble getting the second solution to work. Perl is complaining about "Use of uninitialized value $_ in pattern match (m//)" when the $backup_dir is passed to the sub text_to_shell_lit. I studied the text_to_shell_lit sub for a while and could not figure out what was wrong. (BTW, what is the underscore in the argument of the sub? ... Newbie question, I know.)


        The underscore prototype makes $_ the default argument. Specifically, it makes map text_to_shell_lit, equivalent to map text_to_shell_lit($_),. I always forget I have to use $_[0] and not $_ when I use the "_" prototype.

Re: System command using array and pipe
by philipbailey (Chaplain) on Jun 04, 2011 at 08:57 UTC

    I would use a simpler solution than those presented so far: use system() with a scalar argument and simply quote the path containing a space so that the shell sees it as a single argument.

    system("mysqldump --add-drop-table -uroot -ppassword mydatabase | gzip -9c > '/home/username/Ubuntu One/mydatabase.sql.gz'");
      That's basically the second one I posted except you don't handle "'" in paths, spaces and other symbols in passwords, etc.

        Your comments are fair, but in common use cases appropriate quoting of the shell command line is sufficient, especially if one can control the contents of the password and filename. If these come from arbitrary user input, then your approach is more reasonable.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://908096]
Approved by Gangabass
Front-paged by ikegami
Discipulus personae non gratae..
[Corion]: Your Mother: Hmm - I have the pmdev nodelet, which has some links, but I'm not sure what links/guides there are
[Corion]: pmdev wiki maybe?
[Discipulus]: uh neat karl!
[Your Mother]: Both good things to try, thank you!

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (9)
As of 2017-07-21 08:17 GMT
Find Nodes?
    Voting Booth?
    I came, I saw, I ...

    Results (319 votes). Check out past polls.