passing parameters from shell script to perl program doesn't preserve spaces [solved]

[peterbk]

I have a perl script test.pl which reads arguments from the command line

#!/usr/bin/perl -w
for( @ARGV ) { print( "$_\n" ) } ;

This works well when I pass parameters like "a b" "c d" e on the command line
The result is:

a b
c d
e

If I declare a variable

X='"a b" "c d" e'

and then run

test.pl $X

I get

"a
b"
"c
d"
e

I need to run the perl script from various shell scripts as a helper function and the parameters are calculated by those scripts.
The parameter count is not fixed so I have to pass it as a list.
Alas I cannot find a way to get the perl program handle my parameters as desired.
I have thought of passing the parameters through file but then the manual invocation of the perl script from the command line becomes awkward.
How can I get the perl script preserve the spaces in the paramters?

[choroba]

Order of shell expansions is important here. For example, in bash, word splitting happens after parameter expansion, so X='"a b"' is split into two words, "a and b" . There's no easy and secure way to prevent it.

What shell are you using? If it's bash, you can use an array to hold the parameters:

#! /bin/bash
X=( 'a b'
    'c d'
    e
)
test.pl "${X[@]}"

Update: Inserted the 1st paragraph.

[Discipulus]

Confirmed, i was trying something similar:

/usr/bin/bash$ aa=('a b' 'c d' e);perl -e 'print qq(>$_<\n) for @ARGV'
+ "${aa[@]}"
>a b<
>c d<
>e<

L*

Edit: removed -l perl switch which caused more newlines

PS: now i remember why I choosed Perl!

[peterbk]

In fact the shell scripts are mostly ksh scripts due to backward compatibility with older HP-UX systems.
I have tried with bash but the problem is the same.
Is there an easy way of distinguishing in perl between input from STDIN and using parameters?
If so I would go and pass arguments from scripts through STDIN and arguments from interactive shells as parameters.

[haukex]

Hi peterbk,

Is there an easy way of distinguishing in perl between input from STDIN and using parameters?

For determining whether the script should use STDIN for accepting parameters, you could use the usual convention (at least for many *NIX tools) that the script has no arguments passed to it, or a single argument consisting of a dash:

my @args;
if (!@ARGV || @ARGV==1 && $ARGV[0] eq '-')
  { @args = <STDIN> }
else
  { @args = @ARGV }

Update: This doesn't handle quoting on the arguments passed on STDIN, but it does allow whitespace in arguments if you place the arguments each on their own lines. If you really need to parse quotes, Text::ParseWords's shellwords has already been mentioned.

Hope this helps,
-- Hauke D

[haukex]

Hi peterbk,

It is not Perl which decides how to split the command line arguments, it is the shell. Which shell are you using? I'm not an expert on the intricacies of shell quoting, but from what I do know it can be a minefield of brittle/non-portable solutions.

I would recommend finding a different solution to passing arguments to your script, such as allowing your script to get options from a file in addition to accepting ones from the command line (I don't find that awkward). Also, note that Perl can access environment variables via the %ENV hash, but if those contain quotes, you will have to parse those yourself, such as with shellwords from Text::ParseWords, as Corion suggested.

Using a file to pass parameters gives you a lot of flexibility on formats, which all handle the quoting/escaping of values too (JSON, YAML, XML, any of the Config:: modules, etc.).

Here's a little bash trick I sometimes use for temporary files that get cleaned up when the script ends, you could use this to generate the file which holds the values:

# in your bash script
TEMPFILE="`mktemp`"
trap 'rm -f "$TEMPFILE"' EXIT
echo "whatever" >$TEMPFILE
# ...

Update: If your Perl script happens to be using Getopt::Long, note that it has the function GetOptionsFromString, and if you're using Getopt::Std, see my node here. Both of these solutions use shellwords from Text::ParseWords.

Hope this helps,
-- Hauke D

[Corion]

Have you looked at how your shell passes command line parameters? Traditionally, you need to add the appropriate amount of quoting and escaping to make a shell not split variables on whitespace.

If you really (really) want to reparse @ARGV like "a shell" would (for various amounts of "reparse" and "a shell"), see maybe Text::ParseWords or Text::Shellwords. Personally, I would really avoid that.

[LanX]

Just for completeness, you could see the call arguments (but only up to a fixed length IIRC) in the process table.

Your process ID is in $$

[kschwab]

This works, at least in bash:

$ args=(one "two three" four "five six")
$ perl -e 'for (@ARGV) {print "$_\n"}' "${args[@]}"
one
two three
four
five six

[peterbk]

Thanks @all for the hints.
Parameter passing from shells seems to be not as easy as I had thought.
Thus I have added another parameter to my perl program, which tells it to read the parameters from a file instead of the command line.
This works like desired and circumvents shell interpretations.

[Discipulus]

oh! if you admit to read params from files you can find Modules as configuration files an interesting solution.

Mixing it with xargs made it easy.

L*

[shawnhcorey]

If you put double quotes around the environment variable, its spaces will be preserved.

X='"a b" "c d" e' ; perl -e'for( @ARGV ) { print( "$_\n" ) } ;' "$X"

You can then parse each argument.

X='"a b" "c d" e' ; perl -e'for( @ARGV ) { print "$1\n" while /\G\s*("[^"]*"|\S+)/gc } ;' "$X"