Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Automating execution of shell scripts

by wisely (Initiate)
on Aug 30, 2021 at 11:22 UTC ( #11136202=perlquestion: print w/replies, xml ) Need Help??

wisely has asked for the wisdom of the Perl Monks concerning the following question:

Hi all, This is my first post here and my question is not so much about the exact code implementation at this stage but rather what is the best way to achieve this and is it possible.
So I have these two shell scripts that I need to run to create environments for my work and I can't make any changes to the shell scripts themselves.
So the first shell script takes in files as well as a few arguments and created a directory based on them and it also prints in the terminal the path to the created dir.
So for example my first script will look like this: ~/shell_1.csh file1 arg1 arg2
And the output from it will be a directory like this (number after the underscore is the date):
/project/file1/file1_270821.
I created a perl script to automate the execution of the above shell script I just give it all the files I need and it extracts the needed argument and runs the shell script for each one.
The second shell script I need to execute in each of the created directories.
So for the above example I would change my dir to /project/file1/file1_270821 and then execute shell_2.csh in it.
This is tedious if I have just created 5 such directories for example, so I want to automate this.
What i was thinking is to have it check the directories for all files I have passed to it and run the second shell script in the one with the latest date.
So for example if I have passed file1, my perl script to go into the /project/file1 dir, find the file1_* with the latest date and then run the shell_2.csh in that dir.
Is it possible for this to be achieved with perl? If yes is the way with checking dates the most efficient way?

Replies are listed 'Best First'.
Re: Automating execution of shell scripts
by BillKSmith (Monsignor) on Aug 30, 2021 at 14:44 UTC
    As I understand your problem, you have two shell scripts which you must use "as is". You want a perl program to automate this procedure. For each argument, run script1 which computes a directory name and creates that directory, then run script2 in that directory. Your question is how can your perl program get the name of directory it needs to run script2.

    You tell us that script1 " prints in the terminal the path to the created dir." You should be able to capture that output if you run script1 with backticks. my $script_output = `script1`'. You can change the working directory with the function chdir. You may need the module Cwd to get the directory you want to return to.

    Bill
      Hi Bill,

      Yes, that sums up the problem well :).
      I was not sure how can I get the info that the first script prints out and I wasn't sure if it was a feature.
      So I didn't consider it and went with checking the dates of the folders, which is not as good of an option.
      This sounds like the solution I am searching for I will try to implement it.
      Thank you very much for the help.

      It appears I am missing something about the way this works as I am not able to implement it.
      I added the backticks and it appears to correctly capture the stdout, I tried the following:
      $script2 = `script1 arg1 arg2 arg3 arg4`; $working_dir = &getWorkDir($script2); sub getWorkDir { $text = @_; while($text) { if ($_ =~ ".*user_name.*"){ my $dir = $_; } return $dir; }
      This appears to never enter the if statement even.
      I also tried using @script2 to store the output and then used @text in the sub procedure and then loop through it using foreach.
      That way actually managed to enter the if statement once the line containing the directory was reached but still $dir was left empty.
      I don't understand why the $dir does not get assigned a value. When I check in the debugger once I enter the if statement the $_ actually contains the line but I still can't assign it to $dir and return it.
      What am I missing here?
        $text = @_;

        This evaluates @_ in scalar context, returning the number of parameters instead of the actual value. Instead, use

        my ($text) = @_;

        Also,

        if ($_ =~ ".*user_name.*"){

        is better written as

        if (/user_name/) {

        map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]

        Your code as supplied does not compile:

        $ perl -cw 11136410.pl Missing right curly or square bracket at 11136410.pl line 11, at end o +f line syntax error at 11136410.pl line 11, at EOF 11136410.pl had compilation errors.

        Other problems: you are not using strict, your scope of $dir is incorrect, there is no assignment to $_, you have scalar context when assigning the arguments in your subroutine and your while loop will never end.

        Spend a little time trying to fix these issues and post back here if you get stuck.


        🦛

        There are so many errors in your subroutine that we are not sure what you are trying to do. I recommend that you study the documentation perlsub. I suspect that you are confusing us (and yourself) with a poor choice of variable names ($script1 for the name of a script file and $script2 for its output???). Given the code you have shown, $_ should not contain anything. You probably do not need a subroutine if all it does is extract the directory name from the script output. Use a regex with capture parenthesis in your main program.

        We can provide much better help if you post a short program that we can run and duplicate your problem. Use use strict; use warnings and fix all errors and warnings that they cause. (Read the tutorial Short, Self-Contained, Correct Example)

        Bill
Re: Automating execution of shell scripts
by roboticus (Chancellor) on Aug 30, 2021 at 13:42 UTC

    wisely:

    From your description, it sounds like you have all the information you need in the first script to run the second script, so I'd suggest either wrapping things up into a single script *or* letting your first script write your second script.

    For the first option, one common way to do it is to use perldoc -f fork to split your process so you can have your original process do the work you're currently doing in your first script, and then have the copied process do the work you'd do in the second script. There are also some packages on CPAN that help you manage multiple jobs that you may want to investigate.

    I generally use the second option, though, which is to write your second script from your first. Since you already have a script that does the second task, all you really need to do is create a shell script to run the second task with the parameters you've created in the first task. Then once your first script runs, you can execute the new script that it creates, or even add some code to the end of your script to execute the script automatically.

    #!env perl use strict; use warnings; open my $FH, '>', 'task_part_two.sh' or die "Can't create script 'task_part_two.sh': $!\n"; while (my $arg = shift) { print "Setting things up for task $arg\n"; # . . . do the work for the first script . . . # Create a new directory name my $the_dir = "dir_" . int(1000*rand); # build the directory and put the required file in it # Now that it's set up, write the command to run the # second script print $FH "/the/script/path.sh '$the_dir' '$arg'\n"; }

    ...roboticus

    When your only tool is a hammer, all problems look like your thumb.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11136202]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (2)
As of 2022-01-24 21:16 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    In 2022, my preferred method to securely store passwords is:












    Results (65 votes). Check out past polls.

    Notices?