Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

When modules install in perl5

by cristofayre (Sexton)
on Sep 25, 2017 at 12:00 UTC ( #1200039=perlquestion: print w/replies, xml ) Need Help??

cristofayre has asked for the wisdom of the Perl Monks concerning the following question:

My shared server has the usual "!#/usr/bin/perl" for use by everyone, but personal installations appear to go into "perl/usr/lib/perl5" folder.

So if my scripts access the former, all is well. But to access the latter ...??? Have I got to change all my scripts to the new path of "!# perl/usr/lib/perl5" or is there a way to 'force' the new path into @inc. How would you do it if some "use xxx" are in /usr/bin/perl, and some in the second folder?

Did my last host use a "symlink" for this, and if so, where did he put it.

Finally, if the "use" entry could not be found, would this generate a "suexec violation" error? A script using "File::Find::Rule", permission 755 designed to update all the file permissions came back with the above for a 500 error. I asked host, and their response: "Please set all your perl files to 755" (In otherwords, they had no idea!)

This is the script:

#!/usr/bin/perl print "content-type: text/html\n\n"; # use CGI::Carp qw( fatalsToBrowser ); use File::Find::Rule; my @files = File::Find::Rule->file()->name('*.pl')->in('/home/cristofa +/public_html'); # set ->in('.') to start from current directory for ($x=0; $x<@files; $x++){ chmod (0755, $files[$x]); print "$files[$x]&lt;br&gt;"; } print "All done";

# CARP is commented out because - whilst CGI is installed - it doesn't specifically list CGI::CARP. I installed the latter ... which comes back to the initial "perl5" path query! Script works on my Windows / Strawberry perl version.

And yes, I know you CAN use a foreach / while loop ... but I still find that a bit 'symbolic' and prefer to 'see' what's happening each loop.

Replies are listed 'Best First'.
Re: When modules install in perl5
by haukex (Bishop) on Sep 25, 2017 at 12:54 UTC

    Note that the shebang line ("#!...") specifies which perl binary should get executed when the script is run from the command line via /path/to/ (and in a few other cases). Different perl binaries usually have completely different module installation directories, i.e. completely different sets of @INC. You can see the @INC directories with the commands perl -V or perl -le 'print for @INC', and you'll see that typically, none of those are actually under /usr/bin/perl (although there might theoretically be some exotic installations where that's not true).

    I'm a little unclear on whether your question is about two different installations of Perl, each with their own sets of modules, or whether it is about a single Perl installation that you want to add another module directory to. It sounds to me like it's the latter, but just in case I'll briefly talk about the former first.

    If you use a shebang line of "#!/usr/bin/env perl", as I usually do, then the env tool will use whatever perl comes first in your PATH environment variable, which is the same thing as your shell would do when you type perl at the command line. So then it's only a matter of managing the PATH variable, which users would normally do in their ~/.profile or equivalent. Users can also install their own entire version of perl (usually into their home directories) manually or using tools such as perlbrew or plenv, which will then take care of managing the different installations. However, when scripts get run by a webserver, the server will typically set up its own environment, including its own PATH. This means this method may not work for CGI scripts - I've found that when you want a script to use a specific installation of Perl, it's usually easiest to hardcode that into the shebang line, i.e. "#!/absolute/path/to/perl" - note how I'm using an absolute path there. This will usually make sure that a webserver will run the script with that perl binary.

    How would you do it if some "use xxx" are in /usr/bin/perl, and some in the second folder?

    Two different installations of Perl would typically not share modules, and this is a good idea because modules compiled for different versions of Perl will often be incompatible! If you want the same modules available in both Perls, it's best to install them to both.

    Now, as for a single installation of Perl with different module installation directories, there are several ways to tell Perl where to look for modules aside from the default locations it was compiled with, in other words, how to modify @INC.

    So which one should you use? Often, lib is easiest, unless you have a lot of scripts that you need to modify, and/or you have installed a bunch of modules into one central location that all of your scripts need access to. I've found that usually, PERL5LIB is easiest for "global" changes, since setting environment variables can also be limited to a per-user basis if desired, and you can usually configure your webserver to supply additional environment variables to CGI scripts. The things to be careful about is that if there are multiple places in the system that set PERL5LIB, that they don't clobber each others' changes, and if you have multiple Perl installations, that you don't accidentally point the currently used Perl at the modules for a different version!

    Note that it is also possible for users to install modules into their home directories. In this case, they can use perl -I..., PERL5LIB, and lib to configure Perl to look for modules in their directories. There are also tools to make this easier, such as local::lib.

    Sorry, I don't know much about suexec - if you're having a problem there, please post a more specific error message with instructions how to reproduce. As for permissions, your CGI scripts would typically have 0755, but that's not required for modules (*.pm files), there 0644 is enough.

    Update: I don't think this is the source of your problems, but for the sake of completeness, note that @INC used to include the current working directory ".", but as of Perl v5.26 it does not. (Also made a few more very minor ninja edits.)

    Update 2: Although I'm going a bit overkill on the "for the sake of completeness", here is a new thread on using FindBin for library paths that are always in the same place relative to the location of the script file. (Also added the sentence about not clobbering the env var.)

      The way I read this question, the PERL5LIB environment variable should be the way to go, and perhaps be the very first thing to try. Just append the user-specific local library folder to the existing value, if it's already set.

        All is well. It finds the scipts in the second folder whilst the paths to full installation stays at "urs/bin/perl", so it must have auto undated the path to folder when CPAN installed

      Gosh. Thanks for the long reply.

      It's actually one installation of PERL on the shared server, but when a user uses the cPanel to install a "private" module, (such as PDF::AP12) it gets places within a "perl/perl5" of that user.

      On cpanel instructions, it says use "usr/bin/perl" or "usr/bin/perlml"

      But it's academic. The script finds my installed modules within the other folder, and works fine. (Mind you, I didn't help the situation by unloading the files as DOS! Whereas "Filezilla" will auto correct, appears cpanel Filemanager does not!) (Now tring to find a way to recognise the CRLF at the end of scripts

      I used "File::Find::Rule" to reiterate through all the *.pl files on the server, then chmod to set to 705. Can't believe it took less than a second to go through 200+ files in numerous sub domains to set them all! (I mention it as I'm NOW writing a script to open first line of files, check end, and if wrong, THEN open the entire file, do substitute, and resave ... if I can get grep to recognise the \n\r !!
Re: When modules install in perl5
by holli (Abbot) on Sep 25, 2017 at 12:19 UTC
    use lib "/wherever/your/modules/are/lib"; use Something; #will look first in /wherever/your/modules/are/lib
    See lib.


    You can lead your users to water, but alas, you cannot drown them.
Re: When modules install in perl5
by pryrt (Monsignor) on Sep 25, 2017 at 13:14 UTC

    My shared-hosting provider uses cPanel and allows installation of modules (including modules that require compilation) thru the cPanel interface (or manually thru ssh access) into the ~/perl5/ hierarchy. In my host's cPanel help describing how to install and access local modules, it explains that I need to either use the shebang #!/usr/bin/perlml, which is set up to automatically look in the locally-installed directory for modules, or to use cPanelUserConfig;, which also tells perl where to find local modules. Your host may be different, but my guess is that they have documentation for how they expect you to install and access locally-installed perl modules.

      Yep, those are the exact lines that are confusing me. In the end, I didn't have to use either in my scripts. Just used "#!/usr/bin/perl" ... and it still found files that CPAN installed in the "~/perl5" folder heirarchy. So what this actually relates to I do nt know. Still, it works. That's all I'm worried about!

Re: When modules install in perl5
by Anonymous Monk on Sep 25, 2017 at 12:30 UTC
    /usr/bin/perl might be the location of the Perl executable, but that executable is tiny. Most of the actual software resides in various libraries, accessed by the current PERL5LIB environment-variable settings as well as a list that is compiled into Perl. The aforementioned perldoc lib is a good place to start to find an explanation of how it all works. Also Google "how to install Perl as a non-root user" (... "on a shared-hosting service").
Re: When modules install in perl5
by cristofayre (Sexton) on Sep 25, 2017 at 12:33 UTC

    Just wondering.

    Since the files to be changed are preset to "644", this would imply they are readable, but not writable or executable ... so in essence, they cannot be modified by an external script, which is where the violation is coming in.

    Maybe the "umask" command set to "000" would temporarily set all the files to "777", and then chmod can change them back to "755" or "705"

      You should consider getting a cloud VM instead of a shared hosting account. You are only seeing the tip of the iceberg as far as the problems you will encounter. You won't be able to install any Perl modules that require compilation, or C libraries that Perl modules need; you won't be able to use a modern version of Perl as some modules require; you can't get on the command line to do normal tasks like changing file permissions (with or without a Perl script to do it), and are forced to try to hack CGI scripts for such work; etc., etc.

      For $10 a month or less you can have your own server with full root access. The time it will take you to learn simple server administration will be less than the time you will spend faffing around in "support" chatrooms with your current host's know-nothing CSRs and do-nothing sysadmins. And you'll be able to set up your working environment as well as your web server just how you like it.

      Personally I use Linode but have heard good things about Digital Ocean as well. Hope this helps!

      The way forward always starts with a minimal test.

        I've used Scaleway without any issues. Starting at 3 euro a month you can have a Dual Core x86 VPS, or 4 dedicated arm core system

      For anyone who's curious, the REAL problem was that the file had been uploaded as DOS (CRLF) rather than UNIX (LF only) "Filezilla" auto corrects on upload - but not so cPanel "FileManager"

      Now I got to write a script to check 200+ files on my Windows7 / Strawbery PERL local machine. I will use File::Find::Rule to reiterate through all the folders and sub domains, and open the first line of each file. If it's a CRLF, THEN I will open the file. make the change, and save it out again. Problem at present is getting grep to find the linefeed. Tried escaping and /\x0A\x0D/, but won't play ball!

      #!/usr/bin/perl print "content-type: text/html\n\n"; use CGI::Carp qw( fatalsToBrowser ); use File::Find::Rule; $string="\x0D\x0A"; #my @files = File::Find::Rule->file()->name('*.pl')->in('/home/cristof +a/public_html/'); # set ->in('.') to start from current directory my @files=('D:/home/cristofa/public_html/cgi-bin/dummy/'); for ($x=0; $x<@files; $x++){ open THEFILE, "<$files[$x]"; $first_line = <THEFILE>; print $first_line; close THEFILE; if (grep(/$string$/,$first_line)){ print File "$files[$x] is incorrect<br>"; # &correct_file; } } sub correct_file{ open (FILE, "<$files[$x]); while(<FILE>){ $tmp.=$_; } $tmp=~s/[\n\r][\n\r]/\n/g; open (FILE, ">$files[$x]); print FILE $tmp; close(FILE); $tmp=''; } print "All done";
      It is only checking the single test file at this stage, which should print "File XYZ is incorrect" if its working

        Since this is a different topic than the one in the root node, if you have more issues/questions, I recommend starting a new thread. There are a few issues with the code, and potential improvements:

        • Use strict and warnings! For example it would have given you a hint that you have a typo in the line print File "$files[$x] is incorrect<br>"; - in fact this is probably one of the issues that is making you think the regex isn't working, when it actually is.
        • You need to check the return value of open for errors, and I'd strongly recommend to use lexical filehandles and the 3-argument form, as in open my $fh, '<:raw', $files[$x] or die "$files[$x]: $!";
        • You should open your files with the :raw layer as I showed above, or binmode them after opening (as the AM post points out), because otherwise, under Windows, your files will be opened with the :crlf layer by default (see PerlIO), another reason your test code might not be working.
        • You've also got a typo in the two open (FILE, "<$files[$x]); lines (again, Use strict and warnings).
        • When reading the entire file into memory, you don't need to read it in a loop, there is a trick to slurp the entire file into memory at once: my $tmp = do { local $/; <$fh> };

        Personally I find the regex s/[\n\r][\n\r]/\n/g a little too flexible: If the file for some reason contains mixed CR/LF line endings, this regex might muck them up even more. Plus, if there are mixed endings, just checking the first line isn't enough. Here's how I might have written a script like this, even though there is no need to reinvent this wheel - for example, the fromdos tool from the Tofrodos package could already be installed on your system.

        #!/usr/bin/env perl use warnings; use strict; use Getopt::Std 'getopts'; our $VERSION = '0.01'; $Getopt::Std::STANDARD_HELP_VERSION = 1; getopts('qi', \my %opts) or die "Usage: $0 [-q] [-i] FILE(s)\n"; if ( !@ARGV || (!$opts{i} && $opts{q}) ) { warn "No actions to perform\n"; exit 1 } for my $file (@ARGV) { open my $ifh, '<:raw', $file or die "$file: $!"; my $data = do { local $/; <$ifh> }; close $ifh; if (not $opts{q}) { my $cr = () = $data=~/\x0D(?!\x0A)/g; my $lf = () = $data=~/(?<!\x0D)\x0A/g; my $crlf = () = $data=~/\x0D\x0A/g; print "$file: ", ( join(', ', $cr?"$cr CR":(), $lf?"$lf LF":() +, $crlf?"$crlf CRLF":() ) || 'no CR or LF' ), "\n"; } if ($opts{i}) { $data =~ s/\x0D\x0A?|\x0A/\n/g; open my $ofh, '>:raw', $file or die "$file: $!"; print $ofh $data; close $ofh; } }
        make sure to binmode the files, and use \x0D\x0A instead of \r\n to play it extra safe
Re: When modules install in perl5
by Anonymous Monk on Sep 25, 2017 at 18:47 UTC
    The biggest problem with shared-hosting is your neighbors. You might be surprised at just how much stuff that you can see in other people's home directories ... and in any case you really can't conceal any of it (including things like, say, database passwords) because the Apache user must have access to your stuff in order to run your scripts. This basically means that everyone else does, too. However, if your virtual-machine hosting service "conveniently" provides something like Plesk, you could be screwed anyway, because that "convenient" software is very easily compromised and usually is. To have any hope of security you must take the entire matter into your own hands and eschew "convenience."
Re: When modules install in perl5
by cristofayre (Sexton) on Dec 03, 2019 at 18:54 UTC

    Not returned to this comment for a while. No, it's ONE perl installation: Strawberry Perl. When it runs, it uses perl.exe located at "C:/Strawberry/Perl/bin" But when it runs the cpan module, it installs the files in "C:/Users/(my_name)/perl5/lib/perl5" And when the script runs ... it cannot find where it has installed the CPAN modules!! You would think the module would install the files, and respective paths to that folder were updated!

    Edit: erzuuli restored content

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1200039]
Front-paged by Corion
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (6)
As of 2021-06-25 12:54 GMT
Find Nodes?
    Voting Booth?
    What does the "s" stand for in "perls"? (Whence perls)

    Results (136 votes). Check out past polls.