Beefy Boxes and Bandwidth Generously Provided by pair Networks Cowboy Neal with Hat
go ahead... be a heretic
 
PerlMonks  

Changing Perl Config settings

by Bloodrage (Monk)
on Feb 22, 2008 at 08:59 UTC ( #669488=perlmeditation: print w/ replies, xml ) Need Help??

In the Seeker of Perl Wisdom thread Changing Perl compile-time configuration settings I (indirectly) asked for the recommended method of changing Perl's compile time settings, i.e. those values given by the Config module that are allegedly set by the Configure and Makefile scripts immediatly prior to compling Perl. Apart from the monsterous man page for Config there is very little documentation on theses settings, and virtually none whatsoever on changing them. Enevitibly the recommendation is "configure, make, make install", all well and good when you have a trustworthy C++ compling environment, but alas I'm on Windows, and Windows dislikes GNU-style libraries. It's quite likely my Perl distribution was cross complied to Win32 from *nix anyway.

No. I'm not going to switch to Linux. It's not what I need Perl for. I need Perl everywhere, but that's for another Meditation, probably about running Win32 Perl off a flash drive.

Deciding the worst I could do was break my Perl installation... well a copy of my Strawberry Perl installation on my U3 flash drive. I wrote a program that's executed when I open up Command Prompt Portable U3 off my U3 drive. It goes through the three config files (CPAN's Config.pm, Perl's Config.pm and Config_heavy.pl) that seem to contain the relevant settings I wanted to change, and modifies them with basic regexes. Various sources warned me that the settings were also complied into the binary executable and that my efforts were futile. This was also reflected in ActiveState's implementation of reloc_perl.bat and ActiveState::RelocateTree to move your Perl installation to a new PREFIX path1, it's basically rather heavy handed search and replace over all the files in the Perl installation, and it also included diddling with the binary files which means you were out of luck if the new path was longer than the old path. Fortunatly it would appear that Perl on Win32 utilises environment variables and other techniques to resolve some of these options, hence my program is unlikely to break Perl or cause unusual behaviours2.

The resulting script works! I now get a completely different set of errors compling modules :) but I can see that it's more of a matter of setting up my C++ libraries better and possibly utilising PPM3 to get around particularly awkward modules. Basically it seems that the include and lib strings are either not magically passed to the compiler4, or these paths are incomplete and each sub-directory needs to be explicitly included in the various lib and include settings. *nix file systems probably get around this with symlinks or some other advanced file system magic. How did I figure that out? With my Perl-onna-stick I used cpan to install Glib which borked, the earliest error claimed that a certain header file was not found. This header file was clearly available in a subdirectory in my X:\GTK\include, adding an explicit path to this directory (X:\GTK\include\glib-2.0 I think) into my script, and install Glib gives a brand new set of errors... starting with another missing header file. This problem was partially solved by changing my program to retrieve the list of include paths from $ENV{'INCLUDE'} instead of manually crufting them into the program, this also means I now only need to maintain my %INCLUDE% environment variable. Remember this program does not stand alone, it requires the environment variables, especially $ENV{'INCLUDE'}, to be set correctly when the command prompt opens up.

Here's the program, it's been tidied up a lot since I started this meditation. It's a tad long so it's behind this

# No Shebang. Windows only script. use strict; use warnings; use File::Copy; # This config script is used to reconfigure a number of perl config fi +les, particularly CPAN's Config.pm, so that they point to the directo +ry structures on my U3 Flash drive. # Requires U3 commmand prompt which configures a bunch of really usefu +l U3 environment variables. # Version 0.01: Reconfigures CPAN's Config.pm to the 'local' path. # Version 0.02: Reconfigures Perl's Config.pm to the local path. # Version 0.03: Code cleanup. Move a few things to subs. reduced doubl +e looping. # Version 0.04: Reconfigures Config_heavy.pl # Version 0.05: Now obatains include paths from the INCLUDE environmen +t varaible, oh and some of the real idiot debugging code's gone. # Version 0.10: Final revised version. Less idiot debugging. more Perl +y style code. Freezing code until I fix includes etc. for compiling C +++ with minGW my $debug = undef; # Find, check and nofify U3_DEVICE_PATH my $u3drive = $ENV{'U3_DEVICE_PATH'} or die "HALT: U3_DEVICE_PATH is e +mpty or undefined!\n"; print "Starting U3 Strawberry Perl configuration on $u3drive\n"; # Set directories my $strawberry_dir = $u3drive."\\strawberry"; my $perl_dir = $strawberry_dir."\\perl"; my $cpanDir = $strawberry_dir."\\cpan"; my $cpanLibDir = $perl_dir."\\lib\\cpan"; my $cpanBuildDir = $cpanDir."\\build"; my $cpanSourceDir = $cpanDir."\\sources"; my $sysRoot = $ENV{"SYSTEMROOT"}; # include paths my $strawberry_include = $strawberry_dir."\\c\\include"; my @include = split ";", $ENV{'INCLUDE'}; # lib paths my $strawberry_lib = $strawberry_dir."\\c\\lib"; my $GTK_lib = $u3drive."\\GTK\\lib"; my $usr_lib = $u3drive."\\usr\\lib"; #Checking those directories are there for(($strawberry_dir, $perl_dir, $cpanDir, $cpanLibDir, $cpanBuildDir, + $cpanSourceDir, $sysRoot, $strawberry_include, $strawberry_lib, $GTK +_lib, $usr_lib, @include)){ -d or die "Error checking the directory $_ ($!)"; } my $perlConfig = $perl_dir."\\lib\\Config.pm"; print "Reconfiguring Perl, $perlConfig\n"; { # Closure used to limit scope... and my editor collapses them... my @content = load_array($perlConfig); backup ($perlConfig); open PERLCONFIG, ">", $perlConfig or die "HALT! Could not open $pe +rlConfig ($!)\n"; $strawberry_dir =~ s/\\/\\\\/g; # need to write \\ instead of \ for(@content){ s/.:\\\\strawberry/$strawberry_dir/g; print PERLCONFIG; } $strawberry_dir =~ s/\\\\/\\/g; # change it back close PERLCONFIG; } print "Done configuring Perl's Config.pm\n"; my $perl_heavy = $perl_dir."\\lib\\Config_heavy.pl"; # Config_heavy.pl is... large ~1200 lines of code and comments, print "Recongfiguring $perl_heavy\n"; { # Closure used to limit scope... and my editor collapses them... my @content = load_array($perl_heavy); backup($perl_heavy); open CONFIGHEAVY, ">", $perl_heavy or die "HALT! Could not open $p +erl_heavy ($!)\n"; my $heavy_include = join " ", @include; for (@content){ if (/^(glibpth|locincpth|xlibpth)=/){ s/'.*'/'$heavy_include'/; } if(/^strings=/){ my $strings_dir = $strawberry_include."\\string.h"; s/'.*'/'$strings_dir'/; } if(/^timeincl=/){ my $time_dir = $strawberry_include."\\time.h"; s/'.*'/'$time_dir'/; } if(/^sysman=/){ my $sysman_dir = $u3drive."\\usr\\man\\man1"; s/'.*'/'$sysman_dir'/; } if(/^loclibpth=/){ s/'.*'/'$strawberry_lib $GTK_lib $usr_lib'/; } s!.:(/|\\)strawberry!$strawberry_dir!g; print CONFIGHEAVY; } close CONFIGHEAVY; } print "Done configuring Config_heavy.pl\n"; my $cpanConfig = $cpanLibDir."\\Config.pm"; print "Reconfiguring CPAN, $cpanConfig\n"; { # Closure used to limit scope... and my editor collapses them... my @contents = load_array($cpanConfig); backup ($cpanConfig); # double those \\ to \\\\ for CPAN's Config.pm map s/\\/\\\\/g,($strawberry_dir, $perl_dir, $cpanDir, $cpanLibDir +, $cpanBuildDir, $cpanSourceDir, $sysRoot, $strawberry_include, $stra +wberry_lib, $GTK_lib, $usr_lib, @include); open CPANCONFIG, ">", $cpanConfig or die "HALT! Could not open $cp +anConfig ($!)\n"; for(@contents){ if (/build_dir\s/) { s/\[.*\]/\[$cpanBuildDir\]/; } if (/cpan_home\s/){ s/\[.*\]/\[$cpanDir\]/; } if (/\sftp\s/){ s/\[.*\]/\[$sysRoot\\\\system32\\\\ftp.exe\]/; } if (/histfile\s/){ s/\[.*\]/\[$cpanDir\\\\histfile\]/; } if (/keep_source_where\s/){ s/\[.*\]/\[$cpanSourceDir\]/; } if (/\smake\s/){ s/\[.*\]/\[$strawberry_dir\\c\\bin\\dmake.exe\]/; } if (/make_install_arg\s/){ s/\[.*\]/\[PREFIX=$perl_dir]/; } if (/makepl_arg\s/){ #Note: This one might need to be modified + further my $cpan_include = "-I".join " -I", @include; s/\[.*\]/\[PREFIX=$perl_dir LIBS=$strawberry_lib INC="$cpa +n_include"\]/; } if (/pager\s/){ s/\[.*\]/\[$sysRoot\\\\system32\\\\more.com]/; } if (/prefs_dir\s/){ s/\[.*\]/\[$cpanDir\\\\prefs]/; } if (/shell\s/){ # This one probably needs to be changed to use + the U3 command prompt s/\[.*\]/\[$sysRoot\\\\system32\\\\cmd.exe]/; } print CPANCONFIG; } close CPANCONFIG; # Undo the doubleup map s/\\\\/\\/g,($strawberry_dir, $perl_dir, $cpanDir, $cpanLibDir +, $cpanBuildDir, $cpanSourceDir, $sysRoot, $strawberry_include, $stra +wberry_lib, $GTK_lib, $usr_lib, @include); } print "Done configuring CPAN\n"; print "Done configuring U3 Strawberry Perl\n"; # Subroutines only after this # Returns a files contents in an array. No chomping we need those newl +ines. sub load_array{ my $file_name = shift; open THEFILE, "<", $file_name or die "HALT! Could not open $file_n +ame ($!)\n"; my @content = (); push @content, $_ while (<THEFILE>); close THEFILE; @content; } # Back it up! If there is no file appended .orig back it up to there, +otherwise back it up to .last sub backup { for my $original (@_){ my $backup = $original.".orig"; if (-e $backup){ $backup = $original.".last"; print "$backup of original exists, using \"$backup\" inste +ad.\n" if $debug; } else { print "$backup doesn't exist, assuming $original is the or +iginal.\n" if $debug; } unlink $backup; # yes, we don't care if this fails. copy $original, $backup or die "HALT! Could not copy to $backu +p ($!)\n"; } } }

These are the questions from this project:

  • Why are these config files full of Unix style paths that don't exist in the Windows installation?
  • If Windows versions of Perl makes an exception for these shouldn't it be documented, ideally in the man pages or directly in the config files?
  • Are these hidden paths part of what makes compliling Perl modules painful in windows? (testing with Glib and GTK indicate yes!)
  • Is this a good method of changing these Perl settings? Since no-one has suggested a Perl::Reconfig module, I suspect it may be the only option, except for Adamk's mentioned below.

Adamk is trying a different method, making self configuring scripts, which I think has potential too. I'd like to read a meditation on that sometime.


Updated: This is a semi-final version which will remain unchanged until I explore flattening my include library and using the PPM module... unless I need to reconfigure PPM's config file too.

1 Props to CountZero for pointing out reloc_perl.bat and ActiveState::RelocateTree first.

2 Thanks again to Corion for pointing this out.

3Thanks to syphilis this time

4Thanks again to syphilis for this one too.

Comment on Changing Perl Config settings
Select or Download Code
Re: RFC: Changing Perl Config settings
by Corion (Pope) on Feb 22, 2008 at 09:07 UTC
    Are these settings really complied into the executable?

    On Windows, they aren't, because on Windows, it's possible for an executable to find out where its file lives. Perl uses that capability of Windows to populate its @INC.

    Are these hidden paths part of what makes compliling Perl modules painful in windows? (testing with Glib and GTK indicate yes!)

    These are the same hidden paths that make compiling libraries painful everywhere. You need to set up $ENV{INCLUDE} and $ENV{LIB} correctly. For Strawberry 5.10.0, I attach the path.cmd file I use to set up the environment for it below.

    I think I've made a flash drive with a portable Perl installation ...which totally subverts a Windows PC's own Perl installation...

    Which is how I manage the various versions of Perl I have on various machines too. Every Perl directory has a path.cmd which sets up the environment so that perl.exe is the "correct" Perl for that application and all other tools are also the correct versions. I use different versions of Perl for historical reasons, mostly because the applications using it Just Work and I don't want to invest the time to consolidate them all to a single Perl.

    set BASE=%~dp0 path %BASE%perl\bin;%BASE%c\bin;%PATH% set INCLUDE=%BASE%c\include;%INCLUDE% set LIB=%BASE%c\lib;%LIB%
      Snap. This is my commandprompt.bat that's executed when I open my command prompt. As you can see it's all there. U3_DEVICE_PATH is set by Command Prompt Portable, and is the current drive letter of my flash drive.
      @echo off color 07 prompt $p$g title Command Prompt Portable cls ver echo Setting Path to other files path %U3_DEVICE_PATH%\usr\bin;%PATH% set LIB=%U3DEVICE_PATH%\usr\lib;%LIB% set INCLUDE=%U3DEVICE_PATH%\usr\include;%INCLUDE% echo Setting Path to Strawberry Perl path %U3_DEVICE_PATH%\strawberry\c\bin;%U3_DEVICE_PATH%\strawberry\per +l\bin;%PATH% set LIB=%U3_DEVICE_PATH%\strawberry\c\lib;%U3_DEVICE_PATH%\strawberry\ +perl\bin;%LIB% set INCLUDE=%U3_DEVICE_PATH%\strawberry\c\include;%U3_DEVICE_PATH%\str +awberry\perl\lib\CORE;%INCLUDE% echo Setting Path to GTK+ path %U3_DEVICE_PATH%\GTK\bin;%PATH% set LIB=%U3_DEVICE_PATH%\GTK\lib;%LIB% set INCLUDE=%U3_DEVICE_PATH%\System\GTK\INCLUDE;%U3_DEVICE_PATH%\Syste +m\GTK\INCLUDE\GTK-2.0;%U3_DEVICE_PATH%\System\GTK\INCLUDE\GLIB-2.0;%U +3_DEVICE_PATH%\System\GTK\INCLUDE\PANGO-1.0;%U3_DEVICE_PATH%\System\G +TK\INCLUDE\CAIRO;%U3_DEVICE_PATH%\System\GTK\INCLUDE\ATK-1.0;%U3_DEVI +CE_PATH%\System\GTK\INCLUDE\GTKGLEXT-1.0;%U3_DEVICE_PATH%\System\GTK\ +LIB\GTK-2.0\INCLUDE;%U3_DEVICE_PATH%\System\GTK\LIB\GLIB-2.0\INCLUDE; +%U3_DEVICE_PATH%\System\GTK\LIB\GTKGLEXT-1.0\INCLUDE;%U3_DEVICE_PATH% +\System\GTK\INCLUDE\LIBGLADE-2.0;%U3_DEVICE_PATH%\System\GTK\INCLUDE\ +LIBXML2;%INCLUDE% set GTK_BASEPATH=%U3_DEVICE_PATH%\GTK set PKG_CONFIG_PATH=%U3_DEVICE_PATH%\GTK\lib\pkgconfig echo Setting Path to Cygwin path %U3_DEVICE_PATH%\cygwin\bin;%PATH% echo Update CPAN config file perl %U3_DEVICE_PATH%\system\bin\u3perlfig.pl cd\
      A simple test script tells me that my $ENV{'INCLUDE'} environment variable is full of directories, which aren't being passed to the compiler via Perl or CPAN... ...I think I need to rewrite my script to build an @include from $ENV{'INCLUDE'}, but this is why I needed meditate on it :)
      use strict; use warnings; my @include = @INC; print '@INC =('.(join ",", @include).")\n\n"; print '%INCLUDE%=('.$ENV{'INCLUDE'}.")\n";
      Gives the following output:
      I:\Documents\Perl\U3 Perl Config scripts>perl testenv.pl @INC =(I:/strawberry/perl/lib,I:/strawberry/perl/site/lib,.) %INCLUDE%=(I:\System\GTK\INCLUDE;I:\System\GTK\INCLUDE\GTK-2.0;I:\Syst +em\GTK\INC LUDE\GLIB-2.0;I:\System\GTK\INCLUDE\PANGO-1.0;I:\System\GTK\INCLUDE\CA +IRO;I:\Sys tem\GTK\INCLUDE\ATK-1.0;I:\System\GTK\INCLUDE\GTKGLEXT-1.0;I:\System\G +TK\LIB\GTK -2.0\INCLUDE;I:\System\GTK\LIB\GLIB-2.0\INCLUDE;I:\System\GTK\LIB\GTKG +LEXT-1.0\I NCLUDE;I:\System\GTK\INCLUDE\LIBGLADE-2.0;I:\System\GTK\INCLUDE\LIBXML +2;I:\straw berry\c\include;I:\strawberry\perl\lib\CORE;\usr\include;C:\System\GTK +\INCLUDE;C :\System\GTK\INCLUDE\GTK-2.0;C:\System\GTK\INCLUDE\GLIB-2.0;C:\System\ +GTK\INCLUD E\PANGO-1.0;C:\System\GTK\INCLUDE\CAIRO;C:\System\GTK\INCLUDE\ATK-1.0; +C:\System\ GTK\INCLUDE\GTKGLEXT-1.0;C:\System\GTK\LIB\GTK-2.0\INCLUDE;C:\System\G +TK\LIB\GLI B-2.0\INCLUDE;C:\System\GTK\LIB\GTKGLEXT-1.0\INCLUDE;C:\System\GTK\INC +LUDE\LIBGL ADE-2.0;C:\System\GTK\INCLUDE\LIBXML2;c:\system\usr\include)

      I'm not entirely sure what this is telling me about the relationship of @INC and $ENV{'INCLUDE'}. One seems to be where Perl looks for Perly libraries... which may be set wrong, and the other is as I expect it to be configured and should point the compiler to all the appropriate include directories.

      Note: The I:\ prefixed paths have beens set up by my command prompt script, while the C:\ based paths are from my computer's original environment variables.

        @INC is where Perl looks for *.pm files.

        $ENV{INCLUDE} is where the C compiler looks for *.h and *.c files.

        The two bear little to no relation to each other.

      set INCLUDE=%BASE%c\include;%INCLUDE% set LIB=%BASE%c\lib;%LIB%


      That's interesting. I've noticed that Strawberry sets those two environment variables, but I've never understood why. Afaict, with MinGW (including Strawberry's MinGW), the compiler and linker search those locations by default. In fact, I've unset those 2 envvars, and everything still works fine:
      C:\_32\C>set LIB LIB= C:\_32\C>set INCLUDE INCLUDE= C:\_32\C>type try.c #include <stdio.h> int main(void) { printf("Hello from strawberry\n"); return 0; } C:\_32\C>gcc -o try.exe try.c C:\_32\C>try Hello from strawberry C:\_32\C>
      Similarly, if I'm building a perl extension using Strawberry Perl, the standard headers and libraries get found fine - even though those 2 envvars are unset. Perhaps those 2 envvars could be useful when it comes to finding headers and libraries that are stored in non-default locations. (I haven't tried that. I prefer to keep LIB and INCLUDE unset, and provide the non-standard locations with the -I and -L switches.)

      Update: I've since found time to take a closer look - and I can find no evidence that MinGW and Strawberry Perl take any notice at all of the INCLUDE and LIB environment variables. Visual Studio certainly does ... but not MinGW.

      Cheers,
      Rob

        Fantastic, so I wasn't imagining things when I thought things weren't working as they should. It looks like I need to spend some time getting MinGW to compile some things outside the CPAN shell. It also explains why my program helps, because it explictly sets the -I and -L switches when it reconfigures CPAN's Config.pm.

        I'm now considering the benefits of flattening my include directory structures into X:\strawberry\c\include rather than to have one for a) strawberry, b) GTK/Glade/Glib/Cairo... and c) a usr/include dir for 'one off' libraries. The current multiple include locations has the advantage of being able to update one without breaking the others, where the flattened version may result in overwriting 'good' with 'bad' when installing new material.

        PS: I think I can circumvent the difficulty compiling GTK2-Perl by manually installing the ActivState PPM... except I don't want this compiling issue to continue being a PITA when in the field with this portable Perl-onna-stick thing.

Re: RFC: Changing Perl Config settings
by Bloodrage (Monk) on Feb 25, 2008 at 22:21 UTC

    I've meditated enough, I've figured out what to do next. It looks like I need to get PPM working in the interim, look at flattening my include libraries, and do some work with compiling with minGW outside the CPAN shell. So I've updated the original post to integrate the most excellent comments.

    Gratias sodales ab camelus

    Update: here's what I found on PPM.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://669488]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (13)
As of 2014-04-18 15:13 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (469 votes), past polls