Table of Contents

0. Why Upgrade?
1. Make life with the CPAN module easier
2. Create A Bundle of Existing Modules
3. Update your port
4. Install the New Perl
5. Keep old perl active while you finish working
6. Install Updated Modules
7. Some notable workarounds
8. Verify the work
9. Some CGI.pm issues
10. Update system to use perl port
11. Reality Check Projects
12. Be Prepared to Revert
13. Some Other Perl 5.8 nodes of interest

0. Why Upgrade?

I'd been using Perl 5.005 since it was released and was very satisfied with it, in terms of features and functionality, stability and performance and lack of bugs that affected me. Primarily for these reasons, we never upgraded our servers to Perl 5.6.x when they were released.

However, Perl on our system is primarily used for CGI programming, commonly with a database backend. The CGI and DBI modules are core to our use of Perl at Summersault. The most recent versions of CGI.pm now include a "require" statement for Perl 5.6, and the Changes file for DBI 1.30 threatens to require at least Perl 5.6 soon. These occurances provide two reasons to upgrade. First to keep to up to date with modules that are core to our usage of Perl. Secondly, these are strong endorsements for upgrading by module authors I respect.

It also didn't seem likely that it would be reasonable to hold out and skip the whole the 5.8 series without upgrading from 5.005, if only because modules would increasingly require a version of Perl newer than what we were running. Since I'd heard good things about 5.8.0 being a very stable, quality release, my feeling was that this could be a good way to skip the hassle of first upgrading to 5.6 and possibly be "set" for a while, possibly until Perl 6. Now we've been running Perl 5.8 for a couple of weeks, and have noticed no glitches other than the things noted in the upgrade process below, which don't appear to be cauesd by bugs in Perl from what I can tell.

1. Make life with the CPAN module easier

The CPAN module will be used a lot in this process to download and install new modules. Here are some tips that worked well to keep it functioning smoothly.

• Make sure tar points to gnutar. locate your CPAN/Config.pm file and update the line for tar like this:

   'tar' => q[/usr/local/bin/gtar],

  You may need to install gtar from a package or port if you don't have it already. I got some errors until I did this.

• Avoid Net::FTP and LWP modules. On my system these two modules were slow to fetch files or just didn't work. Instead of having CPAN.pm try them each time, I updated my CPAN/Config.pm file to avoid loading them:

  'dontload_hash' => {"Net::FTP" => 1, "LWP" =>1  },

• Increase speed by caching metadata. Add this to your CPAN/Config.pm file to do that:

   cache_metadata=>,

  Perl 5.8 may actually complain if this hash key is missing.

2. Create A Bundle of Existing Modules

To make sure the new version of Perl has all the same modules installed as the last version, the first step is create a bundle file of all the modules that are currently installed. This will allow you to easily re-install the latest versions of these modules to work with the new version of Perl.

# As root
$ /usr/bin/perl -MCPAN -e autobundle
    
[download]

That will create a file with a name like "Bundle/Snapshot_2002_07_23_00.pm" that you can use to re-install the modules from. This file will be created in your .cpan directory, which is in /root by default.

3. Update your port

Next you will need an up-to-date version of the Perl 5.8 port for FreeBSD. There are a number of ways to update your ports, here's how I updated mine:

$ cd /usr/ports/lang
$ mkdir -p perl5.8/files
$ cd perl5.8
$ ftp ftp://ftp.freebsd.org/pub/FreeBSD/branches/4.0-stable/ports/lang
+/perl5.8/
# at this point I enter the FTP shell
> prompt
> mget *
> lcd files
> cd files
> mget *
> exit
$ cd ..
[download]

4. Install the New Perl

Actually installing Perl is easy enough:

$ cd /usr/ports/lang/perl5.8/
$ make install
</pre>
    
[download]

This finished in about an hour on my 233 Mhz machine with 256 MB RAM and a SCSI hard drive. YMMV.

5. Keep old perl active while you finish working

The installation process will install about 20 binaries in /usr/local/bin related to then new version of Perl. Since some software may find these versions of perl binaries before they look in /usr/bin, I wanted to move these versions out of the way and symlink these binaries back into /usr/bin/ while I finished the upgrade. I found a definitive list of the binaries installed like this:

        pkg_info -L perl-5.8.0* | grep local/bin | perl -pe 's!/usr/lo
+cal/bin/!!'>perl_binaries.txt
    
[download]

With this, I used a little shell scripting to generate the commands I needed to systematically move and symlink all these binaries:

for f in `cat perl_binaries.txt`; { 
    echo "mv /usr/local/bin/$f /usr/local/bin/$f.copy; ln -s /usr/bin/
+$f /usr/local/bin/$f";
}
         
[download]

That will print out the commands but not actually run them for you.

6. Install Updated Modules

Now it's time to re-install all your old modules for the new version of Perl. To do this, your CPAN/Config.pm file needs to be created or updated for 5.8. Since I already had one that worked from my last installation, I just re-used it, like this:

cp /usr/libdata/perl/5.00503/CPAN/Config.pm /usr/local/lib/perl5/5.8.0
+/CPAN/Config.pm
    
[download]

Once that is done, you can proceed to installing the modules. You will need to be in the directory where your "Bundle" directory is located that you created above. (/root/.cpan by default) Note: It's important to make sure you are using the new version of Perl when you do this.

/usr/local/bin/perl58.copy -MCPAN -e 'install Bundle::Snapshot_Your_Na
+me_Here' | tee perl_module_upgrade_results.txt
    
[download]

Depending on how many modules you need to upgrade and your net connectivity, this could take several hours to complete. Some modules may ask for interactive feedback during the install process, so it's important to actively watch what's happening. For that reason, I use tee above so that I can work with the program interactively and still has a log of what happened for future reference.

I ended up running this command several times, as I would review the output, fix some modules, and try again. Each time I changed the name of the output file, so I could compare the various results if I needed to.

In my case the system tried to install a mod_perl module which wanted to know where my Apache source tree was, which didn't exist. I was able to use Control-C to cause that module to forcefully fail, while continuing on with the rest of the modules. (which was fine in my case, since I didn't need the mod_perl module). Other modules may fail without any interaction. In my case, the DBD::Pg module complained about not being able to find the Postgres libraries in environment variables. (hint to DBD::Pg users: define POSTGRES_LIB AND POSTGRES_INCLUDE before you start).

For cases like that, it will be important to analyze the resulting log to see what modules, if any, need some manual intervention to be installed properly. You can look for this kind of wording in the

perl_module_upgrade_results.txt<code> 
            file:
            <p>
            <code>
Running make test
  Make had some problems, maybe interrupted? Won't test
Running make install
  Make had some problems, maybe interrupted? Won't install
Bundle summary: The following items in bundle Bundle::Test had install
+ation
problems:
  DBD::Pg
  </pre>
  
[download]

7. Some notable workarounds

Here are some patterns that were used when getting troublesome modules to install under Perl 5.8.

• Using FreeBSD ports. In some cases there was a FreeBSD port of the module which would install cleanly. I had to workaround one catch though: By default, the ports tree uses the active Perl in the system, and 5.8 was not yet active. I used the following hack to make the ports system work with Perl 5.8 in this case. This works fine, but there's probably a better way to do this. I'd like to hear about it if you know. :) I added the following lines to the bottom of each Makefile for the perl ports I wanted to install:

  PERL_VERSION=5.8.0
  PERL_VER=5.8.0
  PERL5=/usr/local/bin/perl58.copy
  PERL=/usr/local/bin/perl58.copy
  PERL_ARCH=mach
          
  [download]

  My list of modules that need to be installed from ports included:
  • IPC::Shareable
  • Image::Magick
  • GIFgraph
  • p5-libwww
  If you upgrade Image::Magick from the port, be sure first to upgrade for the older Perl, and then again for the 5.8 perl, so the projects depending on the older perl don't break until you switch to 5.8.
• Use the POLLUTE option. Postgres.pm failed to "make" with many errors and needed to be installed for 5.8 to support an aging project. Manually making the port worked when the POLLUTE option was added:

              perl58.copy Makefile.PL
              make install POLLUTE=1
              
  [download]

  This option is explained in the ExtUtils::MakeMaker perldocs as follows:

         POLLUTE
           Release 5.005 grandfathered old global symbol names by
           providing preprocessor macros for extension source
           compatibility.  As of release 5.6, these preprocessor
           definitions are not available by default.  The POLLUTE
           flag specifies that the old names should still be
           defined:
  
             perl Makefile.PL POLLUTE=1
  
           Please inform the module author if this is necessary to
           successfully install a module under 5.6 or later
          </pre>
      </blockquote></li>
  
          <li><b>GnuPG</b>. GnuPG needed to  be patched to <code>make
  [download]

  . My bug report and patch for this available here.

• Image::Magick, after upgrading, it was noticed that while our Image::Magick installation was generally working, writing out PNG files was failing in at least one case. The syntax being used looked like this:

                  $img->Write(file=>\*IMAGE,filename=>$filename);
              
  [download]

  This was fixed by updating the code to just use the "filename" key:

                  $img->Write(filename=>$filename);
  
              
  [download]

8. Verify the work

Now I wanted to verify two things about the modules I had installed for Perl 5.8:

• Compared with the modules I had installed for 5.5, did the differences look reasonable?
• All the modules installed for 5.5 that could pass a basic use Module loading test could still pass the same test for 5.8. (And the modules that couldn't pass this test for 5.5, well, those are cruft).

To prepare for both of these verifications, I wanted a basic list of all the modules installed for both versions. The Bundle files I had already created were a close approximation of this, but they included version numbers which I didn't want, since many of the modules were updated as part of the upgrade. Here's how I did that:

1. I made a copy of the 5.5 Bundle file and used vim to strip out all the text in the file above and below the module list. I named this 5.5-Bundle.modified
2. I created the final list with this command:
   perl -ane 'print "$F[0]\n" if (/\S/);' 5.5-Bundle.modified|sort >5.5-bare-module-list.txt
3. I then repeated the first two steps for the 5.8 Bundle I had.

Now to address my first verification test, I wanted to look at the modules that just appeared in the 5.5 list. Anything in this should either be something I've abandoned, is obsolete, or a mistake. Creating this report was easy enough:

diff 5.5-bare-module-list.txt 5.8-bare-module-list.txt | grep '>' | te
+e only-in-5.5.txt | more
[download]

Creating the "use module"     verification took a little more effort. First, I didn't wanted to load every module at the same time for fear of the memory consumption. Also, I needed to run the test for all the 5.5 modules as well, and have a good way way to compare the two result sets. I decided to produce a report that gave the module name, an arbitrary delimiter of "XXX" and then a failure message, if any. I used this kind of syntax for both reports, being careful to change the file names and the perl interpreter I was using for the 5.8 report:

for f in `cat 5.5-bare-module-list.txt`; { echo -n "$f XXX "; perl -M$
+f -e 1; echo; } 2>&1 | tee 5.5-use-module-failures.txt
    
[download]

With reports for both versions in hand, I could now produce a report of modules that were broken in 5.8 but not in 5.5. I did that like this:

diff 5.5-use-module-failures.txt 5.8-use-module-failures.txt | grep '>
+.*XXX \w' | tee modules_broken_in_58_and_not_55.txt  | more
    
[download]

Any errors at this stage will again need to be addressed by hand.

9. Some CGI.pm issues

After upgrading, I noticed some issues with the CGI.pm module. You might look into these before your upgrade is final if you are CGI.pm user.

• The first thing I noticed was that this was appearing in the logs when it hadn't before: Ambiguous call resolved as CORE::die(). It didn't appear to cause any user visible problems, but it merited investigation. It turned out there were two versions of CGI/Carp.pm installed for Perl 5.8:

  /usr/local/lib/perl5/5.8.0/CGI/Carp.pm
  /usr/local/lib/perl5/site_perl/5.8.0/CGI/Carp.pm
              
  [download]

  The one in site_perl was older was the one being called and emitting the error. I fixed this by copying the newer one over top of the older one:

  cp /usr/local/lib/perl5/5.8.0/CGI/Carp.pm /usr/local/lib/perl5/site_pe
  +rl/5.8.0/CGI/Carp.pm
                  
  [download]

• The second CGI.pm related issue was stranger it looked like it might actually be related to the Perl upgrade and not to CGI. In some cases there was code that looked like this:

  use CGI qw(param);
  # ....
  print popup_menu(-name=>'foo',values=>[0,1]);
              
  [download]

  It's questionable whether this should have worked in the first place, since popup_menu was explicitly imported. After the upgrade, errors appeared like this:

   Undefined subroutine &Sumsault::CgiDB::popup_menu called at    
                  
  [download]

  This apparently means that popup_menu was no longer appearing in the namespace when it had been before. This was addressed by explicitly importing the missing function calls:

  use CGI qw(param popup_menu);

10. Update system to use perl port

Now it's time to actually switch from having the system use the old perl to having it use the new perl.

• Before you do this, you might double-check the permissions on the critical binaries before you start. In particular, check that your "setuid" perl binaries end up with the same permissions that they had before. Here's s sample ls command to list these binaries:

            ls -l /usr/bin/{perl,suidperl,perl5*,sperl*}
        
  [download]

  You may want to include other perl related binaries in this list if you have changed the default permissions.

• First I ran the official FreeBSD script for switching between the base system perl and the perl port:

  /usr/local/bin/use.perl port
    
  [download]

• Next, I copied all the perl binaries that I had moved out of the way back into place:

  for f in `cat perl_binaries.txt`; { echo "mv /usr/local/bin/$f.copy /u
  +sr/local/bin/$f; "; }
              
  [download]

  Like last time, this will print out the suggested commands to screen and allow me visually verify things before I copy and pasted them to run.

• Reality check things by reviewing the Perl related binaries in /usr/bin and /usr/local/bin to make sure they look reasonable.

11. Reality Check Projects

Now it's time to test some real perl scripts to check for problems that may have been missed elsewhere in the process. Since nearly all of our Perl is used for CGI scripting, we checked the log files of many websites on the machine to check for errors. A bash command like this was used to find all the error logs and grep them for errors that might be perl related. Season to taste.

locate 'httpd-error_log' | grep -v '\.' | xargs tail -50 | egrep -v \ 
'does not exist|script not found|invalidate|authentication failure|una
+ble to include|commit ineffective|uninitialized value|explicit discon
+nect' | more    
[download]

12. Be Prepared to Revert

At this point, it's still good to be prepared for the worst case scenario of reverting to the old version of Perl if something is breaking that can't be fixed in a reasonable amount of time. Again, this should be an easy two step process.

1. Use the official script to reference the base system perl.

           /usr/local/bin/use.perl system
     
   [download]

2. Move all your new perl binaries out of the away, following the steps above.

13. Some Other Perl 5.8 Nodes of Interest

• How ready is 5.8.0 for production stable work?
• POD documentation in PDF format for 5.8.0
• Does 5.8.0 suck?
• Favorite changes in Perl 5.8?
• relative speed of 5.8.0

---

Copyright © 2002 Summersault, LLC. All rights reserved.

update: improved formatting at jens suggestion, and html entities thanks to runrig. Added "Why" section on 8/24/02