Re: Packaging Perl Programs (is) Painful

[tsee]

So you've had your opportunity to vent your frustration with this post. I had mine with another reply. Let's get back to work and analyze how we can improve things for the next guy. I'll just address some parts of your post and refer to the PAR toolkit only:

• The "use lib" bug": You reported it and it's fixed in the Module::ScanDeps trunk. It'll be gone after the next release.

• The slowness of Module::ScanDeps: The bulk of the time taken to package an application with PAR::Packer (== pp) is taken up by Module::ScanDeps. It scans your code for potential dependencies. Then recurses into them and does the same. This results in a lot of IO *and* in a lot of CPU churn. This is partly alleviated for multiple repeated builds by using the caching feature. But not entirely. It turns out that there's an underlying problem. Just recently, Chia-liang Kao reported this to me. I'm paraphrasing:

  The expanded DateTime::Locale modules all use utf8 and create a n-by-m dependency matrix with add_info [a Module::ScanDeps method for adding a dependency link]. They all get pushed into the return value which gets constantly iterated over. The duplicated calls only get skipped in the next level.

  This shows that there's clearly room for improvement. Neither Chia-Liang nor I have had the time to investigate a fix and it's not likely to happen any time soon. Volunteers and discussion on the PAR mailing list would be most welcome!

• Couldn't package Wx application: Others have pointed out that the external wxWidgets libraries need manual treatment. This isn't really a flaw in PAR (or other packagers). The only way to find out about what shared libraries would be needed by an XS module would be to ask the run-time linker. But that would be so utterly platform dependent and probably prone to breakage that nobody has bothered to attempt it. Instead, there are things like Wx::Perl::Packager that have the special cases for common libraries built-in.

• The --gui option hides errors: Well. It is documented as:

  Build an executable that does not have a console window. This option is ignored on non-MSWin32 platforms or when -p is specified.

  I think that this is quite clear about the effect of the option.

• Executable size: Unfortunately, PAR packaged executables tend to be very large. It's because your Perl installation (which is packaged in the exe, mostly gzip'd) is very large, too. Add some shared libs... and you easily arrive at a couple of MB or even over ten. An alternative is something like Cava, which instead creates a "portable" library tree next to the slim executable loader. Choose your poison. There's one avenue to gain a bit in the case of PAR. The components that are required to bootstrap the application at run-time so far can't be compressed. Nowadays, that makes up for quite some overhead. Roderich Schupp and I have been experimenting with statically linking the executables to libz and decompressing the bootstrap-components on the fly. Roderich's code is available as a branch in the PAR repository. I'm sure he'd be open to contributions.

I hope this constitutes a start in the direction of rational discussion (and action) with the goal of making our tools better.

[Anonymous Monk]

Couldn't package Wx application: Others have pointed out that the external wxWidgets libraries need manual treatment. This isn't really a flaw in PAR (or other packagers). The only way to find out about what shared libraries would be needed by an XS module would be to ask the run-time linker. But that would be so utterly platform dependent and probably prone to breakage that nobody has bothered to attempt it.

It is a pita, but I doubt its too fragile (its how I do it). One way is with ldd, another is with sysinternals listdlls.

Better yet is gcc/mingw objdump , ex

$ objdump -p C:\strawberry\perl\site\lib\auto\Wx\Wx.dll  |grep "DLL Na
+me"
        DLL Name: KERNEL32.dll
        DLL Name: msvcrt.dll
        DLL Name: msvcrt.dll
        DLL Name: perl510.dll
        DLL Name: wxbase28_gcc_custom.dll
        DLL Name: wxmsw28_adv_gcc_custom.dll
        DLL Name: wxmsw28_core_gcc_custom.dll

or microsofts dumpbin /IMPORTS:msvcrt.dll

or depends.exe /c /f:1 /ot:temp.txt msvcrt.dll

Once you get a list , add each dll with pp -l option

[tsee]

Just lost my reply due to "Preview" ne "Submit". Oops. I'll keep it short:

Doing this kind of thing would be fragile if attempted for the "general case". I.e. all elements of the "platform X compiler tool chain" matrix. In particular, we'd need a blacklist of libraries to never package. Maybe an interactive front-end that prompts the user to select which of the detected dependencies he really wants, etc.

I'd love to have this even just for some platforms, but in a way that doesn't end up with users plastering "PAR is broken on platform X!" all over the internet if this particular bit of functionality is not available for their favourite OS.

[Anonymous Monk]

In particular, we'd need a blacklist of libraries to never package.

Um, yeah, some logic needs to be worked out ... PPM/DEB/RPM... packagers all manage to avoid packing c-runtime... should be able to borrow some ideas/lists from them. On win32 you can filter system dlls by %windir%.

I'd love to have this even just for some platforms, but in a way that doesn't end up with users plastering "PAR is broken on platform X!" all over the internet if this particular bit of functionality is not available for their favourite OS.

I don't see how it can get worse than the current situation. PAR needs a one-click-checkout, patents be damned :)

[tsee]

Addendum re Module::ScanDeps performance. I just sped up the run time of the Module::ScanDeps tests by a factor of three. In a test with "pp -e 'use Padre'", this moves Module::ScanDeps down from being the slowest part of a "pp" run to making Archive::Zip-related operations the culprit.

This took half an hour. What I'm trying to say is that there are plenty of low-hanging fruit for everyone.