Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Solving XS and dyld issues on Perl Mac OS X

by ait (Friar)
on Feb 23, 2012 at 18:31 UTC ( #955807=perlquestion: print w/ replies, xml ) Need Help??
ait has asked for the wisdom of the Perl Monks concerning the following question:

Hello fellow a monk,

I am considering this as an RFC for a tutorial on Perl Mac OS X so if you think it's useful as a tutorial on PM, make your comment and I will expand it and draft into a general tutorial for Perl Mac OS X.

After some research I discovered that a lot of people have been having problems with Mac OS X and Perl modules that use XS bindings to C even here on PM dyld.

CPAN modules will rarely fail but modules from source or trying to cook your own on OS X may not be so easy. I've been banging my head trying to get an XS module to work for the past day and a half so after getting it working I though it was useful to share how...

The main thing to understand is that Mac OS X will ship with a relatively new version of Perl and some thing depend on this so you shouldn't be using this Perl for your development. This is similar to Solaris in that the base system depends on a very specific version of Perl.

You should then have 2 Perls in your Mac. A good option is MacPorts but there are others, including installing from source. I use MacPorts so

which perl

Will result in

/opt/local/bin/perl

Libraries and CPAN modules will install in /opt/local as well keeping it separate from the system Perl.

Thats why it's important to use the newer tendency in shebang lines:

#!/usr/bin/env perl

Instead of the retrogade:

#!/usr/bin/perl

In fact in Mac OS X this is very important because it is expected that system Perl is /usr/bin/perl !

OK, so now that we clearly have 2 Perls and our development Perl is NOT the system Perl we try to build or compile an XS module.

I won't go into the details of compiling a dynamic library for Darwin but let me point out for reference that there are some important differences from other Operating Systems. Most notably is the distinction between shared libraries and dynamically loadable modules ("a.k.a. bundles"). On ELF systems both are the same and en in .so, on OS X however, bundles end in .bundle or more confusingly in .so whilst dynamic libraries (the .so we are used to in ELF-based systems) en in .dylib.

The other interesting feature of Perl in Mac OS X is that it supports both 32 and 64 bit code. The exact mechanisms are described here

https://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man1/perlmacosx5.10.0.1.html

OK, so now you know that your XS code will probably be using a dylib and the lib will probably be in /opt/local/lib. Again just for reference just in case you want to compile your C library on a Mac here is the difference from "soname" on ELF systems and you should build with something like this in your Makefile:

os := $(shell uname) libboxr : boxr.c # MacOSX ifeq ($(os),Darwin) gcc -fno-common -c boxr.c gcc -dynamiclib -install_name /opt/local/lib/libboxr.1.dylib \ -compatibility_version 1.0 -current_version 1.0.0 \ -o libboxr.1.0.0.dylib boxr.o #Linux else gcc -shared -fPIC -g -Wall -Wl,-soname,libboxr.so.1 \ -o libboxr.so.1.0.0 -lc boxr.c endif

Assuming you compiled and installed you .dylib correctly, you should now be able to run hsxs -x foo.h and it should locate foo.h from /opt/local/include

So far so good, but try to make test and you will get an undefined symbol, which is what many people have been getting. This is because the standard Makefile built from Makefile.PL will have PERL_DL_NONLAZY=1 which will force the dynloader to resolve all the DL symbols when Perl fires up, which is of course a good thing for developing and testing your XS code. In fact, the XS distro built by default will already have a couple of tests written for you! Anyway, this undefined symbol can occur for several reasons including:

  • The dylib cannot be found
  • The dylib cannot be loaded (64/32 bit problems)
  • The symbols really don't exist on the dynamic lib

Not surprisingly the solution is actually quite simple. Edit the Makefile.PL and add -lfoo to the LIBS variable before you make the Makefile like so:

WriteMakefile( NAME => 'Boxr', VERSION_FROM => 'lib/Boxr.pm', [snip] LIBS => ['-lfoo -lbar'], [snip]

And that's it folks. Hopefully that should fix most of your XS pain on Mac OS X.

Cheers,
--
Alejandro Imass

Comment on Solving XS and dyld issues on Perl Mac OS X
Select or Download Code
Re: Solving XS and dyld issues on Perl Mac OS X
by Anonymous Monk on Feb 23, 2012 at 20:09 UTC
    I recommend perlbrew over relying on macports' perl. Many other ports rely on it's version of perl, so using a cpan tool with that version can mess things up, just as with the system version. Also, do you have example of which CPAN modules fails on OSX. I've never had any problems with the many XS modules I've installed and developed, except for EV, which was just the author being refusing to workaround a compiler bug.
      Also, do you have example of which CPAN modules fails on OSX. I've never had any problems...

      Only the references to MySQL and Oracle modules and a few other mostly seemed that users were using source instead of the CPAN that came with their distro of Perl. My findings as mentioned in the OP were encountered whilst creating an XS modules on the Mac which for me was a first and I never had these problems in Linux.

      I use the CPAN shell all the time with my MacPorts Perl and have never noticed any conflicts with system Perl. Could you please expand a little on how these problems may happen?

        :) yeah, all those problems are getting the wrong binary packages and trying to mix them with other incompatible binary packages -- nothing perl specific except perl was involved

      Did that ever get solved? Did you find a way to work around it?
Re: Solving XS and dyld issues on Perl Mac OS X
by kcott (Abbot) on Feb 24, 2012 at 07:55 UTC

      Indeed.   The Perl versions that ship with Lion vs. Snow Leopard are entirely different.   You will need to prepare your (earnestly awaited!) tutorial with careful parallel references to both, as both are now in common use and are likely to be for some time to come.

        Off topic, but related: I was trying to figure out what percentages of users were on the different OSX versions but my google fu is weak on this one. Seems like Apple would tell us their platform statistics.

        I was assuming that in practice there is probably very few people still on Snow Leopard... yet I am still on Snow Leopard so maybe it is more common.

Re: Solving XS and dyld issues on Perl Mac OS X
by aufflick (Deacon) on Feb 24, 2012 at 10:59 UTC
    Also useful when developing/testing is the DYLD_LIBRARY_PATH environment variable.
Re: Solving XS and dyld issues on Perl Mac OS X
by trwww (Priest) on Feb 25, 2012 at 07:01 UTC

    My contribution for this is at Re: 2nd Perl installation on Mac OSX. I think my comment and your meditation overlap almost completely, but maybe theres something there that could simplify yours.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://955807]
Approved by Corion
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (2)
As of 2014-10-02 05:42 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    What is your favourite meta-syntactic variable name?














    Results (49 votes), past polls