Solaris problem or compile options

[ckevinj]

I've been trying to compile Crypt::OpenSSL::X509 on my Solaris boxes and I am losing my hair too fast over it. It compiled fine under Mac OSX.

So I'm down to wondering if it something with the way the Perl binaries were compiled.

Here's the rundown, I'm hoping someone has some wisdom for me.

Initial problems right away with the execution of 'perl Makefile.PL', complaining about about an option (-lcrypt). I wonder if this isn't a linux thing so I comment that out of the Makefile.PL and adjust the inc and lib paths in the PL to point to a known, good, 64bit OpenSSL library and the include file for the opencsw location:

use inc::Module::Install;
name('Crypt-OpenSSL-X509');
license('perl');
perl_version('5.005');
all_from('X509.pm');
homepage 'https://github.com/dsully/perl-crypt-openssl-x509';
bugtracker 'https://github.com/dsully/perl-crypt-openssl-x509/issues';
requires_external_cc();
cc_inc_paths('/opt/csw/include');
cc_lib_paths('/opt/csw/lib/sparcv9');
#cc_lib_links('crypto');
cc_optimize_flags('-O2 -g -Wall -Werror -mcpu=v9 -m64 ');
auto_install();
WriteAll();

I have a 64 bit perl compiled.

ckevinj@wbz-r01:Crypt-OpenSSL-X509-1.800.2 $file `which perl`
/tools/perl/5_14_2/bin/perl:    ELF 64-bit MSB executable SPARCV9 Vers
+ion 1, dynamically linked, not stripped

The Makefile is made cleanly.

ckevinj@wbz-r01:Crypt-OpenSSL-X509-1.800.2 $perl Makefile.PL 
Checking if your kit is complete...
Looks good
Writing Makefile for Crypt::OpenSSL::X509
Writing MYMETA.yml and MYMETA.json

The gmake is clean.

ckevinj@wbz-r01:Crypt-OpenSSL-X509-1.800.2 $gmake -i
cp X509.pm blib/lib/Crypt/OpenSSL/X509.pm
/tools/perl/5_14_2/bin/perl "-Iinc" /tools/perl/5_14_2/lib/5.14.2/ExtU
+tils/xsubpp  -typemap /tools/perl/5_14_2/lib/5.14.2/ExtUtils/typemap 
+-typemap typemap  X509.xs > X509.xsc && mv X509.xsc X509.c
gcc -c  -I/opt/csw/include -mcpu=v9 -m64 -fno-strict-aliasing -pipe -f
+stack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DPERL_USE
+_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -O2 -g -Wall -Werror -mcpu=v9 -m6
+4    -DVERSION=\"1.800.2\" -DXS_VERSION=\"1.800.2\" -fPIC "-I/tools/p
+erl/5_14_2/lib/5.14.2/sun4-solaris-64/CORE"   X509.c
Running Mkbootstrap for Crypt::OpenSSL::X509 ()
chmod 644 X509.bs
rm -f blib/arch/auto/Crypt/OpenSSL/X509/X509.so
gcc  -mcpu=v9 -m64 -G -L/usr/lib/sparcv9 -fstack-protector -G X509.o  
+-o blib/arch/auto/Crypt/OpenSSL/X509/X509.so      \
        \
  
chmod 755 blib/arch/auto/Crypt/OpenSSL/X509/X509.so
cp X509.bs blib/arch/auto/Crypt/OpenSSL/X509/X509.bs
chmod 644 blib/arch/auto/Crypt/OpenSSL/X509/X509.bs
Manifying blib/man3/Crypt::OpenSSL::X509.3

But the object is not useable. It has so many missing symbols that seem to be mostly Perl related.

ckevinj@wbz-r01:Crypt-OpenSSL-X509-1.800.2 $ldd -d blib/arch/auto/Cryp
+t/OpenSSL/X509/X509.so
        libssp.so.0 =>   /opt/csw/lib/sparcv9/libssp.so.0
        libc.so.1 =>     /lib/64/libc.so.1
        libgcc_s.so.1 =>         /opt/csw/lib/sparcv9/libgcc_s.so.1
        symbol not found: main          (blib/arch/auto/Crypt/OpenSSL/
+X509/X509.so)
        symbol not found: PL_sv_undef           (blib/arch/auto/Crypt/
+OpenSSL/X509/X509.so)
        symbol not found: PL_markstack_ptr              (blib/arch/aut
+o/Crypt/OpenSSL/X509/X509.so)
        symbol not found: PL_stack_sp           (blib/arch/auto/Crypt/
+OpenSSL/X509/X509.so)
        symbol not found: PL_stack_base         (blib/arch/auto/Crypt/
+OpenSSL/X509/X509.so)
        symbol not found: PL_op         (blib/arch/auto/Crypt/OpenSSL/
+X509/X509.so)
        symbol not found: PL_curpad             (blib/arch/auto/Crypt/
+OpenSSL/X509/X509.so)
        symbol not found: PL_sv_no              (blib/arch/auto/Crypt/
+OpenSSL/X509/X509.so)
        symbol not found: PL_sv_yes             (blib/arch/auto/Crypt/
+OpenSSL/X509/X509.so)
        symbol not found: i2d_ASN1_HEADER               (blib/arch/aut
+o/Crypt/OpenSSL/X509/X509.so)
        symbol not found: PL_unitcheckav                (blib/arch/aut
+o/Crypt/OpenSSL/X509/X509.so)
        symbol not found: PL_scopestack_ix              (blib/arch/aut
+o/Crypt/OpenSSL/X509/X509.so)
        libm.so.2 =>     /lib/64/libm.so.2
        /platform/SUNW,Sun-Fire-T1000/lib/sparcv9/libc_psr.so.1

My Perl binary is not multi-threaded. I looked for libperl.so in the /tools/perl/5_14_2 path to put in my LD_LIBRARY_PATH_64, but there was only a libperl.a.

I did find some issues with a tool where some one was getting similar symbol issues when they were trying to run a module that expected a multi-threaded perl but it wasn't. So I'm attempting to recompile Perl again as multi-threaded to test that possibility.

[Eliya]

It has so many missing symbols that seem to be mostly Perl related.

The Perl related symbols aren't really a problem. They're just an artifact of ldd trying symbol resolution in isolated context.  In real life circumstances, the shared object will be loaded by the Perl binary, which should provide all the symbols (if it's the right version).

What looks suspicious, though, is the i2d_ASN1_HEADER symbol — it should probably come from the openssl library.  And I think this is the very problem, i.e. that your X509.so hasn't been linked against the openssl library.  The reason could be that you don't have the openssl development libs installed (as opposed to the runtime libs).

[ckevinj]

Thanks, that helps narrow it down.

But I I believe I have the openssl dev files. It shows the openssldev package installed and the files included in the package are what I would typically expect, and are in a location that I was referencing....

...
Pathname: /opt/csw/include/openssl/x509.h
Pathname: /opt/csw/include/openssl/x509_vfy.h
Pathname: /opt/csw/include/openssl/x509v3.h
Pathname: /opt/csw/lib/libcrypto.a
Pathname: /opt/csw/lib/libcrypto.so
Pathname: /opt/csw/lib/libssl.a
Pathname: /opt/csw/lib/libssl.so
...
Pathname: /opt/csw/lib/sparcv9/libcrypto.so
Pathname: /opt/csw/lib/sparcv9/libssl.a
Pathname: /opt/csw/lib/sparcv9/libssl.so
...

that is I'm assuming dev files would be include dir and the libraries. Or is there something else?

Are the symbols found from includes or shared objects?

[Eliya]

... Or is there something else?

No, that looks ok.  But a fact is that X509.so wasn't linked with libssl/libcrypto.  This command should have mentioned -lssl and -lcrypto, together with the correct search path, such as -L/opt/csw/lib.  But it didn't:

gcc  -mcpu=v9 -m64 -G -L/usr/lib/sparcv9 -fstack-protector -G X509.o  
+-o blib/arch/auto/Crypt/OpenSSL/X509/X509.so      \
        \

The most typical reason for this is missing dev libs (which we've excluded by now), but occasionally more subtle problems lead to libs not being detected by MakeMaker, and if MakeMaker thinks a lib is not there, it doesn't link against it...

Anyhow, the most practical solution would probably be to just re-run the above link command manually (from the build directory), appending the link instructions for the libs in question (you can get rid of those line-continuation backslashes).  If that goes well, just continue with make test && make install.

gcc  -mcpu=v9 -m64 -G -L/usr/lib/sparcv9 -fstack-protector -G X509.o  
+-o blib/arch/auto/Crypt/OpenSSL/X509/X509.so -L/opt/csw/lib -lssl -lc
+rypto

[rurban]

Looks like Module::Install did not find your static libperl.a
Just add it manually to your linker line and you'll be fine. Something like:



gcc  -mcpu=v9 -m64 -G -L/usr/lib/sparcv9 -fstack-protector -G X509.o 
-o blib/arch/auto/Crypt/OpenSSL/X509/X509.so
-L/tools/perl/5_14_2/lib/5.14.2/sun4-solaris-64/CORE -lperl

[Eliya]

XS shared object files are never explicitly linked against libperl, simply because they are designed to be dynamically loaded by perl itself (via dlopen). And perl being perl it does have all the Perl symbols already (linking in libperl.a would effectively duplicate the code — so don't do that).

[Khen1950fx]

I had one hell of a time trying to get it to install. I finally modified the X509.pm, so download and manually install the module using this as your X509.pm. I'm on a hybrid Linux system.

package Crypt::OpenSSL::X509;

use strict;
use vars qw($VERSION @EXPORT_OK);
use Exporter;
use base qw(Exporter);

$VERSION = '1.800.2';

@EXPORT_OK = qw(
  FORMAT_UNDEF FORMAT_ASN1 FORMAT_TEXT FORMAT_PEM FORMAT_NETSCAPE
  FORMAT_PKCS12 FORMAT_SMIME FORMAT_ENGINE FORMAT_IISSGC OPENSSL_VERSI
+ON_NUMBER
);

require DynaLoader;
our @ISA = qw(DynaLoader);
'bootstrap Crypt::OpenSSL::X509';
sub dl_load_flags { 0x01 }

sub Crypt::OpenSSL::X509::bit_length {
    my $x509 = shift @_;

    return
      length( $x509->modulus ) * 4;   # each character is one hex digi
+t = 4 bits
}

sub Crypt::OpenSSL::X509::has_extension_oid {
    my $x509 = shift @_;
    my $oid  = shift @_;

    if ( not $Crypt::OpenSSL::X509::exts_by_oid ) {
        $Crypt::OpenSSL::X509::exts_by_oid = $x509->extensions_by_oid;
    }

    return $$Crypt::OpenSSL::X509::exts_by_oid{$oid} ? 1 : 0;
}

sub Crypt::OpenSSL::X509::Extension::is_critical {
    my $ext  = shift @_;
    my $crit = $ext->critical();

    return $crit ? 1 : 0;
}

# return a hash for the values of keyUsage or nsCertType
sub Crypt::OpenSSL::X509::Extension::hash_bit_string {
    my $ext = shift @_;

    my @bits = split( //, $ext->bit_string );
    my $len = @bits;

    my %bit_str_hash = ();

    if ( $len == 9 ) {    # bits for keyUsage

        %bit_str_hash = (
            'Digital Signature' => $bits[0],
            'Non Repudiation'   => $bits[1],
            'Key Encipherment'  => $bits[2],
            'Data Encipherment' => $bits[3],
            'Key Agreement'     => $bits[4],
            'Certificate Sign'  => $bits[5],
            'CRL Sign'          => $bits[6],
            'Encipher Only'     => $bits[7],
            'Decipher Only'     => $bits[8],
        );

    }
    elsif ( $len == 8 ) {    #bits for nsCertType

        %bit_str_hash = (
            'SSL Client'        => $bits[0],
            'SSL Server'        => $bits[1],
            'S/MIME'            => $bits[2],
            'Object Signing'    => $bits[3],
            'Unused'            => $bits[4],
            'SSL CA'            => $bits[5],
            'S/MIME CA'         => $bits[6],
            'Object Signing CA' => $bits[7],
        );
    }

    return %bit_str_hash;
}

sub Crypt::OpenSSL::X509::Extension::extKeyUsage {
    my $ext = shift @_;

    my @vals = split( / /, $ext->extendedKeyUsage );

    return @vals;
}

sub Crypt::OpenSSL::X509::is_selfsigned {
    my $x509 = shift @_;

    return $x509->subject eq $x509->issuer;
}

__END__

=head1 NAME

Crypt::OpenSSL::X509 - Perl extension to OpenSSL's X509 API.

=head1 SYNOPSIS

  use Crypt::OpenSSL::X509;

  my $x509 = Crypt::OpenSSL::X509->new_from_file('cert.pem');

  print $x509->pubkey() . "\n";
  print $x509->subject() . "\n";
  print $x509->issuer() . "\n";
  print $x509->email() . "\n";
  print $x509->hash() . "\n";
  print $x509->notBefore() . "\n";
  print $x509->notAfter() . "\n";
  print $x509->modulus() . "\n";
  print $x509->exponent() . "\n";
  print $x509->fingerprint_sha1() . "\n";
  print $x509->fingerprint_md5() . "\n";
  print $x509->as_string(Crypt::OpenSSL::X509::FORMAT_TEXT) . "\n";

  my $x509 = Crypt::OpenSSL::X509->new_from_string(
    $der_encoded_data, Crypt::OpenSSL::X509::FORMAT_ASN1
  );

  # given a time offset of $seconds, will the certificate be valid?
  if ($x509->checkend($seconds)) {
    # cert is ok at $seconds offset
  } else {
    # cert is expired at $seconds offset
  }

  my $exts = $x509->extensions_by_oid();

  foreach my $oid (keys %$exts) {
    my $ext = $$exts{$oid};
    print $oid, " ", $ext->object()->name(), ": ", $ext->value(), "\n"
+;
  }


=head1 ABSTRACT

  Crypt::OpenSSL::X509 - Perl extension to OpenSSL's X509 API.

=head1 DESCRIPTION

  This implement a large majority of OpenSSL's useful X509 API.

  The email() method supports both certificates where the
  subject is of the form:
  "... CN=Firstname lastname/emailAddress=user@domain", and also
  certificates where there is a X509v3 Extension of the form
  "X509v3 Subject Alternative Name: email=user@domain".

=head2 EXPORT

None by default.

On request:

    FORMAT_UNDEF FORMAT_ASN1 FORMAT_TEXT FORMAT_PEM FORMAT_NETSCAPE
    FORMAT_PKCS12 FORMAT_SMIME FORMAT_ENGINE FORMAT_IISSGC


=head1 FUNCTIONS

=head2 X509 CONSTRUCTORS

=over 4

=item new ( )

Create a new X509 object.

=item new_from_string ( STRING [ FORMAT ] )

=item new_from_file ( FILENAME [ FORMAT ] )

Create a new X509 object from a string or file. C<FORMAT> should be C<
+FORMAT_ASN1> or C<FORMAT_PEM>.

=back

=head2 X509 ACCESSORS

=over 4

=item subject

Subject name as a string.

=item issuer

Issuer name as a string.

=item serial

Serial number as a string.

=item hash

Subject name hash as a string.

=item notBefore

C<notBefore> time as a string.

=item notAfter

C<notAfter> time as a string.

=item email

Email address as a string.

=item version

Certificate version as a string.

=item sig_alg_name

Signature algorithm name as a string.

=back

=head2 X509 METHODS

=over 4

=item subject_name ( )

=item issuer_name ( )

Return a Name object for the subject or issuer name. Methods for handl
+ing Name objects are given below.

=item is_selfsigned ( )

Return Boolean value if subject and issuer name are the same.

=item as_string ( [ FORMAT ] )

Return the certificate as a string in the specified format. C<FORMAT> 
+can be one of C<FORMAT_PEM> (the default), C<FORMAT_ASN1>, or C<FORMA
+T_NETSCAPE>.

=item modulus ( )

Return the modulus for an RSA public key as a string of hex digits. Fo
+r DSA, return the public key. Other algorithms are not supported.

=item bit_length ( )

Return the length of the modulus as a number of bits.

=item fingerprint_md5 ( )

=item fingerprint_sha1 ( )

Return the specified message digest for the certificate.

=item checkend( OFFSET )

Given an offset in seconds, will the certificate be expired?

=item pubkey ( )

Return the RSA or DSA public key.

=item num_extensions ( )

Return the number of extensions in the certificate.

=item extension ( INDEX )

Return the Extension specified by the integer C<INDEX>.
Methods for handling Extension objects are given below.

=item extensions_by_oid ( )
=item extensions_by_name ( )
=item extensions_by_long_name ( )

Return a hash of Extensions indexed by OID or name.

=item has_extension_oid ( OID )

Return true if the certificate has the extension specified by C<OID>.

=back

=head2 X509::Extension METHODS

=over 4

=item critical ( )

Return a value indicating if the extension is critical or not.
FIXME: the value is an ASN.1 BOOLEAN value.

=item object ( )

Return the ObjectID of the extension.
Methods for handling ObjectID objects are given below.

=item value ( )

Return the value or data of the extension.
FIXME: the value is returned as a string but may represent
a complex object.

=back

=head2 X509::ObjectID METHODS

=over 4

=item name ( )

Return the long name of the object as a string.

=item oid ( )

Return the numeric dot-seperated form of the object identifier as a st
+ring.

=back

=head2 X509::Name METHODS

=over 4

=item as_string ( )

Return a string representation of the Name

=item entries ( )

Return an array of Name_Entry objects. Methods for handling Name_Entry
+ objects are given below.

=item has_entry ( TYPE [ LASTPOS ] )

=item has_long_entry ( TYPE [ LASTPOS ] )

=item has_oid_entry ( TYPE [ LASTPOS ] )

Return true if a name has an entry of the specified C<TYPE>. Depending
+ on the function the C<TYPE> may be in the short form (e.g. C<CN>), l
+ong form (C<commonName>) or OID (C<2.5.4.3>). If C<LASTPOS> is specif
+ied then the search is made from that index rather than from the star
+t.

=item get_index_by_type ( TYPE [ LASTPOS ] )

=item get_index_by_long_type ( TYPE [ LASTPOS ] )

=item get_index_by_oid_type ( TYPE [ LASTPOS ] )

Return the index of an entry of the specified C<TYPE> in a name. Depen
+ding on the function the C<TYPE> may be in the short form (e.g. C<CN>
+), long form (C<commonName>) or OID (C<2.5.4.3>). If C<LASTPOS> is sp
+ecified then the search is made from that index rather than from the 
+start.

=item get_entry_by_type ( TYPE [ LASTPOS ] )

=item get_entry_by_long_type ( TYPE [ LASTPOS ] )

These methods work similarly to get_index_by_* but return the Name_Ent
+ry rather than the index.

=back

=head2 X509::Name_Entry METHODS

=over 4

=item as_string ( [ LONG ] )

Return a string representation of the Name_Entry of the form C<typeNam
+e=Value>. If C<LONG> is 1, the long form of the type is used.

=item type ( [ LONG ] )

Return a string representation of the type of the Name_Entry. If C<LON
+G> is 1, the long form of the type is used.

=item value ( )

Return a string representation of the value of the Name_Entry.

=item is_printableString ( )

=item is_ia5string ( )

=item is_utf8string ( )

=item is_asn1_type ( [ASN1_TYPE] )

Return true if the Name_Entry value is of the specified type. The valu
+e of C<ASN1_TYPE> should be as listed in OpenSSL's C<asn1.h>.

=back

=head1 SEE ALSO

OpenSSL(1), Crypt::OpenSSL::RSA, Crypt::OpenSSL::Bignum

=head1 AUTHOR

Dan Sully, E<lt>[e-mail]<gt>

=head1 CONTRIBUTORS

David O'Callaghan, E<lt>[e-mail]<gt>
Daniel Kahn Gillmor E<lt>[e-mail]<gt>

=head1 COPYRIGHT AND LICENSE

Copyright 2004-2011 by Dan Sully

This library is free software; you can redistribute it and/or modify
it under the same terms as Perl itself.

=cut

[Anonymous Monk]

I finally modified the X509.pm,

Would you care to explain your modification?

They don't look particularly useful

[Khen1950fx]

I moved DynaLoader more to the top of the script and basically modified it a bit. Specifically, this is what I did.

require DynaLoader;
our @ISA = qw(DynaLoader);
'bootstrap Crypt::OpenSSL::X509';
sub dl_load_flags { 0x01 };

[zengargoyle]

Add /opt/csw/lib/sparcv9 to your LD_LIBRARY_PATH. Whatever method Makefile.PL is using to probe libraries isn't finding the openssl libraries and therefore not linking them in.

You may also have to set RPATH or LD_RUN_PATH so that the libraries can be found later when the module is loaded and LD_LIBRARY_PATH is unset (or you'll have to make sure to keep it set).

You may also want to build your own openssl anyway. Depending on whether or not CSW stuff is build with GCC or Sun's compiler. Weird things can happen if you mix and match. (net-snmp's SNMP module complains loudly if you try to mix compilers).

Short versioon: you'll need to get X509.so built with a rpath pointing to /opt/csw libs. The other CSW software that uses CSW libs does so by rpath. (unless your systems administrators have already added /opt/csw/lib/... to the default library search path).

Solaris is such a PITA.