Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight

Re: XS from XS?

by creamygoodness (Curate)
on Apr 10, 2010 at 14:40 UTC ( #834004=note: print w/replies, xml ) Need Help??

in reply to XS from XS?

I know of two ways to do it.

First, you can go through the Perl interface using the callback API from perlcall. That would definitely not qualify as "easy", though.

Alternatively, you can get the source library to export its C functions somehow. There are numerous ways to accomplish this, some of which are discussed in Dynaloader/XS: sharing C symbols across shared objects. Cross-platform exporting is difficult. I ultimately adopted the technique from Time::HiRes described by salva.

Here's code from Snowball.xs which makes 4 symbols from the Snowball stemming library available (sb_stemmer_list, etc):

MODULE = Lingua::Stem::Snowball PACKAGE = Lingua::Stem::Snowball PROTOTYPES: disable BOOT: { SV *sb_stemmer_list_sv = newSViv(PTR2IV(sb_stemmer_list)); SV *sb_stemmer_new_sv = newSViv(PTR2IV(sb_stemmer_new)); SV *sb_stemmer_delete_sv = newSViv(PTR2IV(sb_stemmer_delete)); SV *sb_stemmer_stem_sv = newSViv(PTR2IV(sb_stemmer_stem)); SV *sb_stemmer_length_sv = newSViv(PTR2IV(sb_stemmer_length)); hv_store(PL_modglobal, "Lingua::Stem::Snowball::sb_stemmer_list", +39, sb_stemmer_list_sv, 0); hv_store(PL_modglobal, "Lingua::Stem::Snowball::sb_stemmer_new", 3 +8, sb_stemmer_new_sv, 0); hv_store(PL_modglobal, "Lingua::Stem::Snowball::sb_stemmer_delete" +, 41, sb_stemmer_delete_sv, 0); hv_store(PL_modglobal, "Lingua::Stem::Snowball::sb_stemmer_stem", +39, sb_stemmer_stem_sv, 0); hv_store(PL_modglobal, "Lingua::Stem::Snowball::sb_stemmer_length" +, 41, sb_stemmer_length_sv, 0); }

That works for a small list of functions. It would be cumbersome and impractical for a large export list.

Replies are listed 'Best First'.
Re^2: XS from XS?
by BrowserUk (Pope) on Apr 10, 2010 at 15:12 UTC

    If I understand that right--and I'm not certain I am--you are importing the Perl callable functions in one XS module, into the namespace of another XS module?

    If so, that's not quite what I'm after--though I understand that my question was unclear.

    My thought--albeit perhaps not fully thought through--is to have an XS module that is callable/usable from Perl in the normal way. But to also allow other XS modules to make use of the internal C functions (that sit behind the Perl-callable XS interface), directly at runtime. That is, without #includeing the sources, or linking against the objects or a lib. There would be two or three entrypoints to import at most.

    The notion was to remove the compile-time dependancy upon the headers or a lib for potential XS users of the module. They could then decide at runtime whether it is available and use it. Or fallback to other mechanisms. Ie. If the module is installed locally, load it and "obtain the required entrypoints".

    Hm. I know I can do this on Windows. LoadModule()/GetProcAddr() are all I need. But I would need a portable solution, and I don't know enough about other platforms to know if there are equivalents.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      What about creating a nice set of C wrapper functions for the OS dependant LoadLibrary/GetProcAddr/FreeLibrary calls? Makefile.PL / Build.PL could decide which one to use, or panic if the OS is unknown / not implemented.

      BTW: Linux seems to use dladdr, dlclose, dlerror, dlopen, dlsym, dlvsym, the first four are POSIX.1-2001, the other two are GNU extensions, according to the man page. So, if you use the Win32 API for Windows as a special case, and simply try to use dlopen and friends on other platforms, you could get the job done for most Unix platforms. Other platforms should fail and complain loudly during make.


      Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

      I believe that second technique I described (the Time::HiRes approach) does exactly what you want. In the exporting module, you store function pointer address values within a global Perl hash as integer scalar values. In the importing module, you recover those function pointer address values from the integer scalars and cast them to function pointers of the appropriate type.

      If I understand that right--and I'm not certain I am--you are importing the Perl callable functions in one XS module, into the namespace of another XS module?

      He's saving the address of functions callable from C in the current Perl package.

        The PL_modglobal hash isn't quite the same thing as the general symbol table. From perlapi:
        PL_modglobal is a general purpose, interpreter global HV for use by extensions that need to keep information on a per-interpreter basis. In a pinch, it can also be used as a symbol table for extensions to share data among each other. It is a good idea to use keys prefixed by the package name of the extension that owns the data.

        However, storing the addresses in scalar package globals accessed from C via get_sv() would be another approach and might actually be better.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://834004]
holli just tried to open a soda bottle with a bottleopener
[holli]: ok, that doesn't *sound* that bad
[Corion]: holli: Was it a "twist-to-open" soda bottle? Or a German Mehrweg bottle with a twisty cap?

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (9)
As of 2017-09-25 12:32 GMT
Find Nodes?
    Voting Booth?
    During the recent solar eclipse, I:

    Results (280 votes). Check out past polls.