Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Extending perl with C dynamic library.

by Martin90 (Sexton)
on Aug 16, 2013 at 19:54 UTC ( #1049776=perlquestion: print w/ replies, xml ) Need Help??
Martin90 has asked for the wisdom of the Perl Monks concerning the following question:

Hello monks, I would like to extend my perl module with some functions from C. I started with basic Hello World example and failed ;/

test.c

#include <stdio.h> void hello() { printf("Hello world!\n"); }
I compiled test.c to test.so and add to dir.

test.pm

package Example; use 5.006; use strict; use base qw/Exporter DynaLoader/; our $VERSION = '0.01'; our @EXPORT_OK = qw/ hello /; bootstrap Test $VERSION; 1;

in Test.pl I call C function with:

#!/usr/bin/perl use strict; use Example; print Example::hello;
Unfortunatelly it doesn't work, returning error "Can't find 'boot_test' symbol in ./test.so at test.pl line 4" Probably I use dynaloader in bad way, but have no idea how to make it works ;/ Yes, I know ther is an option with XS language but I would like to avoid it and just writte .c, code then compile it to .so and use from perl. I had heard that it must works. Thanks for reasonable advices ;)

Comment on Extending perl with C dynamic library.
Select or Download Code
Re: Extending perl with C dynamic library.
by kennethk (Monsignor) on Aug 16, 2013 at 20:08 UTC
    You may consider Inline::C rather than DynaLoader for connecting the two. Inline modules compile your external code and then link it automatically via autogenerated XS.

    Barring that, you'll want to review the DynaLoader documentation, specially bootstrap() and @dl_require_symbols, which explain why it's looking for boot_test.


    #11929 First ask yourself `How would I do this without a computer?' Then have the computer do it the same way.

      Yes I know there is something missing but I don'w know what. What is this "symbol" and how to add this to my .c code ? I have to mention that with XS code compiled to c and then to .so all works. But I want to complie pure c instead of work with XS. So my question is how to add this symbol and how to make it works ?
        What is this "symbol" and how to add this to my .c code ?
        A symbol is just a name for something, a function in this case. It you can't use XS to generate a boot_<module> function for you, you'll have to write it yourself. Not an easy task, but perhaps achievable. Try looking at *.c files generated by xsubpp from *.xs files and writing similar code manually.
Re: Extending perl with C dynamic library.
by BrowserUk (Pope) on Aug 16, 2013 at 23:20 UTC

    Here is what happens when you write a small pure C function and build it with Inline::C.

    Start with the C function and the Inline::C support structure:

    #! perl -slw use strict; use Config; use Inline C => Config => BUILD_NOISY => 1, CCFLAGS => $Config{ccflags +}." -DDEBUG=1"; use Inline C => <<'END_C', NAME => 'ICexample', CLEAN_AFTER_BUILD =>0 +; int add( int a, int b ) { int result; result = a + b; return result; } END_C my $a = 12345; my $b = 23456; my $c = add( $a, $b ); print $c;

    When you run that, Inline::C wraps that C function up with an XS function to call it into a .XS file which gets XS-preprocessed to a .c file which is what gets compiled to create the dynamic llibrary (.dll/.so). Here I've thrown away a lot of boilerplate code to make it easier to see what is going on. The resultant .c file contains 3 functions.

    The original C function:

    int add( int a, int b ) { int result; result = a + b; return result; }

    A wrapper function that unpacks the input arguments from their Perl variables; calls the C function with the extracted values; and then wraps up the return value from the C function into a Perl variable for returning to the Perl code:

    XS_EUPXS(XS_main_add) { dVAR; dXSARGS; int a = (int)SvIV(ST(0)) int b = (int)SvIV(ST(1)) int RETVAL; dXSTARG; RETVAL = add(a, b); XSprePUSH; PUSHi((IV)RETVAL); XSRETURN(1); }

    And finally, it adds a bootstrap function who purpose is two-fold:

    1. to provide a known entrypoint that bootstrap can call in the .dll/.so; The name is always 'boot_' + the name of the dll/.so file.
    2. To return the Perlish names of the function the .dll/.so exports -- these are the XS wrappers around the C functions -- and their entrypoint addresses so these can be fixed up in the Perl programs symbol table and thus become callable from Perl.
    XS_EXTERNAL(boot_ICexample) { dVAR; dXSARGS; const char* file = __FILE__; XS_VERSION_BOOTCHECK; newXS("main::add", XS_main_add, file); XSRETURN_YES; }

    So, you should be able to see from this that you cannot avoid the XS wrapper functions because the C code would not know what to do with perl variables. And you cannot do away with the 'boot_module' entrypoint because perl would not know what functions are exported by the .dll/.so.

    I strongly urge you to play with Inline::C and explore the files it creates. You'll learn far more, far more quickly that banging your head on the wall of ignorance trying to go your own way.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      BrowserUk , huge thanks for this !

      I've tested Inline:C version and it works almost the same way as XS. For me big adventage of Inline:C is the fact that code is more visable and well separated.

      But I didn't mentioned why I am looking for another way of produce dynamic libraries (.so) if I already have XS solution.

      The answer is .so file size. With XS and Inline::C I get .so file size aprox. 50 kb (suprisingly big for just one C function !) I have seen other people .so with much more complex code involved several function with just 10-20 Kb size.

      Well, why even simple .so weight so much ? How to slim it down ? ;)

      Thanks.
        How to slim it down ?

        Try stripping the symbols from it:
        strip -s your.so
        Often, but not always, perl is built with a cflags setting that ensures that the shared object is automatically stripped.

        Cheers,
        Rob
        Well, why even simple .so weight so much ? How to slim it down ? ;)

        Quite frankly, I wouldn't bother.

        If you achieved a full 40k saving it only represents 0.001% of the most modestly configured workstation or server memory.

        And that's if the whole of the on-disk representation actually gets loaded into memory, which it probably doesn't.

        And if your thought is smaller file will take less time to load. The time spent finding the directory entry and seeking to the file will completely swamp any difference in loading 10k instead of 50k. It is doubtful if you could ever actually measure any difference it might make.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
        Counter-Question: WHY do you want to use C?

        Or put differently: What is your real goal?

        Do you just want to see it work? Do you want to make something more performant? Is there something that's not implementable in perl?

        I'm asking this because shortly after trying to implement inline C you're worried about size of the resulting library. To me the first priority would be: Is it worth it? And that means some kind of performance/functionality test. Stripping down the C-beast afterwards is only necessary if there is an actual gain - otherwise stay with pure perl for easier portability, maintainability, conciseness...

Re: Extending perl with C dynamic library.
by spx2 (Chaplain) on Aug 19, 2013 at 07:39 UTC

    If you want to learn XS, there's a good book on it, I mean this one : Extending and Embedding Perl

    Also, use proper tooling such as h2xs to generate the skeleton you need, and you can build on that.

      syphilis, thanks that helped and .so are smaller ;)

      spx2 thanks for good resource !

      Unfortunately I encountered another problem, this time when I try to add perl compiler to C.

      #!/usr/bin/perl use strict; use warnings; use Config; use Inline C => Config => BUILD_NOISY => 1, CCFLAGS => $Config{ccflags +}." -DDEBUG=1"; use Inline C => <<'END_C', NAME => 'Example', CLEAN_AFTER_BUILD =>0; static PerlInterpreter *my_perl; int main(int argc, char **argv) { char* command_line[] = {"", "-e", "print \"Hello from C!\\n\";"}; my_perl = perl_alloc(); perl_construct(my_perl); perl_parse(my_perl, NULL, 3, command_line, (char **)NULL); perl_run(my_perl); perl_destruct(my_perl); perl_free(my_perl); return 0; } int add( int a, int b ) { int result; result = a + b; return result; } int hello() { printf("Hello World\n"); } END_C sub message { print "Everything went well" } &message;

      On windows this code fail:

      "Dmake error code 129, while making \/patch_here/Example.dll" "A problem was encountered while attepting to compile and install your inline c code. The command that failed was dmake."

      On Linux (CentOS 6) program compile successfully but problem appears in next step:

      cc -o Example.o -c Example.c `perl -MExtUtils::Embed -e ccopts`

      returns:

      "Example.xs:4:20: error: INLINE.h: No such file or directory Example.c: In function XS_main_main: Example.c:55: warning: initialization makes pointer from integer without a cast"

      Because Inline produce their own .o file I try to compile their .o file, again with no success:

      cc -o Example Example.o `perl -MExtUtils::Embed -e ldopts`

      return:

      "Example.o: In function `XS_main_main': /home/admin/Embed/_Inline/build/Example/Example.c:55: undefined reference to `XS_unpack_charPtrPtr' collect2: ld returned 1 exit status"

      I have no idea what is wrong here, looking for your help again monks ;)

      Also, it is possible to implement full perl subroutine code to C code or just add pointer that refer to perl subroutine somewhere else ?
        On windows this code fail

        On Windows, if (in your code) I replace int main(int argc, char **argv) { with int main(void) { then everything goes fine.
        At least, I get output of Everything went well.

        Does that help ? (I'm a bit lost as regards where this is heading.)

        Cheers,
        Rob

        You are trying to create a dll with a main entrypoint. No wonder it fails.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1049776]
Approved by choroba
Front-paged by toolic
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (5)
As of 2014-09-22 22:22 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (205 votes), past polls