Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

What difference between malloc and Newx, how attach a C string to SV directly?

by xiaoyafeng (Deacon)
on Sep 29, 2019 at 06:23 UTC ( #11106827=perlquestion: print w/replies, xml ) Need Help??

xiaoyafeng has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

When Perl communicate with XS, marshalling C to Perl SV is annoying and slow. So I try to just attach C data to a existing SV?:

use Devel::Peek; my $x = ""; Dump $x; ddd($x); Dump $x; use Inline 'C' => <<'CODE'; int ddd(SV* x){ char* dx = "dddddd\n"; char* bbb=NULL; Newx(bbb, 8, char); Copy(dx, bbb, 8, char); printf("string dx is %p \n", dx); printf("string bbb is %p \n", bbb); // sv_setpvn(x, dx, 8); #copy string to a SV // sv_usepvn(x, dx, 8); #segment fault! sv_usepvn(x, bbb, 8); #"attach" to a SV return 0; } CODE ########## output ########## SV = PV(0x559f4143cfd0) at 0x559f41461018 REFCNT = 1 FLAGS = (POK,IsCOW,pPOK) PV = 0x559f414d58a0 ""\0 CUR = 0 LEN = 10 COW_REFCNT = 1 string dx is 0x7f12ed54f8d8 string bbb is 0x559f41462ad0 SV = PV(0x559f4143cfd0) at 0x559f41461018 REFCNT = 1 FLAGS = (POK,pPOK) PV = 0x559f41462ad0 "dddddd\n\0"\0 CUR = 8 LEN = 16
I found you can't attach a C point to a SV directly. You have to either create a mem chunk by Newx, then use sv_usepvn to attach, or use sv_setpvn to extend mem of SV by perl itself, otherwise you would got a segment fault(like sv_usepvn(x, dx, 8) do, dx is a char pointer to a C string as above). So, as the title of question, what difference between newX and C malloc? Is there a way to attach C memory to a existing SV( for performance)?

Thanks in advance!




I am trying to improve my English skills, if you see a mistake please feel free to reply or /msg me a correction

  • Comment on What difference between malloc and Newx, how attach a C string to SV directly?
  • Download Code

Replies are listed 'Best First'.
Re: What difference between malloc and Newx, how attach a C string to SV directly?
by Corion (Pope) on Sep 29, 2019 at 07:41 UTC

    Newx wraps malloc.

    perlguts shows the following snippet to attach a buffer to an SV:

    Newx(buf, somesize+1, char); /* ... fill in buf ... */ buf[somesize] = '\0'; sv_usepvn_flags(sv, buf, somesize, SV_SMAGIC | SV_HAS_TRAILING_ +NUL); /* buf now belongs to perl, don't release it */

    Note how Perl considers the buffer to belong to Perl now.

    If you want to completely manage the lifetime of the buffer yourself (static data, irrespective of SVs that point to it), I think there is a way to have a PV be a pointer to memory that is not free'd by Perl, but none of my reading of the documentation turns up how that works, or an example of it.

      Note how Perl considers the buffer to belong to Perl now.

      yeah, if buf produced from malloc which means it's not belong to perl, program will throw a segment fault error. I guess Newx maybe not just a simple wrap of malloc, it does a register pointer address to perl when which assure perl could free it safely.

      I think there is a way to have a PV be a pointer to memory that is not free'd by Perl, but...

      yes, I think it too, that's why I post question on perlmonks. ;)




      I am trying to improve my English skills, if you see a mistake please feel free to reply or /msg me a correction

Re: What difference between malloc and Newx, how attach a C string to SV directly?
by ikegami (Pope) on Sep 29, 2019 at 09:17 UTC

    Perl supports the memory allocations systems other than malloc+free.

    As such, memory that will be freed by Perl (such as a scalar's string buffer) will need to be allocated using the allocator Perl is configured to use. This is that to which Newx provides access. If you use an incompatible allocator, Perl won't be able to free the memory, and it might very well crash when it tries.

    You can signal Perl not to free a scalar's string buffer by setting its SvLEN to zero. That way, you could assign malloc'ed data or even static-storage data (e.g. a constant) to a scalar. It becomes your responsibility to free it (e.g. in a the destructor of the object the contains the scalar). Note that setting SvLEN to zero won't stop Perl from replacing the buffer with a new one if do something like $s .= "!";.

Re: What difference between malloc and Newx, how attach a C string to SV directly?
by Anonymous Monk on Sep 29, 2019 at 08:47 UTC

    Hi,

    The reason you get a segmentation fault is because when ddd() ends char* dx gets freed

    I think the idea is to use SvPV to get a char* to give to your C function, something like this but by a real c-programmer :)

    I couldn't get it to compile using SvPV or SvPVX calls, dont understand why , error is

    error: lvalue required as left operand of assignment error: lvalue required as unary '&' operand

    This compiles but I couldn't make this assign ddddd string , needs c-programmer

    #!/usr/bin/perl -- use strict; use warnings; use Inline C => Config => BUILD_NOISY => 1, CLEAN_AFTER_BUILD => 0; use Inline 'C' => <<'CODE', NAME => 'xiaoyafeng'; int ddd(SV* x){ SvUTF8_off(x); SvGROW(x,10); char*buff = SvPV_nolen(x); *buff = "dddddd\n"; printf("SV* x is %p \n", x); printf("SvPV(x) is %p \n", SvPV_nolen(x)); printf("string buff is %p \n", buff); SvCUR_set(x, 10); SvSETMAGIC(x); return 0; } CODE use Devel::Peek qw/ Dump /; my $x = ""; Dump $x; ddd($x); Dump $x; exit( 0 ); __END__ SV = PV(0x3f7a8c) at 0x99bca4 REFCNT = 1 FLAGS = (PADMY,POK,pPOK) PV = 0xf46a04 ""\0 CUR = 0 LEN = 12 SV* x is 0099BCA4 SvPV(x) is 00F46A04 string buff is 00F46A04 SV = PV(0x3f7a8c) at 0x99bca4 REFCNT = 1 FLAGS = (PADMY,POK,pPOK) PV = 0xf46a04 "$\0matched\0" CUR = 10 LEN = 12
      #!/usr/bin/perl -- use strict; use warnings; use Inline C => Config => BUILD_NOISY => 1, CLEAN_AFTER_BUILD => 0; use Inline 'C' => <<'CODE', NAME => 'xiaoyafeng'; int ddd(SV* x){ printf("SV* x is %p \n", x); SvUTF8_off(x); SvGROW(x,8); SvCUR_set(x, 8); printf("SvPV(x) is p(%p) \n", SvPV_nolen(x)); printf("SvPV(x) is s(%s) \n", SvPV_nolen(x)); char*buff = SvPV_nolen(x); Copy( "dddddddd", buff , 8, char); printf("SvPV(x) is p(%p) \n", SvPV_nolen(x)); printf("SvPV(x) is s(%s) \n", SvPV_nolen(x)); printf("string buff is p(%p) \n", buff); printf("string buff is s(%s) \n", buff); SvSETMAGIC(x); return 0; } CODE use Devel::Peek qw/ Dump /; my $x = ""; Dump $x; ddd($x); Dump $x; exit( 0 ); __END__ SV = PV(0x3f7a8c) at 0x99bca4 REFCNT = 1 FLAGS = (PADMY,POK,pPOK) PV = 0xb2ac54 ""\0 CUR = 0 LEN = 12 SV* x is 0099BCA4 SvPV(x) is p(00B2AC54) SvPV(x) is s() SvPV(x) is p(00B2AC54) SvPV(x) is s(ddddddddng) string buff is p(00B2AC54) string buff is s(ddddddddng) SV = PV(0x3f7a8c) at 0x99bca4 REFCNT = 1 FLAGS = (PADMY,POK,pPOK) PV = 0xb2ac54 "dddddddd" CUR = 8 LEN = 12
Re: What difference between malloc and Newx, how attach a C string to SV directly?
by xiaoyafeng (Deacon) on Sep 30, 2019 at 02:46 UTC

    Thanks Everyone for your help! After doing some dig with your valuable suggestion, I attach a C string (string constant) to a SV successfully. Here is the UPDATE

    When attach an mem-chunk to an SV, perl does much work for you background, not just save a pointer address. First, perl free the previous PV in SV, then perl will realloc this mem size to mem size +2bytes( I don't understand why though), last, perl set cur and len slot and return. So if you want to attach a C string, you must tell perl not doing so many smarts:

    use strict; use warnings; use Devel::Peek; my $x = ""; Dump $x; ddd($x); print $x; Dump $x; use Inline Config => BUILD_NOISY => 1, ClEAN_AFTER_BUILD =>0; use Inline 'C' => 'Config' => ccflagsex => "-g"; use Inline 'C' => <<'CODE'; int ddd(SV* x){ char* dx = "dddddd\n"; printf("string dx is %p \n", dx); // You have to use sv_use* family, since these functions free the prev +ious PV in SV first for you, but don't use sv_usepvn otherwise perl w +ill realloc pv. sv_usepvn_flags(x, dx, 8, SV_HAS_TRAILING_NUL); // Thanks ikegami, must set LEN to 0, otherwise, perl will try to free + SV when leave scope. SvLEN_set(x, 0); return 0; } CODE
    As a interesting side-effect, I found you can't enlarge the SV containing C string by using .=, no any error throw though.

    __END__ SV = PV(0x55ac3017e050) at 0x55ac301a20b8 REFCNT = 1 FLAGS = (POK,IsCOW,pPOK) PV = 0x55ac30220b00 ""\0 CUR = 0 LEN = 10 COW_REFCNT = 1 string dx is 0x7f0adfd33bfd dddddd SV = PV(0x55ac3017e050) at 0x55ac301a20b8 REFCNT = 1 FLAGS = (POK,pPOK) PV = 0x7f0adfd33bfd "dddddd\n\0" CUR = 8 LEN = 0




    I am trying to improve my English skills, if you see a mistake please feel free to reply or /msg me a correction

      Bug 1:

      You are including the NUL in the buffer, and you are falsely claiming there is a trailing NUL after the buffer.

      sv_usepvn_flags(x, dx, 8, SV_HAS_TRAILING_NUL);
      should be
      sv_usepvn_flags(x, dx, 7, SV_HAS_TRAILING_NUL);
      and it should be written as
      sv_usepvn_flags(x, dx, strlen(dx), SV_HAS_TRAILING_NUL);

      Bug 2:

      The buffer can't be modified, but you offer no protection. For example, $x =~ s/./a/s; will result in undefined behaviour (which fortunately segfaults on my machine).

      Add the following:

      SvREADONLY_on(x);

      I found you can't enlarge the SV containing C string by using .=, no any error throw though.

      I already mentioned that. Or rather, I mentioned that it causes the buffer to get replaced with a new one that contains a copy.

      Before $x .= "a";:

      SV = PV(0x3e1a1785e0) at 0x3e1a197528 REFCNT = 1 FLAGS = (POK,pPOK) PV = 0x35260c8bbd5 "dddddd" <-- Pointer into C static storage CUR = 6 LEN = 0 <-- Doesn't belong to Perl.

      After $x .= "a";:

      SV = PV(0x3e1a1785e0) at 0x3e1a197528 REFCNT = 1 FLAGS = (POK,pPOK) PV = 0x3e1a19fe50 "dddddda"\0 <-- New buffer (copy + "a") CUR = 7 LEN = 17 <-- Belongs to Perl.

      However, SvREADONLY_on(x) will prevent this. So if you want to allow this, you'll need to remove the READONLY flag, which means you'll need to make a copy of the constant string.

        Many Thanks, ikegami, just one question, how perl decide to realloc pv or just make a copy?

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11106827]
Approved by holli
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (2)
As of 2021-09-28 14:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?