XS malloc and free

[Brutha]

Hello fellow monks

after lots of reading (but less understanding) of perlxstut, perlxs, perlguts and Perl XS: garbage-collecting my malloc'd buffer, but I still cannot figure out when to free external allocated memory.

I have an external library, that allocates memory, which i have to free. It returns the pointer in a char* variable, to which I provide an address. So one try of my code looks like this:

void
PropertyGet(self, propName)
    fwyHANDLEobj self
    char* propName
 PREINIT:
    char* pValue;
    int rc;
 PPCODE:
    rc = fwyPropertyGetChar( self, propName, &pValue );
    if (rc == 0) {
        XPUSHs(sv_2mortal(newSVpv(pValue, 0)));
        # free(pValue);   <=== does not work
    }
    else {
        XSRETURN_UNDEF;
    }

The C function fwyPropertyGetChar allocates memory and return it in pValue. I am supposed to free it. Now tye answered in the mentioned thread:

When you tell XS that you have a "char *" input buffer, it pulls out a pointer to the string value (if any) stored in the scalar you pass in. So that buffer is allocated just like any other scalar string value buffer in Perl and is free()d in the same situations. So you don't need to worry about a memory leak.

Is that true in my case?
Does sv_2mortal(newSVpv(pValue,0) take the pointer and handle it via garbage collection?

If yes, then I do not understand the line in perlguts (different context though)

    sv_setpv(sv, dberror_list[dberror]);

that would try to free a constant constant string. Wouldn't it need an strdup then?

On Windows I get "Free to wrong pool 15d2d20 not 80104 at t/07-sample.t line 80" when I uncomment free, but I get this with every pointer that the API gives me, even when it is not converted to Perl variables, e.g pointers hold in a structure that I free in DESTROY.

Any guidelines for freeing external alocated memory in Perl? Thanks in advance.

[tye]

Googling for "Free to wrong pool" shows that this error is specific to Perl, so your call to free() is calling Perl's own free() but surely your library isn't calling Perl's own malloc(). And my keen insight tells me that this is probably due to a C-preprocessor macro. So I chdir to perl/lib/CORE and "grep free *.h" and get:

XSUB.h:#    define free         PerlMem_free

which is unfortunate. Preventing XS routines from being able to easily choose which malloc system they want to use (Perl's or C's -- which are sometimes the same) is not the best idea.

In the meantime, you can "#undef free" before your call to free() to work around the problem (but don't use "free" any further down in your code, even indirectly via some other obscure C-preprocessor macro).

[BrowserUk]

If I got the function pointer syntax right then something like this would be a workaround maybe?

void (perls_free)( void*p ) = &free;
#undef free

// free() should now refer to the crt free() and use perls_free() for 
+XS

Though the long term solution would be to remove the #define from the header file in favour of Safefree().

That said, the whole area of memory allocation functions and macro's in the Perl codebase is a mess. Sometime last year I did a not totally crude grep and eliminate duplicates process on the sources and discovered that between the 4 basic functions--malloc(), calloc(), realloc() & free()--there were (from memory) 114 different ways of calling them through the multiplicitous variations of functions and macros. It's truely amazing that there are many more memory problems than there are.

[tye]

I think you are missing a "*":

void (*perls_fre­e)( void*p ) = &free;
#undef free

But that still risks the second half of my warning:

(but don't use "free" any further down in your code, even indirectly via some other obscure C-preprocessor macro).

which seems rather difficult to avoid with any certainty given your assessment of the current messiness. :)

Update: BTW, I am aware that the "&" is "optional", but I consider this simply to be a mistake on the part of the ANSI C definers. (:

[ikegami]

Not easy indeed. None of the following work:

#define d a

#include <stdio.h>

void a() { printf("a\n"); }
void b() { printf("b\n"); }

int main() {
   a();
#define a b
   a();

   (*&a)();

   /* \a(); */

#define c a
   c();

   d();

   return 0;
}

[tye]

Of course not. (: The C preprocessor is mostly just text substitution and is defined to not care about the order in which things are defined.

A more interesting case that also doesn't work is:

#include <stdio.h>

void flee(int i)     { printf("flee(%d)\n",i); }
void perlFlee(int i) { printf("perlFlee(%d)\n",i); }

int main()
{
   flee(1);
#define flee perlFlee
   flee(2);

#define _stitch(a,b) a ## b
#define Free(x) _stitch(fl,ee)( x )
   Free(3);

   return 0;
}

Which produces "perlFlee(3)" as its last line of output.

And this tenancity of the C preprocessor is why (at least when the idea of XS came about), there should have been (similar to several other changes that were made for similar reasons) a global s/\bfree\b/Perl_free/, and s/\bmalloc\b/Perl_malloc/g done to the Perl source code so "#define free..." could be eliminated (replaced with "#define Perl_free free" for some systems, of course).

[Courage]

from perldoc perlguts

    To free an SV that you've created, call "SvREFCNT_dec(SV*)". Norma
+lly
    this call is not necessary (see "Reference Counts and Mortality").

You do not free SV, you decrement its usage, and, when refcount goes to zero, it will be destroyed, and may be some destruction-time action will be activated

addition to free memory out from external library, consult that library's logic. Perl do not uses external char*, it will copy its data into newly created SV

[ikegami]

He's not asking about freeing the SV. He asking about freeing the string from which he created an SV.

Furthermore, SvREFCNT_dec should not be used here. Making the SV mortal (using sv_2mortal) will automatically free the SV at the end of the XSUB. Think of sv_2mortal as a delayed SvREFCNT_dec.

[salva]

is not there any function available in the library you are using just to release memory?

I am not a Windows expert, but if I remind correctly, there memory has to be released from the same library where it was allocated.

[Brutha]

The api documentation states clearly:

The user is responsible for calling free() on the value.

and example C-programs call it and it works :-(