converting a C pointer to a perl reference

[jithoosin]

Hi monks,
I am using perlembed for calling perl subroutines from C . I need to pass huge files (around 1Mb) from C to perl. The files are stored in character buffers .char *htmFile .The "htmFile" variable holds the entire file content.

I use "call_pv" for calling perl subroutines from C after pushing the arguments onto perl stack using "xPUSHs". I want to know is there any way by which i could pass the char pointer to perl and use it as a reference in perl. I cannot pass the entire buffer using XPUSHs(sv_2mortal(newSVpvn(htmFile)))
as the perl stack is creating problems(SEG fault) when the file size is around 64kb.What could be the possible reason for this?. Could any of the esteemed monks tell me an efficient manner to pass the huge file to perl? (ofcourse without storing it in harddisk in C and accesing from perl)

Thanks,
Kiran.

[bart]

Just how are you creating those buffers? If you are allocating them in pure C, then you should memory manage them in C.

A typical way of dealing with such things, is to create the buffer in the high level language, in this case Perl (but it's been done in like this in VB, too), and pass the pointer to it to C, which can fill it up. That way, there's no alloc() related madness waiting on your C code. All you have to do is prefill a scalar with a large enough string.

At the worst, you can still copy the contents of the buffer over to the Perl scalar just before you finish. In VB it was customary to use an API call for that: hMemCpy() in Windows 3.1, and RtlMoveMemory for Win32 — the only kind of Windows that's still alive. But of course, the strings were never that long, so maybe that approach isn't so practical.

As an aside, there are several pointer related templates available for pack/unpack, such as "P"/"p".

[creamygoodness]

Hmm. I'm not sure why that might be a problem. There's a HAS_64K_LIMIT defined in dosish.h, but it seems unlikely to be related. Try this as an alternative and see what happens.

htm_sv = newSV(filesize + 1);
SvPOK_on(htm_sv);
SvCUR_set(filesize);
ptr = SvPVX(htm_sv);
Copy(*htmFile, ptr, filesize, char);
XPUSHs(sv_2mortal(htm_sv));

(newSVpvn takes two arguments -- a char* and a STRLEN -- while I only see one in your example... but I assume that's just a typo, as it would have been a compile-time error and there's no equivalent that uses only one arg.)

[syphilis]

My hunch is that there's something in your code (which you haven't provided) that's causing the problem. I believe the following Inline::C script demonstrates that we can pass strings (in both directions) via the stack that are considerably larger than 64kb. At least that's what it demonstrates on my Win32 box - as the script ouptputs "8000000".

If the same script produces the same output on your box, then I think we can conclude that the problem you're facing is something other than a stack size limit.

use warnings;
use strict;

use Inline C => Config =>
    BUILD_NOISY => 1;

use Inline C => <<'EOC';

void foo(char * htmFile, int len) {
     dXSARGS;
     sp = mark;
     XPUSHs(sv_2mortal(newSVpv(htmFile, len)));
     PUTBACK;
     XSRETURN(1);
}

EOC

my $len = 8000000;
my $z = 'z' x $len;

my $copy = foo($z, $len);

print length($copy), "\n";


Cheers,
Rob

[beppu]

http://search.cpan.org/~beppu/Tie-Array-Pointer-0.000059/lib/Tie/Array/Pointer.pm

I need to finish it one of these days.