Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic

Pure Perl Internals (with Pure Perl Segfaults)

by Ovid (Cardinal)
on Mar 11, 2004 at 22:06 UTC ( #335993=perlmeditation: print w/replies, xml ) Need Help??

So you've always been curious about the internals of Perl, right? Maybe you're afraid of C or maybe you're on a Windows box and you don't have a C compiler handy, so you've not gotten around to it. Well, as it turns out, Brian Ingerson had an itch to scratch and he scratched it. Using his new Pointer module (not even remotely the same thing as Devel::Pointer), he's written (in pure Perl) a module that let's you explore the Perl internals.

Taking an incredibly brief tour of how the internals work, let's see how a string is stored. (If you want to learn more, see Perlguts Illustrated).

First of all, a basic, null scalar (my $x;) looks something like this:

| ANY           |
| REFCOUNT      |

"ANY" is a pointer to more data, but in the case of a an undefined scalar (SvNULL), it's not pointing to anything. The other fields hold various bits of information that are used for garbage collection, identifying the type of scalar, whether or not it's blessed, etc.

What if you want to store a string? In that case, you have "ANY" point to an SvPV struct, whose PVX pointer points to the actual array of characters representing the string (I can't remember what all of the shorthands mean, either).

              sv       svpv
+---------------+   +-----+
| ANY           |-->| PVX |-->|H|e|l|l|o|,| |w|o|r|l|d|\0| | |
+---------------+   +-----+
| REFCOUNT      |   | CUR |
+---------------+   +-----+
| FLAGS  | TYPE |   | LEN |
+---------------+   +-----+

Note that the first two structs each contain one pointer. How do we get to them? First, create a pointer of "Hello, world":

use Pointer; my $x = "Hello, world"; my $sv = pointer->of_scalar($x); my $svpv = $sv->get_pointer; my $pointer_to_string = $svpv->get_pointer; print $pointer_to_string->get_string; # or print pointer->of_scalar("Hello, world")->get_pointer->get_pointer->ge +t_string;

As you begin to learn more about Perl internals, one of the things that you discover is that Perl can store several representations of a variable internally. For instance, the following can still print "Hello, world", even though the apparent value of $x has been altered.

my $x = "Hello, world"; $x = 42; my $sv = pointer->of_scalar($x); my $svpv = $sv->get_pointer; my $pointer_to_string = $svpv->get_pointer; print $pointer_to_string->get_string;

How does that work? Well, now the scalar looks sort of like this:

              sv     svpviv
+---------------+   +-----+
| ANY           |-->| PVX |-->|H|e|l|l|o|,| |w|o|r|l|d|\0| | |
+---------------+   +-----+
| REFCOUNT      |   | CUR |
+---------------+   +-----+
| FLAGS  | TYPE |   | LEN |
+---------------+   +-----+
                    | IVX |

Now the second struct (svpviv) is similar to the SvPV struct, but it has an extra field that contains the integer value. However, it still has a pointer to the original string value.

Now what happens if we try to get the string value of a scalar which has never had a string assigned to it?

$ perl -MPointer -le 'print pointer->of_scalar(42)->get_pointer->get_p +ointer->get_string;' Segmentation fault

A segfault with pure Perl :)

Pointer is fairly new and the documentation (and tests!) could use some work. For example, I could not figure out, from the docs, how to get the integer out of an SpIV, or how to get at the refcount. Still, with a copy of Perlguts illustrated and this module, you can have fun exploring Perl internals. I'm looking forward to seeing this module developed further.

Update: A quick email exchange with Ingy revealed how to get an integer from an SpIV:

perl -MPointer -MPointer::int -le 'print ((pointer->of_scalar(42)->get +_pointer("int") + 3)->get)' 42


New address of my CGI Course.

Replies are listed 'Best First'.
Re: Pure Perl Internals (with Pure Perl Segfaults)
by diotalevi (Canon) on Mar 11, 2004 at 23:54 UTC
    Why not use B and Devel::Peek and maybe Devel::Pointer::PP? In fact, does this work anyplace that Emily Dickenson in perl doesn't? It looks like Brian was assuming a standard layout in memory but I don't think this is portable to big endian cpus or to compilers with different packing or reordering.

      It's pure Perl (only your last example is) and it's the start of a clean, OO method of walking through Perl's structs. Frankly, I can't think of any reason I would use this (Ingy needed a pure Perl method of determining if a scalar contained an integer or a string), but it's fun (to me).

      As for portability, Ingy admits that this is alpha code, but he's using Config to try to make it as portable as possible.


      New address of my CGI Course.

        Eh. Sure B is compiled but by being so, it doesn't have to guess at how the compiler that constructed the interpreter arranged things in memory nor does it have to care about big vs little endian. If you do it in pure perl then you have to solve both of those problems as well.

        ... Oh I see. That's nicely done.

        Ingy needed a pure Perl method of determining if a scalar contained an integer or a string
        While his results are cool, he could have saved a lot of work if he'd asked around to see if anyone had already found a solution to this problem (note the unary "~", it's important)
        sub is_integer { ~$_[0] !~ /\D/ }
        Here are some tests to demonstrate that it works:
        #!/usr/bin/perl -wT use strict; use Test; BEGIN { plan tests => 345 } for my $num (-56789, -300, -1, 0..100, 5345, 6574572, 23457356) { ok( is_integer($num) ); my $string = "$num"; ok( not is_integer($string) ); } for my $string ((map { chr($_) } 0..127), 'dog', 'cat', 'mouse') { ok( not is_integer($string) ); } sub is_integer { ~$_[0] !~ /\D/ }
        I found the is_integer() implementation in my cool-snippets-from-perlmonks directory, but can't seem to find the original thread.....


Re: Pure Perl Internals (with Pure Perl Segfaults)
by Aristotle (Chancellor) on Mar 12, 2004 at 00:03 UTC
    Mightn't we be able to write C library wrappers in pure Perl eventually with this thing? That's the most interesting application I can think of. (Who needs Perl6. :) )

    Makeshifts last the longest.

Re: Pure Perl Internals (with Pure Perl Segfaults)
by stvn (Monsignor) on Mar 12, 2004 at 18:41 UTC

    It certainly looks really cool. But after playing around with Perl internals (more specifically the B:: modules), I got used to the somewhat convoluted, but ultimately logical (sorta) naming conventions of Perl's guts. So the fact that this introduces another interface to the mix, I am not so fond of. I wouldn't mind if maybe it just expanded the exisiting abbreviations though, that would actually be better IMHO.

    Cool stuff none the less.


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://335993]
Approved by rob_au
Front-paged by kvale
[1nickt]: Corion HPs? ugh. I was impressed with Lenovo's gaming laptops; if it weren't for the red backlighting and general flashiness of the aesthetic, I might have gone with that. But all that was until I discovered that the Dell Precision line is still around.

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (15)
As of 2017-03-27 13:39 GMT
Find Nodes?
    Voting Booth?
    Should Pluto Get Its Planethood Back?

    Results (320 votes). Check out past polls.