Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Understanding Devel::Peek output

by syphilis (Bishop)
on Nov 29, 2020 at 12:32 UTC ( #11124361=perlquestion: print w/replies, xml ) Need Help??

syphilis has asked for the wisdom of the Perl Monks concerning the following question:


On my 64-bit Windows 7 OS, I have:
1) perl-5.32.0 built using a 32-bit gcc-8.3.0 compiler;
2) perl-5.32.0 built using a 64-bit gcc-8.3.0 compiler.

On both of those perls, the ivtype is the 8-byte "long long".

On the 32-bit build I get:
C:\>perl -MDevel::Peek -e "Dump 1 << 62;" SV = IV(0x1dc8d64) at 0x1dc8d64 REFCNT = 1 FLAGS = (PADTMP,IOK,READONLY,PROTECT,pIOK) IV = 4611686018427387904
Note that the first line contains two occurrences of the same hex value - 0x1dc8d64 in this instance. Every time I run that command on that 32-bit build of perl, the two hex values are always equivalent - though, of course, they're not always 0x1dc8d64.

When I run the same command on the 64-bit build, I get:
C:\>perl -MDevel::Peek -le "Dump 1 << 62;" SV = IV(0x33d430) at 0x33d440 REFCNT = 1 FLAGS = (PADTMP,IOK,READONLY,PROTECT,pIOK) IV = 4611686018427387904
For this build of perl, it is always the case that the first hex value is less than the second value by 0x10.

How is this differing behaviour between the 32-bit build and the 64-bit build explained ?

IIUC, on the 64-bit build, the amount of memory being allocated for the IV is 0x10 (16) bytes more than is necessary ... and I wonder what is responsible for that extravagance.
Is it perl ? ... or the OS ? ... or the compiler ? ...


Replies are listed 'Best First'.
Re: Understanding Devel::Peek output
by dave_the_m (Monsignor) on Nov 29, 2020 at 13:22 UTC
    In general an SV consists of two parts: a head, containing some flags, a reference count and a pointer to the second part: a body, containing various type-specific fields; for example a PV body contains a size, current length and a pointer to a string.

    In fact as an optimisation the head contains one extra general-purpose field, which can hold a pointer, or an IV or whatever. The presence of this field means that for "small" SVs like IVs, a body doesn't have to be allocated. In this case, the head's "pointer to body" in fact points back to itself, but possibly with a small offset to make the alignment of an IV in a body (xiv_u.xivu_iv) the same as the alignment for an IV value in the head (sv_u.svu_iv).


Re: Understanding Devel::Peek output
by ikegami (Pope) on Nov 29, 2020 at 20:39 UTC

    I'll be referring to this PDF, Reini Urban's PerlGuts Illustrated. It contains all the information needed to answer your question, but I'll arrange in the information in a more convenient format here.

    There isn't just one type of scalar. See the graph at the bottom of the first page of the linked document. The ident that follows "SV =" indicates the type of the scalar. In this case, we have a SVt_IV.

    The type of scalar controls what kind of values can be stored in that scalar. In a SVt_IV, one can store an IV (signed int) or a UV (unsigned int), but not a string or floating point number.

    So what happens if you want to store a string inside the scalar? Well, Perl must first upgrade the scalar to a type that can handle the string. In this case, it will switch to a SVt_PVIV which can store a string in addition to a IV or UV.

    $ perl -MDevel::Peek -e'my $x = 123; Dump($x); $x = "abc"; Dump($x);' SV = IV(0x2764760) at 0x2764770 REFCNT = 1 FLAGS = (IOK,pIOK) IV = 123 SV = PVIV(0x2766c38) at 0x2764770 REFCNT = 1 FLAGS = (POK,IsCOW,pPOK) IV = 123 PV = 0x27c1258 "abc"\0 CUR = 3 LEN = 10 COW_REFCNT = 1

    Tangent: Note that while the upgraded scalar can hold both a string and an IV/UV, it only contains a string because POK is set but not IOK. In other words, while the slot for an IV exists (and holds the old IV value), the slot isn't storing any part of the scalar's value at this time because of the flags.

    The scalar is potentially referenced by multiple pointers, so how does Perl manage to upgrade the scalar to a larger one without invalidating those pointers? It does so by using two memory blocks to represent a scalar.

    The second address in the dump is the address of the head. This is what the various pointers reference. As you can see, the address of the head didn't change when the scalar was upgraded.

    Inside the head is a pointer to the second block, called the body. This is the first address in the dump. Upgrading a scalar upgrades a field in the head indicating the type of the scalar, and it replaces this block with a larger one.

    Now, let's look at the 64-bit example.

    Memory allocation is relatively expensive, so SVt_IV and SVt_RV utilize a hack to avoid memory allocation.

    One would expects a SVt_IV scalar to look like this:

    Head Body +---------------+ +---------------+ | Ptr to body -------> | IV/UV slot | +---------------+ +---------------+ | Ref count | +---------------+ | Flags & type | +---------------+ | Unused | +---------------+

    Instead, it looks like this:

    Head Body +---------------+ +---------------+ | Ptr to body -------> | Unused | +---------------+ +---------------+ | Ref count | | Unused | +---------------+ +---------------+ | Flags & type | | Unused | +---------------+ +---------------+ | Unused | | Unused | +---------------+ +---------------+ | IV/UV slot | +---------------+

    Buy why? Perl gets (probably significant) performance gains by keeping the IV slot at the same offset into the body of a scalar regardless of the type of the scalar.

    But I talked of a hack to avoid allocating a second memory block. We can avoid a memory allocation by overlapping both blocks!

    + - - - - - - - + Unallocated <--+ Head +---------------+ | | Ptr to body -------+ +---------------+ | Ref count | +---------------+ | Flags & type | +---------------+ | IV/UV slot | +---------------+

    Now we have a single memory block that points to itself. This is found at the very top of page 12 of the linked document.

    Finally, the answer to your question.

    The size of some fields in the head and body are independent of architecture (especially since you set the integer size to 64-bits for both builds), and the size of other fields in the head and body are dependent of architecture (e.g. the size of a pointer).

    The difference in the offset is based on differences in the total size of the head and body in the two builds.

      The difference in the offset is based on differences in the total size of the head and body in the two builds.

      Yep - thanks ikegami and dave_the_m.
      In the course of pondering the reason that #18363 was afflicting only x64 -Duselongdouble builds, I was idly doing some Devel::Peek::Dump() operations when I noticed the puzzling behaviour.
      I couldn't think of a reason that the SvIV should be exhibiting the differing behaviour between the two architectures ... but, as you've pointed out, ptrsize will change. (Another one I can think of is sizesize.)

Re: Understanding Devel::Peek output
by perlfan (Vicar) on Nov 30, 2020 at 18:55 UTC
    Outside of the memory addressing related question you have, it is worth noting for future readers that the essential value of the IV struct's meta data is the same, namely:
    REFCNT = 1 FLAGS = (PADTMP,IOK,READONLY,PROTECT,pIOK) IV = 4611686018427387904

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://11124361]
Approved by Athanasius
Front-paged by Corion
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (9)
As of 2021-01-22 13:27 GMT
Find Nodes?
    Voting Booth?