Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Re^4: In need of a Dumper that has no pretentions to being anything else.

by BrowserUk (Pope)
on Feb 23, 2005 at 02:51 UTC ( #433555=note: print w/ replies, xml ) Need Help??


in reply to Re^3: In need of a Dumper that has no pretentions to being anything else.
in thread In need of a Dumper that has no pretentions to being anything else.

but your request is self-conflicting, and you haven't said exactly what you don't like about Data::Dumper.

Actually, I thought I had spelt out what I was looking for pretty carefully.

I am dealing with a huge (> 500 MB) heavily nested data structure consisting of lots of small hashes and arrays. Data::Dumper

  • consumes huge amounts of memory (pushing my machine into swaping) checking for circular references when I know there will be none.
  • Either dumps everything one element per line indented, or totally flattened without any structure.
  • Produces this
    use Math::Pari qw[ :int factorint sqrtint divisors PARI ]; $f = factorint 1000000; print Dumper $f; $VAR1 = bless( [ bless( [ bless( do{\(my $o = 33884400)}, 'Math::Pari' + ), bless( do{\(my $o = 33884376)}, 'Math::Pari' + ) ], 'Math::Pari' ), bless( [ bless( do{\(my $o = 33884388)}, 'Math::Pari' + ), bless( do{\(my $o = 33884364)}, 'Math::Pari' + ) ], 'Math::Pari' ) ], 'Math::Pari' ); Attempt to free unreferenced scalar: SV 0x19f41f4 at c:\Perl\bin\p1.pl + line 14, <STDIN> line 3. Attempt to free unreferenced scalar: SV 0x19f42cc at c:\Perl\bin\p1.pl + line 14, <STDIN> line 3.

    Or this

    $Data::Dumper::deepcopy=1; print Dumper $f; $VAR1 = bless( [ bless( [ bless( do{\(my $o = 33884400)}, 'Math::Pari' + ), bless( do{\(my $o = 33884376)}, 'Math::Pari' + ) ], 'Math::Pari' ), bless( [ bless( do{\(my $o = 33884388)}, 'Math::Pari' + ), bless( do{\(my $o = 33884364)}, 'Math::Pari' + ) ], 'Math::Pari' ) ], 'Math::Pari' ); Attempt to free unreferenced scalar: SV 0x19f43bc at c:\Perl\bin\p1.pl + line 14, <STDIN> line 5.

    Or this

    $Data::Dumper::Indent=0; print Dumper $f; $VAR1 = bless( [bless( [bless( do{\(my $o = 33884400)}, 'Math::Pari' ) +,bless( do{\(my $o = 33884376)}, 'Math::Pari' )], 'Math::Pari' ),bles +s( [bless( do{\(my $o = 33884388)}, 'Math::Pari' ),bless( do{\(my $o += 33884364)}, 'Math::Pari' )], 'Math::Pari' )], 'Math::Pari' ); Attempt to free unreferenced scalar: SV 0x19f37c8 at c:\Perl\bin\p1.pl + line 14, <STDIN> line 7.

    When what I want is something more akin to this:

    print "[@$_]" for @$f; [2 5] [6 6]

    Or this

    print "[@{[ join', ', map{ \"[@$_]\" } @$f ]}]"; [[2 5], [6 6]]

    Except that there are thousands of arrays at varying depths of nesting.

    I can write one myself, perhaps based around Data::Rmap or similar, but I thought I look and see if there is an existing one available. My search didn't turn up anything promising, but it seems a reasonably simple enough requirement that someone might know or have one already written?


    Examine what is said, not who speaks.
    Silence betokens consent.
    Love the truth but pardon error.


  • Comment on Re^4: In need of a Dumper that has no pretentions to being anything else.
    Select or Download Code
    Re^5: In need of a Dumper that has no pretentions to being anything else.
    by fergal (Chaplain) on Feb 23, 2005 at 06:27 UTC
      Apart from the "Attempt to free unreferenced scalar" stuff (which seems like a bug) what's wrong with this? Math::Pari objects are just a simple scalar blessed into a class as far as Perl is concerned. Nothing is going to dump them out as anything more sensible unless it specififcally knows how to understand Math::Pari objects.

      You'll need to play with DD's Freezer stuff to get anywhere on that.

        Nothing is going to dump them out as anything more sensible unless it specififcally knows how...

        What I'm doing is:

        perl> use Math::Pari qw[factorint];; perl> $f = factorint 1000000;; perl> use overload;; perl> print overload::StrVal $f;; Math::Pari=ARRAY(0x22ac84)

        Once you kick away the overloading, you can see that you have an array and then dump is elements by accessing them, recursing if need, be until you get a non-reference value returned.

        I think this should work for any overloaded reference?


        Examine what is said, not who speaks.
        Silence betokens consent.
        Love the truth but pardon error.
          One problem there is that Math::Pari integers are not just a ref to an integer that's been overloaded. When you see
          bless( do{\(my $o = 33884400)}, 'Math::Pari')
          coming out of DD, the 33884400 here is a pointer to a memory location holding C structure that came out of the Pari lib. Simply printing out whatever you find after derefencing everything won't work. You need something that understands Math::Pari or you need to use DD's Freezer facility to make DD understand Math::Pari.

    Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Node Status?
    node history
    Node Type: note [id://433555]
    help
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others surveying the Monastery: (3)
    As of 2014-09-18 04:13 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      How do you remember the number of days in each month?











      Results (108 votes), past polls