Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re: In need of a Dumper that has no pretentions to being anything else.

by halley (Prior)
on Feb 23, 2005 at 04:42 UTC ( #433573=note: print w/ replies, xml ) Need Help??


in reply to In need of a Dumper that has no pretentions to being anything else.

Remember, many people do like Data::Dumper because it does presume to be a serialization standard, albeit in text form. It is mission critical for a lot of small glue applications out there.

If I read fergal's patch right, then you'd be removing the circular reference protection for everyone who turns off Deepcopy. Just because BrowserUk doesn't break up his data for analysis doesn't mean everyone should have a new behavior here. DD should still protect the iterator from cycling around forever, even if it doesn't produce reference syntax in its output.

I implemented a fourth format style to Data::Dumper once, which tries to fit whole arrays on one line if possible, or word-wrap the elements of arrays on a minimum number of lines if the elements are non-references.

But two bad things happened: (1) I lost that patch at some point, and (2) the stock DD is now not written in Perl but in native code. Changing the Perl version is a waste of effort.

I think the answer here is not to wade through 500_000 elements written in Perl syntax, looking for a programmatic error on your part. Instead, formulate some theories as to the fault and analyze the structure with a couple of lines of Perl, or find a smaller dataset which exhibits the fault.

Also, turn on Terse, turn off Purity, and consider overloading the string-izing operator for certain objects so you don't get so much bless(do{...},'Math::Pari') noise.

--
[ e d @ h a l l e y . c c ]


Comment on Re: In need of a Dumper that has no pretentions to being anything else.
Download Code
Re^2: In need of a Dumper that has no pretentions to being anything else.
by BrowserUk (Pope) on Feb 23, 2005 at 05:44 UTC

    Please note that I didn't pass any judgement upon Data::Dumper or any other existing module--I just simply asked if there was an alternative that fit my needs.

    And tried to head off the offers of the "other" modules that I have already looked at, that also do not fit my needs.

    Just because BrowserUk doesn't break up his data for analysis...

    I have a requirement. I asked if anyone knew of a module that fit those requirements. You have no concept of what I am doing, or why I want this format.

    For the record, I am not "looking for a programmatic error". I am looking for patterns in the data.

    You'd be surprised at how the human eye and brain can detect patterns in data, even if the text is displayed in a very small (unreadable) font, and the data is scrolling quite fast. Once I perceive a pattern of some kind, I can then increase the font size and view the boundaries and repeat points of the pattern, so that I can then write code to pick out those boundaries and then plot the data graphically for further analysis.

    For an example of something similar, download and run the code I posted at Re: Testing for randomness (spectral testing). Try uncommenting the constant line flagged as ## Really BAD!! and then allow the program to cycles for 2 or 3 minutes and see how it immediately shows up just how bad a Linear Congruential PRNG is with badly chosen values.

    Then for contrast, install Math::Random::MT and run that for a few minutes (or hours or days) and see that the spectral test generated remains almost totally even, with just enough evenly distributed variation to show true randomness.

    The whole point of the exercise, is to view a large volumes of data, together, in a consistent and repetative format, so that any patterns become obvious. So please, keep your supercilious and judgemental comments in your closet where they belong.


    Examine what is said, not who speaks.
    Silence betokens consent.
    Love the truth but pardon error.
      (1) I'm not going to dig any more into your task; I could not care less what the task is. But if you happen to be developing a PRNG and you do find any patterns, then it IS a programming error. A common one, an important one, but a flaw nonetheless.

      (2) I downvoted you for your last sentence. I think your whole thread smacks of being frustrated, and you're letting it get the best of you. Think rationally, not emotionally, and you wouldn't be in this situation. This is a forum, and you brought your frustration to the forum. Don't tell us to shut up. I don't care how you vote my response, but keep the discourse civil, thanks.

      --
      [ e d @ h a l l e y . c c ]

        Updated: To clarify what I meant by "obtuse".

        (1) I'm not going to dig any more into your task; I could not care less what the task is. But if you happen to be developing a PRNG and you do find any patterns, then it IS a programming error. A common one, an important one, but a flaw nonetheless.

        I am not and was not "developing a PRNG". I demonstrated a method--spectral testing--whereby the flaws in PRNGs can be detected quickly and simply without recourse to extensive numerical analysis.

        Had you bothered to follow the link and read post, that would have been obvious, but I guess you'd rather hide behind your mis-assumptions.

        (2) I downvoted you for your last sentence. I think your whole thread smacks of being frustrated, and you're letting it get the best of you. Think rationally, not emotionally, and you wouldn't be in this situation. This is a forum, and you brought your frustration to the forum. Don't tell us to shut up. I don't care how you vote my response, but keep the discourse civil, thanks.

        I didn't tell any "us" to shut up. I asked you to refrain from snide remarks.

        And you would contend that your post was rational, civil, unemotional?


        Examine what is said, not who speaks.
        Silence betokens consent.
        Love the truth but pardon error.

        Considered: Tanktalus: Delete: flamebait
        Unconsidered: ysth: enough keep votes - Keep/Edit/Delete: 14/0/4

Re^2: In need of a Dumper that has no pretentions to being anything else.
by fergal (Chaplain) on Feb 23, 2005 at 06:22 UTC
    My patch doesn't remove circular ref protection. Circular ref protection comes from keeping a track of all the refs we've seen so far and I've not touched that.

    DD also keeps track of all scalars that it sees so that when you run this

    use Data::Dumper; my %s=('key',1); my $s=\$s{'key'}; print Dumper([\%s, $s])
    you get
    $VAR1 = [ { 'key' => 1 }, \$VAR1->[0]{'key'} ];
    that is, it was able to spot that the 1 in %s{key} is actually the same 1 that's in $s.

    My patch removes this ability when $Deepcopy = 0. This is consistent with the rest of the deepcopy behaviour and means that now that the only thing that can cause a backreference to some other part of the structure is circularity.

      Fair enough-- the patch doesn't give a lot of context and I didn't look at the whole. It's squirrelly code. I was just putting in a note that any quick change could break a lot of people.

      --
      [ e d @ h a l l e y . c c ]

      Note that DD only gets it right with print Dumper([\%s, $s]) but not with print Dumper([$s, \%s])

      For that you need a better dumper. I wonder if its arguable that the tracking you are talking about disabling is actually not required for Purity=0 dumps.

      ---
      demerphq

        Depends on what you mean by "right". I get
        $VAR1 = [ \1, { 'key' => ${$VAR1->[0]} } ];
        which can't be evaluated correctly by Perl's eval but it does contain all the necessary information. A human can see what's going on and it's probably possible to write something that would reconstruct the correct sturucture but it would involve some sort of constraint solver.

        Purity just controls whether it should be compatible with eval(). Even with Purity=0, this information should be preserved otherwise certain structures would have identical output even though they're not identical.

        Deepcopy controls whether we care about cross references so it definitely should be dropped when you have Deepcopy=1.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://433573]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (8)
As of 2014-08-01 10:57 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Who would be the most fun to work for?















    Results (5 votes), past polls