Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Data::Dumper issue

by jesuashok (Curate)
on Jun 07, 2006 at 15:32 UTC ( #554053=perlquestion: print w/ replies, xml ) Need Help??
jesuashok has asked for the wisdom of the Perl Monks concerning the following question:

dear monks,

I have encountered a situation where the output of Data::Dumper cannot be incorporated in a script to restore the original data, specifically where a string contains the sequence ^M^J and $Useqq is unset. (Note: this arises in my work because I have packed numbers in a large data structure, and on a few occasions the packed sequence happens to include ^M^J as a subsequence.)

The issue arises because perl seems to have a "friendly" way to handle DOS files on Unix platforms. My guess is that at a really early stage of reading the script file, ^M^J is translated to ^J, presumably before perl is even parsing the input. Obviously, it is ignoring the single quotes in this example, since the single quotes should suppress any interpretation of the input. (Note that I realize this is generally a desirable behaviour, so I think the bug is with Data::Dumper, not with how files are read.)

Note also that, as my script shows, this is not a problem when we use eval on the output of dump, but only when the code is saved to a file a compiled back from the file. (Unfortunately for me, I wish to save the dump to a file for frequent reuse, so that doesn't help me much...)

use Data::Dumper; $Data::Dumper::Purity = 1; # This makes no difference my $a = "\x0d\x0a"; print "Initial length: ", length($a), "\n"; my $dump = Data::Dumper->Dump([$a], [qw(a)]); print "Dumped code with Useqq = 0:\n", $dump; eval $dump; print "New length after eval: ", length($a), "\n"; $Data::Dumper::Useqq = 1; $dump = Data::Dumper->Dump([$a], [qw(a)]); print "Dumped code with Useqq = 1:\n", $dump; eval $dump; print "New length after eval: ", length($a), "\n";
Out put is as follows :-
Initial length: 2 Dumped code with Useqq = 0: $a = ' '; New length after eval: 2 Dumped code with Useqq = 1: $a = "\r\n"; New length after eval: 2
what is the problem when $Useqq = 0. It seems, that does not work as expected.

your thoughts on the same.

"Keep pouring your ideas"

2006-10-16 Unapproved by planetscape once evidence of habitual plagiarism uncovered.

Comment on Data::Dumper issue
Select or Download Code
Re: Data::Dumper issue
by Joost (Canon) on Jun 07, 2006 at 15:37 UTC
      Hi Joost
      # This is dump's output when $Useqq = 0. -- Doesn't work! $a = ' '; print "New length after dumped code (with Useqq = 0): ", \ length($a), "\n"; # This is dump's output when $Useqq = 1. -- Works correctly $a = "\r\n"; print "New length after dumped code (with Useqq = 1): ", \ length($a), "\n";
      "Keep pouring your ideas"
        You have still failed to explain how this "doesn't work." It works fine as far as I can tell.
        $a in your original code consists of a carriage return and a newline character - so that works. The fact that you can't readily see it doesn't matter.

        The output of your original program looks like this in my editor (vim on linux):

        Initial length: 2 Dumped code with Useqq = 0: $a = '^M '; New length after eval: 2 Dumped code with Useqq = 1: $a = "\r\n"; New length after eval: 2
        Note the ^M (carriage return character)
Re: Data::Dumper issue
by liverpole (Monsignor) on Jun 07, 2006 at 15:42 UTC
    Hi jesuashok,

        It seems, that does not work as expected.

    I guess the $64,000 question is "how did you *expect* it to work?"

    I get the same thing when I run it, but it seems that assigning Useqq to a nonzero value will cause whitespace to be represented differently (as well as "unsafe" characters).  And that seems to be exactly what's happening, no?


    s''(q.S:$/9=(T1';s;(..)(..);$..=substr+crypt($1,$2),2,3;eg;print$..$/
      Hi liverpole

      Thank you for your Reply.

      I request you to read the reply which I have given to Joost.

      "Keep pouring your ideas"
        I have read the reply, but must confess that I'm still confused.

        You say:

        # This is dump's output when $Useqq = 0. -- Doesn't work! $a = ' ';

        But that's how Data::Dumper has always worked for me, and it seems like it's doing the exact right thing!  Specifically, why do you think that assigning $a to the two characters <return><linefeed> (or ^M^J) would give you anything other than the output you're seeing?


        s''(q.S:$/9=(T1';s;(..)(..);$..=substr+crypt($1,$2),2,3;eg;print$..$/
Re: Data::Dumper issue
by Joost (Canon) on Jun 07, 2006 at 17:07 UTC
    After reading your update:

    I have encountered a situation where the output of Data::Dumper cannot be incorporated in a script to restore the original data, specifically where a string contains the sequence ^M^J and $Useqq is unset. (Note: this arises in my work because I have packed numbers in a large data structure, and on a few occasions the packed sequence happens to include ^M^J as a subsequence.)
    This should "just work", but you should use binmode() on a filehandle that you intend to read/write binary data to.

    The issue arises because perl seems to have a "friendly" way to handle DOS files on Unix platforms. My guess is that at a really early stage of reading the script file, ^M^J is translated to ^J, presumably before perl is even parsing the input.
    Perl does no such thing (edit: on unix). However, writing "\n" on a non-binmode()d filehandle on windows will write a return/linefeed pair and vice versa on reads. On (most?) unixes, there is no difference between binary files and text files.

    Obviously, it is ignoring the single quotes in this example, since the single quotes should suppress any interpretation of the input. (Note that I realize this is generally a desirable behaviour, so I think the bug is with Data::Dumper, not with how files are read.)
    Your example simply fails to demonstrate anything like that since you're not reading/writing any files. The output of Data::Dumper is correct, as me and others have already pointed out to you.

    Note also that, as my script shows, this is not a problem when we use eval on the output of dump
    So Data::Dumper is in fact working correctly.
    ..., but only when the code is saved to a file a compiled back from the file. (Unfortunately for me, I wish to save the dump to a file for frequent reuse, so that doesn't help me much...)

    And you're not showing us how you're reading/writing that file. I strongly suspect the error is somewhere in the code you're not showing.

Re: Data::Dumper issue
by runrig (Abbot) on Jun 07, 2006 at 23:48 UTC
    Maybe Storable would be a better choice?
Re: Data::Dumper issue = PLAGIARISM
by liverpole (Monsignor) on Oct 16, 2006 at 14:12 UTC
    Now I understand why jesuashok is asking this question!  He has run into the EXACT same problem as Eric Joanis did on this perl5 porter newsgroup submission (but written over 4 years ago, on March 2, 2002).

    And by "EXACT", I mean word-for-word the same:

    ----------------------------------------------------------------- [Please enter your report here] I have encountered a situation where the output of Data::Dumper cannot + be incorporated in a script to restore the original data, specifically wh +ere a string contains the sequence ^M^J and $Useqq is unset. Please find attached a script which illustrates the problem. (Note: this arises +in my work because I have packed numbers in a large data structure, and o +n a few occasions the packed sequence happens to include ^M^J as a subsequence.) The issue arises because perl seems to have a "friendly" way to handle + DOS files on Unix platforms. My guess is that at a really early stage of reading the script file, ^M^J is translated to ^J, presumably before p +erl is even parsing the input. Obviously, it is ignoring the single quote +s in this example, since the single quotes should suppress any interpretati +on of the input. (Note that I realize this is generally a desirable behaviour, so I think the bug is with Data::Dumper, not with how files + are read.) Note also that, as my script shows, this is not a problem when we use +eval on the output of dump, but only when the code is saved to a file a compiled back from the file. (Unfortunately for me, I wish to save th +e dump to a file for frequent reuse, so that doesn't help me much...) An obvious solution is to set $Useqq to 1, but it seems to me this mod +ule ought to output correct code in all cases, so I'm still submitting thi +s as a bug report. A somewhat ugly solution is to use something like this: Replace 'stuff^M^Jmorestuff' by 'stuff' . "\r\n" . 'morestuff' I don't really like it, but I can't think of any other way to get arou +nd the fact that single quotes don't permit any escape sequences (other t +han \\ and \'). Here is the script I wrote to illustrate the problem: #!/pkgs/perl-5.006/bin/sparc-sun-solaris2.5.1/perl -w # This short script illustrates a bug with Data::Dumper, where it will # create code that does not correctly reconstruct the data structure # dumped when copied into source code. use Data::Dumper; $Data::Dumper::Purity = 1; # This makes no difference my $a = "\x0d\x0a"; print "Initial length: ", length($a), "\n"; my $dump = Data::Dumper->Dump([$a], [qw(a)]); print "Dumped code with Useqq = 0:\n", $dump; eval $dump; print "New length after eval: ", length($a), "\n"; $Data::Dumper::Useqq = 1; $dump = Data::Dumper->Dump([$a], [qw(a)]); print "Dumped code with Useqq = 1:\n", $dump; eval $dump; print "New length after eval: ", length($a), "\n"; # This is dump's output when $Useqq = 0. -- Doesn't work! $a = ' '; print "New length after dumped code (with Useqq = 0): ", length($a), " +\n"; # This is dump's output when $Useqq = 1. -- Works correctly $a = "\r\n"; print "New length after dumped code (with Useqq = 1): ", length($a), " +\n"; [Please do not change anything below this line] -----------------------------------------------------------------

    For more information about jesuashok's habit of wholesale "borrowing" of material from other websites, please refer to this node.


    s''(q.S:$/9=(T1';s;(..)(..);$..=substr+crypt($1,$2),2,3;eg;print$..$/

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://554053]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (7)
As of 2014-12-25 09:50 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (159 votes), past polls