Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Data::Dumper turns floating points numbers into strings

by Hue-Bond (Priest)
on Jun 14, 2011 at 17:07 UTC ( #909619=perlquestion: print w/ replies, xml ) Need Help??
Hue-Bond has asked for the wisdom of the Perl Monks concerning the following question:

Esteemed monks,

After playing with JSON::RPC::Client and obtaining the error "value is type str, expected real" (not sure if the error comes from client validation or from the server but that doesn't matter) I investigated and found that Data::Dumper was turning my floating point parameters into strings. I tried changing a couple of configuration variables listed in the module's documentation and was surprised to find that Useperl solves the problem:

use warnings; use strict; use Data::Dumper; use JSON; my $json = JSON->new; for my $up (0, 1) { print "useperl: $up\n"; $Data::Dumper::Useperl = $up; my $data = { foo => 'bar', qux => 0.42 }; printf "before: '%s'\n", $json->encode ($data); Dumper $data; #Data::Dumper->Dump ([$data], ['data']); printf "after: '%s'\n", $json->encode ($data); } __END__ useperl: 0 before: '{"qux":0.42,"foo":"bar"}' after: '{"qux":"0.42","foo":"bar"}' useperl: 1 before: '{"qux":0.42,"foo":"bar"}' after: '{"qux":0.42,"foo":"bar"}'

Both Dumper and Data::Dumper->Dump show this effect. When working with integers, we get more consistent results:

use JSON; my $json = JSON->new; for my $up (0, 1) { print "useperl: $up\n"; $Data::Dumper::Useperl = $up; my $data = { foo => 'bar', qux => 42 }; printf "before: '%s'\n", $json->encode ($data); Dumper $data; #Data::Dumper->Dump ([$data], ['data']); printf "after: '%s'\n", $json->encode ($data); } __END__ useperl: 0 before: '{"qux":42,"foo":"bar"}' after: '{"qux":42,"foo":"bar"}' useperl: 1 before: '{"qux":42,"foo":"bar"}' after: '{"qux":42,"foo":"bar"}'

Is this difference in behaviour to be expected?

--
 David Serrano
 (Please treat my english text just like Perl code, i.e. feel free to notify me of any syntax, grammar, style and/or spelling errors. Thank you!).

Comment on Data::Dumper turns floating points numbers into strings
Select or Download Code
Re: Data::Dumper turns floating points numbers into strings
by ikegami (Pope) on Jun 14, 2011 at 17:16 UTC

    When one stringifies a scalar, the string is attached to the scalar as an optimisation. It doesn't remove the existing number. This isn't a bug in D::D, but it doesn't mean that D::D couldn't be changed to avoid adding a string to the scalar. It already does in other circumstances.

    (pPOK = has string) >perl -MDevel::Peek -MData::Dumper -e"$_=0.42; Dumper($_); Dump($_);" SV = NV(0x3335fc) at 0x328fe4 REFCNT = 1 FLAGS = (NOK,pNOK) NV = 0.42 >perl -MDevel::Peek -MData::Dumper -e"$_={ qux => 42 }; Dumper($_); Du +mp($_->{qux});" SV = IV(0x57b260) at 0x57b264 REFCNT = 1 FLAGS = (IOK,pIOK) IV = 42 >perl -MDevel::Peek -MData::Dumper -e"$_={ qux => 0.42 }; Dumper($_); +Dump($_->{qux});" SV = PVNV(0x5490e4) at 0x25b264 REFCNT = 1 FLAGS = (NOK,POK,pNOK,pPOK) IV = 0 NV = 0.42 PV = 0x4c9f84 "0.42"\0 CUR = 4 LEN = 36

    The problem is that JSON doesn't allow you to specify how to handle a variable that's both a float and a string.

Re: Data::Dumper turns floating points numbers into strings
by ig (Vicar) on Jun 14, 2011 at 18:43 UTC

    Note that this effect isn't unique to Data::Dumper. Any operation that stringifies a number or integer will change how JSON (the pure Perl implementation, I didn't look at the XS) represents it.

    use strict; use warnings; use JSON; my $json = JSON->new; my $data = { foo => 'bar', qux => 0.42 }; printf "before: '%s'\n", $json->encode($data); print "$data->{qux}\n"; printf "after: '%s'\n", $json->encode($data);

    gives

    before: '{"qux":0.42,"foo":"bar"}' 0.42 after: '{"qux":"0.42","foo":"bar"}'

    This problem isn't unique to JSON either. I have seen similar issues in Win32::OLE, for example. Anything that inspects the scalar type and behaves differently depending on what it finds must resolve the ambiguity, shown by ikegami / Devel::Peek, that sometimes exists. When using such modules, you must be careful with your data.

    One way to avoid the problem would be to make a copy of your data and stringify the copy.

Re: Data::Dumper turns floating points numbers into strings
by Hue-Bond (Priest) on Jun 14, 2011 at 22:41 UTC

    Thank you both for the explanation about the stringification. Now the weird thing is that the Perl implementation of Data::Dumper doesn't add this string representation, while the XS one does...

    --
     David Serrano
     (Please treat my english text just like Perl code, i.e. feel free to notify me of any syntax, grammar, style and/or spelling errors. Thank you!).

Re: Data::Dumper turns floating points numbers into strings
by mje (Curate) on Jun 15, 2011 at 08:18 UTC

    I had a similar problem with JSON::XS and DBD::Oracle. JSON::XS looks at the scalar and basically does if SvPOKp stringify else if SvNOKp treat as float else if SvIOKp treat as integer.

    The problem is that DBD::Oracle does not allow you to bind a column as an integer. The object I wanted to JSONify was very large so all those extra " around integers were adding a lot to the size of the JSON encoded object. To get rid of the " I had to loop through my data and for every integer I had to add 0 to it - this took time. In the end we changed DBD::Oracle to add options to discard the pv.

    I documented what we found and did at Support binding of integers in DBD::Oracle so they are returned as IVs

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://909619]
Approved by sundialsvc4
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (4)
As of 2015-07-03 16:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (54 votes), past polls