Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Data::Dumper turns floating points numbers into strings

by Hue-Bond (Priest)
on Jun 14, 2011 at 17:07 UTC ( #909619=perlquestion: print w/ replies, xml ) Need Help??
Hue-Bond has asked for the wisdom of the Perl Monks concerning the following question:

Esteemed monks,

After playing with JSON::RPC::Client and obtaining the error "value is type str, expected real" (not sure if the error comes from client validation or from the server but that doesn't matter) I investigated and found that Data::Dumper was turning my floating point parameters into strings. I tried changing a couple of configuration variables listed in the module's documentation and was surprised to find that Useperl solves the problem:

use warnings; use strict; use Data::Dumper; use JSON; my $json = JSON->new; for my $up (0, 1) { print "useperl: $up\n"; $Data::Dumper::Useperl = $up; my $data = { foo => 'bar', qux => 0.42 }; printf "before: '%s'\n", $json->encode ($data); Dumper $data; #Data::Dumper->Dump ([$data], ['data']); printf "after: '%s'\n", $json->encode ($data); } __END__ useperl: 0 before: '{"qux":0.42,"foo":"bar"}' after: '{"qux":"0.42","foo":"bar"}' useperl: 1 before: '{"qux":0.42,"foo":"bar"}' after: '{"qux":0.42,"foo":"bar"}'

Both Dumper and Data::Dumper->Dump show this effect. When working with integers, we get more consistent results:

use JSON; my $json = JSON->new; for my $up (0, 1) { print "useperl: $up\n"; $Data::Dumper::Useperl = $up; my $data = { foo => 'bar', qux => 42 }; printf "before: '%s'\n", $json->encode ($data); Dumper $data; #Data::Dumper->Dump ([$data], ['data']); printf "after: '%s'\n", $json->encode ($data); } __END__ useperl: 0 before: '{"qux":42,"foo":"bar"}' after: '{"qux":42,"foo":"bar"}' useperl: 1 before: '{"qux":42,"foo":"bar"}' after: '{"qux":42,"foo":"bar"}'

Is this difference in behaviour to be expected?

--
 David Serrano
 (Please treat my english text just like Perl code, i.e. feel free to notify me of any syntax, grammar, style and/or spelling errors. Thank you!).

Comment on Data::Dumper turns floating points numbers into strings
Select or Download Code
Re: Data::Dumper turns floating points numbers into strings
by ikegami (Pope) on Jun 14, 2011 at 17:16 UTC

    When one stringifies a scalar, the string is attached to the scalar as an optimisation. It doesn't remove the existing number. This isn't a bug in D::D, but it doesn't mean that D::D couldn't be changed to avoid adding a string to the scalar. It already does in other circumstances.

    (pPOK = has string) >perl -MDevel::Peek -MData::Dumper -e"$_=0.42; Dumper($_); Dump($_);" SV = NV(0x3335fc) at 0x328fe4 REFCNT = 1 FLAGS = (NOK,pNOK) NV = 0.42 >perl -MDevel::Peek -MData::Dumper -e"$_={ qux => 42 }; Dumper($_); Du +mp($_->{qux});" SV = IV(0x57b260) at 0x57b264 REFCNT = 1 FLAGS = (IOK,pIOK) IV = 42 >perl -MDevel::Peek -MData::Dumper -e"$_={ qux => 0.42 }; Dumper($_); +Dump($_->{qux});" SV = PVNV(0x5490e4) at 0x25b264 REFCNT = 1 FLAGS = (NOK,POK,pNOK,pPOK) IV = 0 NV = 0.42 PV = 0x4c9f84 "0.42"\0 CUR = 4 LEN = 36

    The problem is that JSON doesn't allow you to specify how to handle a variable that's both a float and a string.

Re: Data::Dumper turns floating points numbers into strings
by ig (Vicar) on Jun 14, 2011 at 18:43 UTC

    Note that this effect isn't unique to Data::Dumper. Any operation that stringifies a number or integer will change how JSON (the pure Perl implementation, I didn't look at the XS) represents it.

    use strict; use warnings; use JSON; my $json = JSON->new; my $data = { foo => 'bar', qux => 0.42 }; printf "before: '%s'\n", $json->encode($data); print "$data->{qux}\n"; printf "after: '%s'\n", $json->encode($data);

    gives

    before: '{"qux":0.42,"foo":"bar"}' 0.42 after: '{"qux":"0.42","foo":"bar"}'

    This problem isn't unique to JSON either. I have seen similar issues in Win32::OLE, for example. Anything that inspects the scalar type and behaves differently depending on what it finds must resolve the ambiguity, shown by ikegami / Devel::Peek, that sometimes exists. When using such modules, you must be careful with your data.

    One way to avoid the problem would be to make a copy of your data and stringify the copy.

Re: Data::Dumper turns floating points numbers into strings
by Hue-Bond (Priest) on Jun 14, 2011 at 22:41 UTC

    Thank you both for the explanation about the stringification. Now the weird thing is that the Perl implementation of Data::Dumper doesn't add this string representation, while the XS one does...

    --
     David Serrano
     (Please treat my english text just like Perl code, i.e. feel free to notify me of any syntax, grammar, style and/or spelling errors. Thank you!).

Re: Data::Dumper turns floating points numbers into strings
by mje (Deacon) on Jun 15, 2011 at 08:18 UTC

    I had a similar problem with JSON::XS and DBD::Oracle. JSON::XS looks at the scalar and basically does if SvPOKp stringify else if SvNOKp treat as float else if SvIOKp treat as integer.

    The problem is that DBD::Oracle does not allow you to bind a column as an integer. The object I wanted to JSONify was very large so all those extra " around integers were adding a lot to the size of the JSON encoded object. To get rid of the " I had to loop through my data and for every integer I had to add 0 to it - this took time. In the end we changed DBD::Oracle to add options to discard the pv.

    I documented what we found and did at Support binding of integers in DBD::Oracle so they are returned as IVs

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://909619]
Approved by sundialsvc4
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (14)
As of 2014-09-19 15:38 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (142 votes), past polls