Weird number formatting

[licking9Volts]

Hi all,

I've got a file with weird numbers that need to be reformatted. Here's a line from the file:

1 1 4 1 0.225120000000000D+06 0.341913000000000D+07 1 1 1 661 660 660 
+659

I need to take the large numbers, add 30 to one of them, and then print them back out in a "normal" format. Here's what I have so far:

#!/usr/sbin/perl -w

use strict;
...
if (/^\s+1/) {
     @info = split(' ', $_, 9999);

     $my_x = $info[4] + 0;
     $my_y = $info[5] + 30;

     for($i = '1'; $i <= $info[2]; $i++) {
          print ("$my_x $my_y $info[($i + 8)]\n");
          $my_y = $my_y + 30;    
     }
}
...

So far it works...kind of. Here's the output:

Argument "0.225120000000000D+06" isn't numeric in addition (+) at H:\d
+em2xyz.pl line 33, <FILE> line 2.
Argument "0.341913000000000D+07" isn't numeric in addition (+) at H:\d
+em2xyz.pl line 34, <FILE> line 2.
225120 3419160 661

225120 3419190 660

225120 3419220 660

225120 3419250 659

Argument "0.225150000000000D+06" isn't numeric in addition (+) at H:\d
+em2xyz.pl line 33, <FILE> line 3.
Argument "0.341913000000000D+07" isn't numeric in addition (+) at H:\d
+em2xyz.pl line 34, <FILE> line 3.
225150 3419160 661

225150 3419190 659

225150 3419220 660

It seems to always give me a warning the first time it tries to print the large numbers. After that, it doesn't seem to mind. Is there a way to convert the numbers in weird notation to their non-decimal counterparts without an error?

Thanks!
^l9v

[zigdon]

Maybe I'm missing something, but shouldn't it be "0.225150000000000E+06"? ("E" instead of "D")? Or is that some locale settings?

Also, just a side note, do you really want 9999 entries in @info, or do you just want to split however many there are? If so, just omit the 3rd paramater to split.

-- zigdon

[tye]

The 'D' means that the values are doubles instead of floats (in FORTRAN).

Updates:

This sounds like a bug that should be reported to p5p. This is such an old and standard notation that it will probably be common for C's strtod() to support it so Perl should be made aware of it in order to avoid the misleading warning.

split didn't used to support -1 as the third argument so old code had to make up some "arbitrarilly large number" to get such functionality. In this case, I would think just dropping the third argument makes the most sense (since trailing spaces should probably not create a final, empty "number" to be processed). But in other cases you should change the "9999" to "-1" so that your code won't break when you actually get 10000 items. I complained about this in Perl 4 and suggested the current behavior but Larry didn't seem impressed so I was glad when this feature showed up. (:

                - tye

[licking9Volts]

I thought the same thing about the E instead of D, but the files are USGS Digital Elevation Models taken from the Texas National Resources Information website. DEM is some kind of standard format for these models. Thanks for the tip on splitting also. I've fixed it.

[VSarkiss]

See, you need one of us old-timers for this kind of stuff. ;-)

In all likelihood, a Fortran program is producing that data. The variable in the program is declared DOUBLE PRECISION, and probably a D or G format is being used to print it. Just substitute E for D and you should be fine.

[rje]

Surely, surely, those 'D's have to mean 'E's... well, of course they don't have to, but if you have to add 30 to one of those numbers, then you have to know what the original number is, right?

So that's a piece of data you need someone to inform you of. If it is an exponent indicator, then can you just do a pattern match before interpreting the number (s/D/E/)?

Rob

[fruiture]

The 'D' causes the problem, perl does not understand these numbers as numbers. Does that 'D' have any other meaning than the 'E',which means '*10^' and _is_ understood by perl?

I don't understand that notation either, so i cannot tell you exactly what you can do, anyways it must include removing that letter from the string.

--
http://fruiture.de

[hv]

If I correctly understand what is happening here, the letter 'D' is a locale-specific exponentiation letter: what locale are you using? Since the output shows that perl is parsing the number correctly, I'd guess that the "Argument isn't numeric" warning is a bug in perl - that it is checking only for "E" as the exponentiation letter - but that the operating system library is then correctly converting the number.

If that is correct, the best workaround I can think of is to locally disable the warning, and check the correctness yourself:

  sub weird2number {
    my $text = shift;
    unless ($text =~ /^\d*\.\d+D[+\-]\d+$/) {
      warn qq{Argument "$text" is not weird};
    }
    no warnings "numeric";
    return $text + 0;
  }

If you also need to output numbers in this format, you may also have problems: the closest is the (s)printf '%E' format, which puts the first significant digit before the decimal point. The simplest way I can think of to get around that is to use sprintf '%E', $number and then modify the result, perhaps like so:

  sub number2weird {
    my $number = shift;
    my $text = sprintf '%.14E', $number;
    $text =~ s{
      (\d) \. (\d+ [DE]) ([+\-] \d+)
    }{ "0.$1$2" . sprintf "%+03d", $3 + 1 }xe
      or warn qq{Failed to make "$text" weird};
    return $text;
  }

Note however that the output format of sprintf '%E' is also dependent on your operating system libraries, so you may need to modify the pattern to match correctly.

Hugo

[hardburn]

This may or may not help, but try putting a 'use integer;' at the top of your program. That should force Perl to do handle numbers as integers instead of floats.

----
Reinvent a rounder wheel.

Note: All code is untested, unless otherwise stated

[licking9Volts]

Thanks hardburn, I tried it, but it ignored the decimals. Maybe because of the notation?

[licking9Volts]

Thanks everyone for all your help! VSarkiss and tye mentioned it might be FORTRAN and it seemed like I'd read that somewhere already. I then went back and started to re-read the Standards for Digital Elevation Models, and in one of the appendices it list the column types as REAL*8 (FORTRAN Notation). So once again, the virtues of "RTFM" are made clearer to me. In the end, I simply substituted D for E and went on.

$inrecord =~ s/D/E/g;
@info = split(' ', $inrecord);

$my_x = $info[4] + 0;
$my_y = $info[5] + 0;

I happened to notice that it still wouldn't consider it a "number" until I performed some kind of math function on it, so I just added 0 and everything worked fine. So once again, thanks everyone!

^l9v