Re^5: Peculiar Reference To U+00FE In Text::CSV_XS Documentation

[Tux]

I do not recall exactly why the docs were written the way they were, but as I am unaware of the DAT format, I cannot verify that U+00FE was referred to because of this format.

Knowing my own way of thinking, it most likely is not 0xFF, as that would be -1, which could be used as a guard marker or something alike (currently it isn't). I might have used 0xFE as it is the next highest byte.

I have just read your post, and the only conflict I see that Text::CSV_XS is not able to do is optional line endings. The optional <CR> before the <NL> is automatically dealt with (just do not specify eol), but you cannot have an extra U+00AE to also end records. If otoh 0xAE is just a placeholder for embedded newlines, that is easy to do (see below).

Another point of care is that Text::CSV_XS does not deal with BOM's, so you'll need File::BOM or other means to deal with that.

my $csv = Text::CSV_XS->new ({
    sep_char    => "\x{14}",
    quote_char  => "\x{fe}",
    escape_char => undef,
    binary      => 1,
    auto_diag   => 1,
    });
while (my $row = $csv->getline ($fh)) {
    tr/\x{ae}/\n/ for @$row;
    # continue as usual
    }

If it doesn't, I'd like to see some data.

Note that the encoded U+00FE is 0xC3BE, which is two bytes, and two bytes cannot be used as a sep_char in Text::CSV_XS, which parses the data as bytes, so the stream has to be properly coded before parsing.