Re: Weird Character in File Makes Perl Think it's EOF


Your skill will accomplish what the force of many cannot
	PerlMonks

Re: Weird Character in File Makes Perl Think it's EOF

by periapt (Hermit)

on Oct 17, 2008 at 20:32 UTC ( #717852=note: print w/ replies, xml )

Need Help??

in reply to Weird Character in File Makes Perl Think it's EOF

You could certainly read the file in using binmode but, as wol noted, you do loose end-of-line handling. Depending on what is happening with your file before the parsing stage, you may want to try preprocessing it before the parse step.

Assuming that your file should only have word characters in it (as defined by \w = [a-zA-Z0-9_]), you could try this one-liner

perl -i.orig -p -e "s/\W+/?/g;" <yourfile>
[download]

This will rename the original file <yourfile>.orig and change every occurance of a non-word character to a question mark. I am assuming here that you want to retain the relative location of the offending byte. If you don't, simply write s/\W+// instead of s/\W+/?/.

If you wanted to write the output to STDOUT say before passing the data to another process you can omit the -i.orig flag

Of course, you could do it with sed or gawk but this is PerlMonks ;o).

PJ
use strict; use warnings; use diagnostics;

Comment on Re: Weird Character in File Makes Perl Think it's EOF
Download Code

Re^2: Weird Character in File Makes Perl Think it's EOF
by Jim (Chaplain) on Oct 19, 2008 at 20:40 UTC

tr -d "\032" < infile > outfile

...or...

tr "\032" " " < infile > outfile

If you use Gawk, you have to set its BINMODE.

Using ActivePerl for Windows, I've never had to use binmode to handle nasty ASCII control characters like NUL (0x00) and SUB (0x1A). It seems to read and write them in text mode just fine.

D:\>perl -e "print qq{\x00\x1A\nfoo\nbar\x1A\x00\n}"
 
foo
bar

D:\>perl -e "print qq{\x00\x1A\nfoo\nbar\x1A\x00\n}" |
perl -ne "print if m/foo/"
foo

D:\>
[download]

Jim

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom

Node Status^?

node history
Node Type: note [id://717852]
help

Chatterbox^?

How do I use this? | Other CB clients

Other Users^?

Others examining the Monastery: (4)

As of 2013-05-23 10:44 GMT

Sections^?

Information^?

Find Nodes^?

Leftovers^?

Voting Booth^?

The best material for plates (tableware) is:

Results (479 votes), past polls