Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw

Re: Regexp and Linux (is it utf issue?)

by aitap (Deacon)
on Jun 28, 2013 at 06:21 UTC ( #1041153=note: print w/replies, xml ) Need Help??

in reply to Regexp and Linux (is it utf issue?)

When you read text files, you should decode them. This is easy using PerlIO layers, Encode module and three-argument form of open:

use Encode; open my $fh, "<:encoding(whatever)", $filename or die $!;
This way, Perl decodes everything automatically, and you only have to work with characters, not bytes.

When you write text to files, writing characters produces the famous warning: "wide character in (sub name)...". You need to encode them using the same technique: open my $write, ">:encoding(whatever)", $filename or die $!;. You can use :utf8 layer to encode characters because they are internally stored as valid UTF-8.

Do not use :utf8 iolayer to decode text because it simply sets "character" flag on the strings read from filehandles without any checks and this is generally unsafe: UTF8 related proof of concept exploit released at T-DOSE.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1041153]
[1nickt]: Tux is cbstream working for you?
[Tux]: I have no idea! Read this article
[choroba]: yes, if the formula contains just +, -, *, /, and digits, calculate it, otherwise fail, or something like that.
[choroba]: or detection of malicious strings like | cmd or what was the security issue about.
[choroba]: re slow PM, yesterday half of my clicks in RATS ended in request timeout.
[Tux]: 1nickt - yes, it works oké
[Tux]: choroba as many formula's have references to other cells, that would only be possible on constants. CSV_XS has no knowledge of the contents of the rest of the document
[Tux]: But it could be an option for Spreadsheet::Read (but I am not tempted to do that. patches welcome?)
[choroba]: makes sense
[1nickt]: Tux Wow, interesting! He seems to say that there is no real protection in Excel other than common sense. Nice story about the Dutch police using the exploit to bring down criminal networks on the Dark Web!

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (5)
As of 2017-10-18 10:56 GMT
Find Nodes?
    Voting Booth?
    My fridge is mostly full of:

    Results (244 votes). Check out past polls.