Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re: Invalid UTF8 data: namespace suggestion needed or wheel reinvented?

by PodMaster (Abbot)
on Mar 02, 2004 at 11:00 UTC ( #333213=note: print w/replies, xml ) Need Help??


in reply to Invalid UTF8 data: namespace suggestion needed or wheel reinvented?

How is the utf-8 usually being corrupted?
What exactly are you doing (are you just removing illegal characters? what else)?
Rules (is it configurable, or will it be, how)?

I don't know about a module name (Convert::FixMalformedUTF8? Encode::FixMalformedUTF8?) cause it sounds like a script to me (cpan likes those), cause you can do all that with the Encode/Unicode and modules like that (that's what you probably did).

MJD says "you can't just make shit up and expect the computer to know what you mean, retardo!"
I run a Win32 PPM repository for perl 5.6.x and 5.8.x -- I take requests (README).
** The third rule of perl club is a statement of fact: pod is sexy.

  • Comment on Re: Invalid UTF8 data: namespace suggestion needed or wheel reinvented?

Replies are listed 'Best First'.
Re: Re: Invalid UTF8 data: namespace suggestion needed or wheel reinvented?
by liz (Monsignor) on Mar 02, 2004 at 11:23 UTC
    How is the utf-8 usually being corrupted?

    A good question. It's a system that has it roots in Perl 4 and which consists of hundreds of pages (CGI scripts) that are being used by many different people 24x7. And which is currently being migrated to a more manageable situation, so there is little point in trying to find out which users are doing what on which page causing the problem.

    What exactly are you doing (are you just removing illegal characters? what else)?

    Indeed. They're mostly free text entries of different sources in various languages.

    Rules (is it configurable, or will it be, how)?

    The only configuration that I see, would be the ability to list the tables to correct (defaulting to all). And how fields fixed are reported (either STDERR or calling a subroutine).

    ...it sounds like a script to me

    Well, it is is a script now. But I dislike scripts as such, because you are never flexible enough. I'd rather have a simple script that calls a module, than that people start copying a script and start adapting it to their needs.

    Liz

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://333213]
help
Chatterbox?
[Lady_Aleena]: This evening, I thought to myself, "I would love to search Netflix for 100 randomfilms from my database to see if they have them." So I went to CPAN to see if there were a modiule which could do such a thing. There is, but GUH! ...
[Lady_Aleena]: ... Does it have to be that hard and complex to use to do a search on Netflix?
Lady_Aleena grumbles.
[Lady_Aleena]: I really need to get back into the swing of Perl. I'm forgetting things from lack of use. I don't know how behind I am too.

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (6)
As of 2017-01-21 02:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you watch meteor showers?




    Results (180 votes). Check out past polls.