Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re: Encoding: my custom encoding fails on one character but works for everything else?!

by ELISHEVA (Prior)
on Sep 13, 2009 at 23:56 EDT ( #795042=note: print w/replies, xml ) Need Help??


in reply to Encoding: my custom encoding fails on one character but works for everything else?!

When in doubt , decompose.

Have you tried dumping the data structure storing the result of parsing the umc file? Does it look right? If you are storing conversion rules in a hash, what happens if you look up '{i}' (or 'i') independently in the hash? Do you get the right unicode?

What 'key' do you need to lookup the unicode for '{i}'? If the data structures are storing the right information, what about the code that is supposed to be converting '{i}' into a form suitable for look-up in your encoding data structures. How is '{i}' being extracted? What happens if you separate out the code that extracts '{i}' and run it separately? Do you get a key suitable for look-up? Or do you get '{i' or 'i}' or even '{i}' when in fact you need '{i}' (or vice versa)?

I'd also investigate further why it was necessary to comment out the mapping for the left curly brace. Are right curlies also a problem? If not, what is special about left curlies? Is this a hint about how the encoding/decoding algorithm works? Perhaps there is something deeper that connects the problems with left curlies and '{i}'?

Best, beth

  • Comment on Re: Encoding: my custom encoding fails on one character but works for everything else?!

Replies are listed 'Best First'.
Re^2: Encoding: my custom encoding fails on one character but works for everything else?!
by herveus (Parson) on Sep 14, 2009 at 12:03 UTC
    Howdy!

    Daud_t.c contains a whole mess of declarations of constants. Each constant name includes the to and from encodings and the subsequence of bytes matched. I was able to see the sequence for {i} in there, and it appeared to be correct. The conversion process seems to be to chase pointers through a tangled mess ultimately leading to an output character. Or something along those lines. I'm doing a poor job of describing it, but I did not see anything out of line there.

    I empirically determined that I needed to comment out LEFT CURLY BRACKET. My surmise is that if it was legal in its own right, it would get consumed early spoiling the matches...waitaminnit...(thinking as I type) I'll try moving that to the end to see if that allows it to catch otherwise unmatched {.

    In the Daud encoding, curlies are the brackets around non-ASCII characters. They are, thus, special in this context. Hmmm...

    Thanks for the thoughts.

    yours,
    Michael

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://795042]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (3)
As of 2021-11-27 06:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?