Re: Issue manipulating data - problem with data file format

[samizdat]

Without seeing samples, it's a little difficult to do much for you. I'd suggest that you open them up in a binary editor which shows binary on one side and ASCII on the other. BVI and BED are Linux examples, but there are freeware and shareware ones available for That Other Nameless OS(TM) as well. What is probably happening is that some of your files are encoded in 16-bit UniCode characters (or maybe Microsoft Rich Text Format, where formatting codes are injected into the text). Unless you specifically include the appropriate modules into your program and enable the switches in your ReGex matchers, you'll have this problem.

So, first, determine what your encoding is, and then you'll be able to learn the next step or ask a more detailed question.

[maccas17]

I really appreciate the responses from all three of you, especially given I didn't provide much info. It sounds plausible to me what you are suggesting and I will try and test it out when I have chance. I have managed to bodge my way round it a bit by another very crude (I have much to learn!) script that reads all the files (using type command) and then outputs to a .new version. When I run the .new version with my original script it works fine.

I do hope to get to the bottom of this in a more intelligent fashion, so I will try out the suggestions.

OS: Win XP
Perl v5.8.8

[samizdat]

We all learn from being better able to explain. Glad to help. :D