Unix vs Windows Text File Format

[merrymonk]

Hopefully this has to be simple!
I have a normal text file which has been created in Windows. What do I have to do to this file so that it is ‘compatible’ to UNIX?

[si_lence]

run dos2unix on it.

This will change the line-endings. Maybe your text editor also provides this functionality (e.g. UltraEdit does)

Cheers

si_lence

[BrowserUk]

Purely out of interest, does this do the job?

perl -l0 -0015 -pe1 infile >outfile

[sgt]

You probably meant:

% perl -0015 -pe chomp infile > outfile

# or even better
% perl -0015 -pe 's{^\cZ$}{}; chomp' infile > outfile


# deparse shows that -l0 cannot work

% stephan@ape (/c/cygvar/tmp) %
% perl -MO=Deparse -l0 -0015 -pe1               
BEGIN { $/ = "\r"; $\ = "\000"; }
LINE: while (defined($_ = <ARGV>)) {
    chomp $_;
    '???';
}
continue {
    print $_;
}
-e syntax OK

# checking

% stephan@ape (/c/cygvar/tmp) %
% echo abc def | xargs -n1 | perl -lpe '$_.="\cM"' | cat -evnt
     1  abc^M$
     2  def^M$

% stephan@ape (/c/cygvar/tmp) %
% echo abc def | xargs -n1 | perl -lpe '$_.="\cM"' | perl -0015 -pe ch
+omp | cat -evnt
     1  abc$
     2  def$

cheers --steph

[BrowserUk]

You probably meant

No. I meant what I posted, but as I indicated, I wasn't sure if it would work. Kinda hard to test without a working *nix environment and my VirtualBox/Ubunto is currently screwed. I hadn't thought of using Deparse.

That said: this should work:

perl -0777 -l -0015 -pe1 in >out

[Utilitarian]

in the shell run

perl -pi '.bak' -e 's/^M//' <FILE_NAME>

Note: ^M above is actually a literal [CTRL]-M, this can be entered in the shell as [CTRL]-V [CTRL]-M
([CTRL]-V - "take the next character as a literal")

Edited as dave_the_m pointed out, I had the run options in the wrong order, I can only put this down to a lack of caffeine

[dave_the_m]

perl -epi ...

Er, no. That executes the code 'pi' with the rest of the line as @ARGV, which is effectively a noop. Also there mustn't be a space between -i and its arg. Lastly, using \r is to be much preferred than a literal carriage-return control character. You want this instead:

perl -pi'.bak' -e 's/\r//' <FILE_NAME>

Dave.

[graff]

What do I have to do to this file so that it is ‘compatible’ to UNIX?

As others have explained, the basic issue is to remove the carriage-return characters (a.k.a. "\r", "\x0d", "\015" and the keycode emitted by 'ctrl-M') which Windows puts in front of every line-feed character.

Of course, a Windows plain-text file with the carriage-returns still present is also "compatible" with UNIX, for a fairly wide range of activities where the presence of carriage-returns makes no difference to the user or the application.

In the context of applications written in Perl, whenever I write Perl for a task where the range of inputs might include Windows text files as well as linux/unix text data, and where the nature of the line termination might be an issue, I just use the following instead of chomp:

s/[\r\n]+$//;

That way, the script will behave consistently on both Windows and linux/unix systems, regardless whether the data is coming from the same system or a different system.