I just built a version of the latest version of Perl, v5.35.6, from source and am having an odd issue with the output of the ASCII Escape character when it is printed.
When I run a script at the Windows 10 command prompt, Perl, instead of dumping the Escape character as a literal character into the console when instructed to do so, prints a left pointing arrow (probably the printable glyph representation of the Escape character).
Before I present a code example, I should add for completeness sake that I am in the default Windows code page of 437. (Verified via chcp, which reports: Active code page: 437 . Playing with the code page (e.g., changing it to 65001 or 1252) doesn't help, other than to make the left-pointing-arrow as shown below into a non-printable "box" character representation, probably because the glyph representation of Escape being printed by Perl as a left pointing arrow is invalid for those two code pages or whatever is coming out is missing from the characters in the Consolas font I am using for my Command Prompt).
Here is an example: (Clarification: a pair of double-quotes is used to cause double-quotes to get escaped for proper cmd.exe interpretation):
perl.exe -e "print ""this \x1B[1;31mred\x1B[0m text\n"" "
Output (Incorrect - Perl is changing the
chr(27) that precedes each left bracket
i.e., the Escape ASCII character, into a left arrow):
this ←[1;31mred←[0m text
What I am trying to do is to use an ANSI Escape Code above to change the color of the text to red for the word "red" (i.e.,
<Esc>[1;31m ==> Set foreground color to 31
Red and
<Esc>[0m ==> Reset all modes/styles). For some bizarre reason, Perl is causing a character U+2190, best I can tell, which is the Unicode representation of a left-arrow (a glyph that is sometimes used to represent the <Esc> character as a printable character) to be output instead of the proper Escape ASCII character I am expecting. There is nothing wrong with the code, mind you, because if I do the following:
perl.exe -e "print ""this \x1B[1;31mred\x1B[0m text\n"" " | cat
Output (correct):
this red text
So, it seems like Perl is treating a redirection of STDOUT differently and actually sending a proper escape character. I also tried the following (redirecting output to a file and outputting said file, instead of piping to cat) to see what happens:
perl.exe -e "print ""this \x1B[1;31mred\x1B[0m text\n"" " >tmp.txt & t
+ype tmp.txt
Output (correct):
this red text
Just to make absolutely certain that the issue is not with the code or some misbehavior of the windows command prompt (i.e., c:\windows\system32\cmd.exe), I ran the code through a different Perl interpreter, the one that comes with Cygwin, to see how it would behave in this case:
c:\cygwin64\bin\perl5.32.1.exe -e "print ""this \x1B[1;31mred\x1B[0m t
+ext\n"" "
Output (correct):
this red text
Perhaps to no surprise, the Cywin Perl interpreter did the right thing, even though its output was not in any way redirected.
Here is what we can rule out, right off the bat:
- It is not a Perl 5.35.x issue, because I have an old 5.34 build that behaves in the same way (built with the same build settings/defines).
- As demonstrated via type/cat/Cygwin Perl, it is not a Windows Command Prompt (i.e., cmd.exe) or active code page issue, either.
I am not sure what the true cause is, but I believe that the problem may somehow stem from my build config. So let me present it next by using
perl -V. I used Visual C++ 2022 as my compiler, for this build and used Visual C++ 2019 for the Perl 5.34 build that behaves in the same way with regard to the Escape character handling:
C:\>perl.exe -V
Summary of my perl5 (revision 5 version 35 subversion 6) configuration
+:
Platform:
osname=MSWin32
osvers=10.0.19043.1348
archname=MSWin32-x64-multi-thread-64int
uname=''
config_args='undef'
hint=recommended
useposix=true
d_sigaction=undef
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=undef
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
Compiler:
cc='cl'
ccflags ='-nologo -GF -W3 -MD -DWIN32 -D_CONSOLE -DNO_STRICT -DNDE
+BUG -DWIN64 -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -D
+_WINSOCK_DEPRECATED_NO_WARNINGS -DPERL_TEXTMODE_SCRIPTS -DMULTIPLICI
+TY -DPERL_IMPLICIT_SYS -DWIN32_NO_REGISTRY'
optimize='-O1 -Ob2 -Oi -Oy -GL -Gm- -GS- -Gy -sdl- -Zc:inline -Zi
+-fp:precise -arch:AVX512 -fp:precise'
cppflags='-DWIN32'
ccversion='19.31.30818'
gccversion=''
gccosandvers=''
intsize=4
longsize=4
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=undef
longlongsize=8
d_longdbl=define
longdblsize=8
longdblkind=0
ivtype='__int64'
ivsize=8
nvtype='double'
nvsize=8
Off_t='__int64'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='link'
ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpat
+h:"c:\perl\5.35.6\lib\CORE" -machine:AMD64 -subsystem:console,"6.00"
+'
libpth="c:\Program Files\Microsoft Visual Studio\2022\Preview\VC\T
+ools\MSVC\14.31.30818\\lib\x64"
libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
+comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi3
+2.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib o
+dbccp32.lib comctl32.lib msvcrt.lib vcruntime.lib ucrt.lib
perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.l
+ib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib net
+api32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.l
+ib odbccp32.lib comctl32.lib msvcrt.lib vcruntime.lib ucrt.lib
libc=ucrt.lib
so=dll
useshrplib=true
libperl=perl535.lib
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_win32.xs
dlext=dll
d_dlsymun=undef
ccdlflags=' '
cccdlflags=' '
lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -ltcg -
+libpath:"c:\perl\5.35.6\lib\CORE" -machine:AMD64 -subsystem:console,
+"6.00"'
Characteristics of this binary (from libperl):
Compile-time options:
HAS_TIMES
HAVE_INTERP_INTERN
MULTIPLICITY
PERLIO_LAYERS
PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_SYS
PERL_MALLOC_WRAP
PERL_OP_PARENT
PERL_PRESERVE_IVUV
USE_64_BIT_INT
USE_ITHREADS
USE_LARGE_FILES
USE_LOCALE
USE_LOCALE_COLLATE
USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC
USE_LOCALE_TIME
USE_NO_REGISTRY
USE_PERLIO
USE_PERL_ATOF
USE_THREAD_SAFE_LOCALE
Built under MSWin32
Compiled at Nov 27 2021 18:54:37
@INC:
c:/perl/5.35.6/lib
Just to compare and contrast, here is what the Perl from Cygwin has as its configuration:
c:\>perl5.32.1.exe -V
Summary of my perl5 (revision 5 version 32 subversion 1) configuration
+:
Platform:
osname=cygwin
osvers=3.2.0(0.34053)
archname=x86_64-cygwin-threads-multi
uname='cygwin_nt-10.0 cygwinpro 3.2.0(0.34053) 2021-03-29 08:42 x8
+6_64 cygwin '
config_args='-des -Dprefix=/usr -Dmksymlinks -Darchname=x86_64-cyg
+win-threads -Dlibperl=cygperl5_32.dll -Dcc=gcc -Dld=g++ -Accflags=-gg
+db -O2 -pipe -Wall -Werror=format-security -D_FORTIFY_SOURCE=2 -fstac
+k-protector-strong --param=ssp-buffer-size=4 -fdebug-prefix-map=/mnt/
+share/cygpkgs/perl/perl.x86_64/build=/usr/src/debug/perl-5.32.1-2 -fd
+ebug-prefix-map=/mnt/share/cygpkgs/perl/perl.x86_64/src/perl-5.32.1=/
+usr/src/debug/perl-5.32.1-2 -fwrapv'
hint=recommended
useposix=true
d_sigaction=define
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler:
cc='gcc'
ccflags ='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -D_GNU_SOURCE -
+ggdb -O2 -pipe -Wall -Werror=format-security -D_FORTIFY_SOURCE=2 -fst
+ack-protector-strong --param=ssp-buffer-size=4 -fdebug-prefix-map=/mn
+t/share/cygpkgs/perl/perl.x86_64/build=/usr/src/debug/perl-5.32.1-2 -
+fdebug-prefix-map=/mnt/share/cygpkgs/perl/perl.x86_64/src/perl-5.32.1
+=/usr/src/debug/perl-5.32.1-2 -fwrapv -fno-strict-aliasing'
optimize='-O3'
cppflags='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -D_GNU_SOURCE -
+ggdb -O2 -pipe -Wall -Werror=format-security -D_FORTIFY_SOURCE=2 -fst
+ack-protector-strong --param=ssp-buffer-size=4 -fdebug-prefix-map=/mn
+t/share/cygpkgs/perl/perl.x86_64/build=/usr/src/debug/perl-5.32.1-2 -
+fdebug-prefix-map=/mnt/share/cygpkgs/perl/perl.x86_64/src/perl-5.32.1
+=/usr/src/debug/perl-5.32.1-2 -fwrapv -fno-strict-aliasing'
ccversion=''
gccversion='10.2.0'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='g++'
ldflags =' -Wl,--enable-auto-import -Wl,--export-all-symbols -Wl,-
+-enable-auto-image-base -fstack-protector-strong'
libpth=/usr/lib
libs=-lpthread -lnsl -lgdbm -ldb -ldl -lcrypt -lgdbm_compat
perllibs=-lpthread -lnsl -ldl -lcrypt
libc=/usr/lib/libcygwin.a
so=dll
useshrplib=true
libperl=cygperl5_32.dll
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=dll
d_dlsymun=undef
ccdlflags=' '
cccdlflags=' '
lddlflags=' --shared -Wl,--enable-auto-import -Wl,--export-all-sy
+mbols -Wl,--enable-auto-image-base -fstack-protector-strong'
Characteristics of this binary (from libperl):
Compile-time options:
HAS_TIMES
MULTIPLICITY
PERLIO_LAYERS
PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_CONTEXT
PERL_OP_PARENT
PERL_PRESERVE_IVUV
PERL_USE_SAFE_PUTENV
USE_64_BIT_ALL
USE_64_BIT_INT
USE_ITHREADS
USE_LARGE_FILES
USE_LOCALE
USE_LOCALE_COLLATE
USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC
USE_LOCALE_TIME
USE_PERLIO
USE_PERL_ATOF
USE_REENTRANT_API
USE_THREAD_SAFE_LOCALE
Locally applied patches:
Cygwin: README
Cygwin: use auto-image-base instead of fixed DLL base address
Cygwin: modify hints
Cygwin: Configure correct libsearch
Cygwin: Configure correct libpth
Cygwin: Encode fix for CVE-2021-36770
Cygwin: Win32 correct UTF8 handling
Built under cygwin
Compiled at Aug 14 2021 09:07:12
%ENV:
CYGWIN="glob:ignorecase winsymlinks:native pipe_byte"
@INC:
/usr/local/lib/perl5/site_perl/5.32/x86_64-cygwin-threads
/usr/local/share/perl5/site_perl/5.32
/usr/lib/perl5/vendor_perl/5.32/x86_64-cygwin-threads
/usr/share/perl5/vendor_perl/5.32
/usr/lib/perl5/5.32/x86_64-cygwin-threads
/usr/share/perl5/5.32
