Re^6: Can't call libcurl with Inline::C on Windows

[syphilis]

I'll try to build latest Net-Curl with perl-5.30 later today, and see how it goes for me.

Here's how it went for me.

First up, I added the location (C:\_64\msys64\mingw64\bin) of the libcurl dll to my path:

set PATH=%PATH%;C:\_64\msys64\mingw64\bin

That, of course, also makes a whole lot of other stuff (such as pkg-config) available.

Then I conceal the curl static lib (C:\_64\msys64\mingw64\lib\libcurl.a) by changing its extension to ".a_hide".
This ensures that the Net-Curl-0.41 build links to the import lib (C:\_64\msys64\mingw64\lib\libcurl.dll.a).
Next, run the Makefile.PL:

C:\sisyphusion\Net-Curl-0.41>perl Makefile.PL
'which' is not recognized as an internal or external command,
operable program or batch file.
which failed:

Found libcurl version 7.65.0
Reading C:/_64/msys64/mingw64/include/curl/curl.h (gcc -E -IC:/_64/msy
+s64/mingw6
4/include C:/_64/msys64/mingw64/include/curl/curl.h)
Skipping '#define CURL_STRICTER': does not define a symbol at Makefile
+.PL line 3
18, <H> line 34.
Skipping '#  define CURL_EXTERN': does not define a symbol at Makefile
+.PL line 3
18, <H> line 116.
Skipping '#  define CURL_EXTERN': does not define a symbol at Makefile
+.PL line 3
18, <H> line 128.
Skipping '#define CURL_DID_MEMORY_FUNC_TYPEDEFS': does not define a sy
+mbol at Ma
kefile.PL line 318, <H> line 448.
Reading C:/_64/msys64/mingw64/include/curl/multi.h (gcc -E -IC:/_64/ms
+ys64/mingw
64/include C:/_64/msys64/mingw64/include/curl/multi.h)
-> found 892 constants (should be 892)
Writing const-defenums-h.inc
Writing const-curl-xs.inc
Writing const-easy-xs.inc
Writing const-form-xs.inc
Writing const-multi-xs.inc
Writing const-share-xs.inc
Writing curl-Easy-c.inc
Writing curl-Easy-xs.inc
Writing curl-Form-c.inc
Writing curl-Form-xs.inc
Writing curl-Multi-c.inc
Writing curl-Multi-xs.inc
Writing curl-Share-c.inc
Writing curl-Share-xs.inc
Writing lib/Net/Curl/examples.pod
<- examples/01-curl-transport.pl
<- examples/02-multi-simple.pl
<- examples/03-multi-event.pl
<- examples/04-share-threads.pl
<- examples/05-irssi-downloader.pl
Writing lib/Net/Curl/Compat.pm
Checking if your kit is complete...
Looks good
Generating a gmake-style Makefile
Writing Makefile for Net::Curl
Writing MYMETA.yml and MYMETA.json

Followed by:

C:\sisyphusion\Net-Curl-0.41>make
cp lib/Net/Curl/Form.pm blib\lib\Net\Curl\Form.pm
cp lib/Net/Curl/Compat.pm blib\lib\Net\Curl\Compat.pm
cp lib/Net/Curl/Share.pm blib\lib\Net\Curl\Share.pm
cp lib/Net/Curl/examples.pod blib\lib\Net\Curl\examples.pod
cp lib/Net/Curl/Multi.pm blib\lib\Net\Curl\Multi.pm
cp lib/Net/Curl.pm blib\lib\Net\Curl.pm
cp lib/Net/Curl/Easy.pm blib\lib\Net\Curl\Easy.pm
Running Mkbootstrap for Curl ()
"C:\_64\perl530_810\bin\perl.exe" -MExtUtils::Command -e chmod -- 644 
+"Curl.bs"
"C:\_64\perl530_810\bin\perl.exe" -MExtUtils::Command::MM -e cp_nonemp
+ty -- Curl.bs blib\arch\auto\Net\Curl\Curl.bs 644
"C:\_64\perl530_810\bin\perl.exe" "C:\_64\perl530_810\lib\ExtUtils/xsu
+bpp"  -typemap C:\_64\perl530_810\lib\ExtUtils\typemap -typemap C:\si
+syphusion\Net-Curl-0.41\typemap  Curl.xs > Curl.xsc
"C:\_64\perl530_810\bin\perl.exe" -MExtUtils::Command -e mv -- Curl.xs
+c Curl.c
gcc -c   -Wall -Wno-unknown-pragmas -IC:/_64/msys64/mingw64/include -s
+ -O2   -DVERSION=\"0.41\" -DXS_VERSION=\"0.41\"  "-IC:\_64\perl530_81
+0\lib\CORE"   Curl.c
In file included from Curl.xs:17:
curl-Share-c.inc: In function 'cb_share_lock':
C:\_64\perl530_810\lib\CORE/perl.h:174:22: warning: unused variable 'm
+y_perl' [-Wunused-variable]
 #  define pTHX  tTHX my_perl PERL_UNUSED_DECL
                      ^~~~~~~
C:\_64\perl530_810\lib\CORE/perl.h:185:19: note: in expansion of macro
+ 'pTHX'
 #    define dTHX  pTHX = PERL_GET_THX
                   ^~~~
curl-Share-c.inc:30:2: note: in expansion of macro 'dTHX'
  dTHX;
  ^~~~
curl-Share-c.inc: In function 'cb_share_unlock':
C:\_64\perl530_810\lib\CORE/perl.h:174:22: warning: unused variable 'm
+y_perl' [-Wunused-variable]
 #  define pTHX  tTHX my_perl PERL_UNUSED_DECL
                      ^~~~~~~
C:\_64\perl530_810\lib\CORE/perl.h:185:19: note: in expansion of macro
+ 'pTHX'
 #    define dTHX  pTHX = PERL_GET_THX
                   ^~~~
curl-Share-c.inc:40:2: note: in expansion of macro 'dTHX'
  dTHX;
  ^~~~
Curl.c: In function 'XS_Net__Curl__Multi_socket_action':
Curl.c:1373:15: warning: overflow in conversion from 'long long unsign
+ed int' to 'int' changes value from '18446744073709551615' to '-1' [-
+Woverflow]
      sockfd = CURL_SOCKET_BAD;
               ^~~~~~~~~~~~~~~
"C:\_64\perl530_810\bin\perl.exe" -MExtUtils::Mksymlists \
     -e "Mksymlists('NAME'=>\"Net::Curl\", 'DLBASE' => 'Curl', 'DL_FUN
+CS' => {}, 'FUNCLIST' => [], 'IMPORTS' => {  }, 'DL_VARS' => []);"
g++ Curl.def -o blib\arch\auto\Net\Curl\Curl.dll -mdll -s -L"C:\_64\pe
+rl530_810\lib\CORE" -L"C:\_64\gcc-mingw-810\mingw64\lib" Curl.o   "C:
+\_64\perl530_810\lib\CORE\libperl530.a" "C:\_64\msys64\mingw64\lib\li
+bcurl.dll.a" "C:\_64\gcc-mingw-810\mingw64\x86_64-w64-mingw32\lib\lib
+moldname.a" "C:\_64\gcc-mingw-810\mingw64\x86_64-w64-mingw32\lib\libk
+ernel32.a" "C:\_64\gcc-mingw-810\mingw64\x86_64-w64-mingw32\lib\libus
+er32.a" "C:\_64\gcc-mingw-810\mingw64\x86_64-w64-mingw32\lib\libgdi32
+.a" "C:\_64\gcc-mingw-810\mingw64\x86_64-w64-mingw32\lib\libwinspool.
+a" "C:\_64\gcc-mingw-810\mingw64\x86_64-w64-mingw32\lib\libcomdlg32.a
+" "C:\_64\gcc-mingw-810\mingw64\x86_64-w64-mingw32\lib\libadvapi32.a"
+ "C:\_64\gcc-mingw-810\mingw64\x86_64-w64-mingw32\lib\libshell32.a" "
+C:\_64\gcc-mingw-810\mingw64\x86_64-w64-mingw32\lib\libole32.a" "C:\_
+64\gcc-mingw-810\mingw64\x86_64-w64-mingw32\lib\liboleaut32.a" "C:\_6
+4\gcc-mingw-810\mingw64\x86_64-w64-mingw32\lib\libnetapi32.a" "C:\_64
+\gcc-mingw-810\mingw64\x86_64-w64-mingw32\lib\libuuid.a" "C:\_64\gcc-
+mingw-810\mingw64\x86_64-w64-mingw32\lib\libws2_32.a" "C:\_64\gcc-min
+gw-810\mingw64\x86_64-w64-mingw32\lib\libmpr.a" "C:\_64\gcc-mingw-810
+\mingw64\x86_64-w64-mingw32\lib\libwinmm.a" "C:\_64\gcc-mingw-810\min
+gw64\x86_64-w64-mingw32\lib\libversion.a" "C:\_64\gcc-mingw-810\mingw
+64\x86_64-w64-mingw32\lib\libodbc32.a" "C:\_64\gcc-mingw-810\mingw64\
+x86_64-w64-mingw32\lib\libodbccp32.a" "C:\_64\gcc-mingw-810\mingw64\x
+86_64-w64-mingw32\lib\libcomctl32.a" -Wl,--enable-auto-image-base "C:
+\_64\perl530_810\bin\perl.exe" -MExtUtils::Command -e chmod -- 755 bl
+ib\arch\auto\Net\Curl\Curl.dll

The build has succeeded (for some definition of "succeeded") but there are some warnings, and the module is unloadable:

C:\sisyphusion\Net-Curl-0.41>perl -Mblib -MNet::Curl -le "print Net::C
+url::VERSION;"
Curl.c: loadable library and perl binaries are mismatched (got handsha
+ke key 0000000010600080, needed 0000000010a00080)

I don't know why that happens, or how to fix it.
The compiler warning about conversion from 'long long unsigned int' to 'int' rings an alarm bell.
And I see places in the generated Curl.c where 'IV' is cast to 'int', which creates a cacophony of alarm bells.

Cheers,
Rob

[xiaoyafeng]

Thanks Rob, I note you use libcurl installed from MSYS2 other than win64 version, that's why you can build successfully I think. Maybe I choose a hard way since I use pure portable strawberry perl suite. ;)

[syphilis]

I note you use libcurl installed from MSYS2 other than win64 version, that's why you can build successfully I think

Yes, that probably makes it a bit simpler - certainly a lot simpler than the builds I did a couple of years ago against static libraries that I had built from source.

I finally solved the problem regarding the "loadable library and perl binaries" mismatch - and in so doing, I discovered that I had faced (and solved) the exact same problem when I built Net-Curl-0.37.
There's a line in the Makefile.PL:

CCFLAGS    => $devel_cflags . ' ' . $curl{cflags} . $bits,
that needs to be modified to:
CCFLAGS    => $Config::Config{ccflags} . ' ' . $devel_cflags . ' ' . $
+curl{cflags} . $bits,

That change then also brings about the need to patch Curl.xs:

--- Curl.xs_orig        2019-11-20 11:44:50 +1100
+++ Curl.xs     2019-11-20 22:55:23 +1100
@@ -17,6 +17,11 @@
 #include "perl.h"
 #include "XSUB.h"

+#ifdef __MINGW32__
+#undef fread
+#undef fwrite
+#endif
+
 #include <curl/curl.h>
 #include <curl/easy.h>
 #include <curl/multi.h>

I got rid of the overflow in conversion from 'long long unsigned int' to 'int' warning by patching Curl_multi.xsh:

--- Curl_multi.xsh_orig 2019-11-20 11:54:58 +1100
+++ Curl_multi.xsh      2019-11-20 11:55:35 +1100
@@ -564,8 +564,8 @@
 int
 socket_action( multi, sockfd=CURL_SOCKET_BAD, ev_bitmask=0 )
        Net::Curl::Multi multi
-       int sockfd
-       int ev_bitmask
+       IV sockfd
+       IV ev_bitmask
        PREINIT:
                int remaining;
                CURLMcode ret;

though it doesn't seem to make any difference to the way that the test suite behaves .... and the test suite doesn't behave very well.
A lot of the tests fail or hang. And the examples scripts hang or crash - except for examples/04-share-threads.pl (which runs fine).

The Net-Curl-0.41 test suite covers much more than the Net-Curl-0.37 suite did. 0.37 passed all of its tests, and 0.41 still passes the same tests.
The problems are with the other tests in 0.41 (ie the ones that weren't in 0.37).
I've no idea how useful Net-Curl is on Windows - I think it's a good idea to stick with Inline::C if you can get it to do what you need.

I also built Net-Curl-0.41 with Strawberry Perl 5.30.0.1 - again, building against the MSYS2 libraries.
I get identical results with Strawberry Perl - which is to be expected.
Strawberry Perl's perl/bin/pkg-config.bat doesn't automatically find the Curl library, so I renamed it to pkg-config.bat_hide. With that done, the MSYS2 mingw64/pkg-config.exe gets used instead, and the Curl library is found.

Cheers,
Rob

[xiaoyafeng]

Really Thanks Rob, these two days I've also dig the issue of libcur and strawberry perl. Plus your suggestions and research, I believe the following points will help build libcurl on windows successfully.
• use MSYS2 libcur-dev, don't download directly or use Alien::Curl etc. since libcurl use libssh2.dll,libcrypto-1_1-x64.dll and many others, you may miss something.
• don't use pkg-config of strawberry perl just edit --libs --inc parameters
• use Inline::C other than Net::Curl, the latter is too linux to run smoothly on windows

Thanks again Rob, your replies and suggestions is like a guiding light.

[swl]

This might be tangential, but have you tried using Alien::curl and forcing a shared install so it downloads and compiles a recent version?

Access to the libs is then via Alien::curl->libs and similar. See the examples in the Alien::curl and Alien::Base documentation.

[xiaoyafeng]

I've tried Alien::Curl, Unfoturatly, it seems download wrong curlxxxx.dll, so it doesn't get things done also ( I don't dig it so much, it seems mingw curl need a runtime dll, )