Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

RFC: Setting up a minGW compiling envronment for Perl 5.10

by Bloodrage (Monk)
on Mar 02, 2008 at 10:30 UTC ( #671479=perlmeditation: print w/ replies, xml ) Need Help??

Introduction

I need to compile Perl modules for Windows. In particular I want to compile Gtk2, but it would seem that it's one of the hardest to compile.

The easy answer of course is to install PPM (randyk tells us how to do it without a working compiler.), but that relies on someone else having a working compiling environment and the same version of Perl as you do, why let them have all the fun anyway. I am using Strawberry Perl 5.10, and unfortunatly it would seem that no-one has recompiled anything exciting for Perl 5.10 and put it up in their repositories, they've all probably given up with the hideous number of failures in the ActiveState PPM repository for 5.10. So as convenient PPM is, it's not a universal solution. Having a working compiler environment is a universal solution, provided you're prepared to swear and curse a bit...

Usually planetscape response with genuinly helpful replies like this or this, and there is this tutorial as well. Unfortunatly the material is slightly dated, but probably still current.

Another common comment is about using the same compiler as your Perl install (often by Corion), this issue is avoided in this case by using Strawberry Perl, which presumably comes with the very same compiler that was used to compile Strawberry Perl (gcc in the form of minGW).

Setup

So far I have:
  • A compiler, minGW, and it's the same as used to complile my Perl version.
  • ...and the set of 'standard' libraries as distributed with Strawberry Perl
  • Precompiled Gtk2 and Glib, binaries and libraries as provided by the Sourceforge Project
  • The include and lib directories from the Windows Platform SDK (OMG it's 1.4GB!... phew the lib and include directories are only about 100MB)
  • All the library files renamed to have the .a extension thanks to this program1

Problems with "undefinded reference to" errors

Starting with Glib because I think that Gtk might be dependent on it, our first compile gives us a thousand or so undefined reference to errors, which syphilis tells me indicates that the compiler can't find some library files. There are three possible reasons for this:

  1. the compiler hasn't been told where the libray file is
  2. the library isn't where the compiler thinks it is, or it's got the wrong name (see the comment above about renaming libraries with a .a file extension)
  3. the library is missing or not installed ("easy" solution, find it, install it)

We managed to make it compile by appending missing Glib libraries to EXTRALIBS and LDLOADLIBS in the Makefile in the general form of $ENV{'GTK_BASEPATH'}.'\lib\'.$lib_file. This works, Glib compiles. Huzzah! Now shouldn't Makefile.PL do this for us? Or throw an error if $ENV{'GTK_BASEPATH'} is not set or null?

This can be generalised for any undefined reference to errors by checking the #     LIBS => comment in the Makefile kindly left there by MakeMaker (in this case -lglib, -lgobject, and -lgthread) and locating the library files listed there and appending their complete paths to EXTRALIBS and LDLOADLIBS.

Missing header files

Next on the list is cairo, which only requires cairo.a added to the library paths. Easy you think. Not quite. Cairo suffers from the other common compiler issue, 'missing' header files, in this case cairo.h. I use 'missing' in quotes because like before, it's more likely that the compiler hasn't been told where the header file is, rather than it's absent. In this case it's simply a matter of adding the path to the directory containing the header file in question INC entry in the Makefile. Now the phrase 'INC' may appear hundreds of times in the Makefile so the key here is to look for 'INC =' ('INC=' won't work because MakeMaker tends to use that format for it's comments). Do that and it's done.

Missing dll reference libraries

On to Gtk2, which exhibits both of the previous issues, and an additional one. After compiling and installing Glib and Ciaro, explicitly linking their libraries, and include directories, there still seem to be at least one library file missing. These are the dll reference libraries, which have not been created or installed by the Makefile. Again syphilis comes to the rescue with instructions on how to use dlltool:

  1. During the building of the Glib module, a file named Glib.def is created. Using Visual Studio's lib utility create, from Glib.def, an import lib named Glib.a as follows:
    lib /OUT:Glib.a /DEF:Glib.def
    With MinGW, the same can be achieved with:
    dlltool --input-def Glib.def --output-lib Glib.a --dllname Glib.dll
  2. Repeat step 1 for the Cairo module.
  3. Copy Glib.a to
    C:/strawberry/perl/site/lib/auto/Glib
  4. and copy Cairo.a to
    C:/strawberry/perl/site/lib/auto/Cairo
    I'm assuming a standard Strawberry Perl directory structure - modify accordingly.
  5. Open the generated Makefile and, to the EXTRALIBS and LDLOADLIBS entries (in the MakeMaker const_loadlibs section), append:
    C:\strawberry\perl\site\lib\auto\Glib\Glib.a C:\strawberry\perl\site\ +lib\auto\Cairo\Cairo.a

Conclusion

The compiler environment provided with Strawberry Perl is complete, but when a module requires additional libraries there will be some Makefile editing required. The irritating part of this problem is that the locations for all the library and header files is specified in the $ENV{'LIB') and $ENV{'INCLUDE'} environment variables, which have been incorporated into CPAN's Config.pm (well they have explicitly by my reconfiguring script) and passed to Makefile.PL. My initial assumption is that Makefile.PL is either discarding this information or it's not doing enough with it to explicitly incorporate it into the Makefile. Now that we have a more standardised compiler environment with minGW that works with both Strawberry Perl and ActiveState Perl, perhaps module developers can be more certain about the compiler environment available to Perl and CPAN, hence might be able to make more robust Makefiles for Windows. Alternativly Makemaker could be updated to do this for them.

Capturing output from the make utility

Thanks to syphilis's reply we can capture compiler and make output can be captured with the following commands:

perl Makefile.PL >p.txt 2>&1 dmake >d.txt 2>&1 dmake test >t.txt 2>&1 dmake install >i.txt 2>&1

Please comment and tell me if I need or should do something.

Other Resources

Finally there are also these resources online:

UPDATE: We have complied Glib, incorporated material into meditation. More updates... progress. Also I want Gtk2 not Gtk. Our most gracious mentor syphilis (give him all your ++) has managed to compile Gtk2, I have now replicated this feat.

UPDATE: I have now added all our work to the root node. Please comment if I have made any gross errors, or misleading anyone.


PS: this is a work in progress, I haven't yet finished reading through all the materials I've linked to. This meditation is as much a reference for me to work from as it is to inform. Eventually I hope to unify all this into a How-I-Did-It kinda post.

1 Thanks to syphilis for mentioning this

Comment on RFC: Setting up a minGW compiling envronment for Perl 5.10
Select or Download Code
Re: RFC: Setting up a minGW compiling envronment for Perl 5.10
by syphilis (Canon) on Mar 02, 2008 at 11:23 UTC
    ..this issue is avoided in this case by using Strawberry Perl, which presumably comes with the very same compiler that was used to compile Strawberry Perl (gcc in the form of minGW)

    That is correct ... but just to confuse you, you should be able to use the very same compiler (and the very same dmake) with recent ActiveState builds of perl.

    Also... is there a compiler flag that redirects the output to a log file

    I use:
    perl Makefile.PL >p.txt 2>&1 dmake >d.txt 2>&1 dmake test >t.txt 2>&1 dmake install >i.txt 2>&1
    Then check the relevant file (ie p.txt, d.txt, t.txt, or i.txt) to get the output.

    I haven't tried to build GTK2 (nor do I presently intend to), but your "precompiled Gtk2 and Glib, binaries and libraries as provided by the Sourceforge Project" should be usable with your MinGW (though you might need to additionally link to some of the MinGW libs - eg libgcc.a ... error messages would be handy). You shouldn't need "the include and lib directories from the Windows Platform SDK" - and I would suggest making them invisible (to begin with, anyway). Usually, a lot of the problems with these modules is that the authors feel obliged to try to guess where the external libraries are located. It's a regrettable state of affairs ... often a result of a misguided attempt to get the module to build via the automated CPAN.pm build procedure. Anyway ... let's start by looking at your 'dmake' output. What is it ?

    Cheers,
    Rob

      First I'll quickly cover generating my Makefile. When I do exactly as you instructed I get this output for generating my Makefile:

      ...but when I use CPAN to generate the makefile, which explicitly passes a more complete set of include and lib paths to Makefile.PL (because of the program I've mentioned earlier) I get this output:

      Which has none of the "No library found" warnings. So I'll continue with the CPAN generated Makefile.

      I'm making Glib first as I suspect that Gtk may be dependent on it. Following your instructions here is the rather copious output from dmake >d.txt 2>&1:

        undefined reference to ....

        Looks like the glib libraries are not being found. If you're using Strawberry Perl, and libraries that were built with Visual Studio, then you probably need to change the '.lib' extensions to '.a'.

        Cheers,
        Rob
      With all the library files renamed with s/.lib$/.a/i the output is now (I now run dmake twice, which reduces the output to just the failed parts), hmm... no -L"G:\GTK\lib", so my GTK libraries are not being handed to the compiler by dmake?

        Appending -L"G:\GTK\lib" to these lines in the Makefile like so:

        LDDLFLAGS = -mdll -s -L"G:\strawberry\perl\lib\CORE" -L"G:\strawberry\ +c\lib" -L"G:\GTK\lib" LDFLAGS = -s -L"G:\strawberry\perl\lib\CORE" -L"G:\strawberry\c\lib" - +L"G:\GTK\lib"

        Makes no difference to the output from dmake except that these libs are now included :(

      A Quick recap for those who can't be bothered drilling down through the reply tree to it's darkest depths. We have successfully compiled Glib by first renaming all the library files s/.lib/.a/i (here's a program), then appending the Glib libraries to EXTRALIBS and LDLOADLIBS in the Makefile. This is dispite Makefile.PL being given this information by CPAN and MakeMaker.

      Hence for any required Glib|Gtk library file, $ENV{'GTK_BASEPATH'}.'\lib\'.$lib_file has to be appended to the end of LDLOADLIBS in the Makefile.

        perl -V:lib_ext

      Righty, after repeating the key steps, to make sure it's repeatable, not at all because when you change your compiler-onna-stick to a new machine with a new drive letter, your hand-crafted Makefile is now broken...

      The next step; dmake test >t.txt 2>&1:

      deee deed dah dee... this takes a while

      Hmm, gets to test 9 and just hangs. (and only fails 6/12 of the 64-bit tests). I'll post this so far and see if it finishes.

      A quick look at 9.t makes me suspect that test 9 is for the timeout and thread yielding handling stuff. If I delete it all the other tests succeed except some in 64-bit.t and filename.t

Re: RFC: Setting up a minGW compiling envronment for Perl 5.10
by Bloodrage (Monk) on Mar 03, 2008 at 08:51 UTC

    Gtk2 want's cairo installed first. Cairo can't even find cairo-perl.h and cairo-perl-private.h... which are located in it's own build directory. :(

    Adding $ENV{GTK_BASEPATH}."\lib\cairo.a" to the end of <code>EXTRALIBS and LDLOADLIBS and added -I$ENV{GTK_BASEPATH}."\lib"  to <code>INC in the Makefile solves most of these issues. Now it cant find auto.h... which is absent from my libraries, so I'm missing a library set somewhere... Ah, bloody line brakes it's actuall </p>cairo-perl-auto.h which is in a sub directory of the biuld directory. Added that to the INC string and it compiles, and tests, and installs!

Re: RFC: Setting up a minGW compiling envronment for Perl 5.10
by rahed (Scribe) on Mar 03, 2008 at 13:53 UTC
    My perl environment on windows, particulary for GUI:

    perl 5.10 from sources, I don't use Strawberry because perl builds easily.
    MinGW 5.1.3 binary
    MSYS 1.0.10 binary
    GNUWin32 packages
    GTK+ precompiled libraries from http://gladewin32.sourceforge.net/modules/news/
    Glade 3.4.1 from sources
    Expat from sources


    This way from cpan prompt I can install any perl module.
    Everything is quite easy but for the gtk-perl modules:
    Gtk2, Glib, Cairo. The necessary fixes can be found in gtk-perl mailing list (http://news.gmane.org/gmane.comp.gnome.gtk%2B.perl).

Re: RFC: Setting up a minGW compiling envronment for Perl 5.10
by Bloodrage (Monk) on Mar 03, 2008 at 23:10 UTC

    Righty I now have Glib and cairo compiled (though they might not be entirely reliable... due to failed tests... or the tests don't work appropriatly on Windows) and installed. Onto Gtk2.

    Firstly, it needs to see it's own build directory added to it's INC entry in the Makefile... repeat for gperl.h, gtk2-perl-versions.h, and cairo-perl.h adding their own paths to INC of course.

    Then we get more undefined reference to errors, time to check the libraries... using the method described earlier.

    The only library passed to the Makefile by Makefile.PL and MakeMaker that I could not locate was gdk_pixbuf-2.0.a... part of the GNOME Imaging Model, ...morelikely it's because I haven't copied the new entries from EXTRALIBS into LOADLIBS :P

    Righty still getting undefined reference to errors, there's also this comment here which makes me think I need to figure out how to get the Makefile to link to Glib.dll.

    ...but for now I need to be brave and reinstall Windows XP on Her computer because it's being wierd.

    1 I just searched for "INC ="

      ... or the tests don't work appropriatly on Windows

      I think that's the case. I'm finding that many of the t/64bit.t tests fail - evidently the 64-bit support (which is expected to be in existence) is missing on Win32. I haven't investigated.
      Also test 14 of t/9.t hangs. All of the other tests in t/9.t pass.
      There are some tests in t/filename.t (and one test in t/g.t) that fail simply because backslashes appear in the path instead of the expected forward slashes. (These failures are inconsequential and can be ignored.)

      I need to figure out how to get the Makefile to link to Glib.dll

      Yes - that seems to be the case. But then there's also the need to link to Cairo.dll as well.

      To EXTRALIBS and LDLOADLIBS in the generated Makefile, I added (before running 'dmake'):
      C:\perl510_M\site\5.10.0\lib\auto\Glib\Glib.dll
      (which is the location of Glib.dll on my perl) and that takes care of the gperl_* references ... but there's still the cairo_* references to attend to. If I also add (to EXTRALIBS and LDLOADLIBS):
      C:\perl510_M\site\5.10.0\lib\auto\Cairo\Cairo.dll
      it doesn't help (unless that's inserted *before* the Glib.dll entry - in which case we've fixed the cairo_* references, but not the gperl_* references). That is, using this hack, we can fix either the cairo_* references or the gperl_* references ... but not both. I don't know why that is - and I haven't yet worked out a hack to fix it. I'm still puzzling over it - and will try to come up with something tomorrow. (Better still if someone can beat me to it.)

      Cheers,
      Rob

        oooh, um what about I:\GTK\lib\libglib-2.0.dll.a and I:\GTK\lib\libcairo-2.0.dll.a? Are these the libraries we're looking for?


        Can't check it here, this PC lacking in CPU... and I'd have to completely rebuild my Makefile.

Re: RFC: Setting up a minGW compiling envronment for Perl 5.10
by Bloodrage (Monk) on Mar 06, 2008 at 21:22 UTC

    syphilis has compiled Gtk2, and I have too using the advice he's given, and the methods I have used earlier.

    Unfortunatly my version fails all the tests! Throwing a Windows warning dialog This application has failed to start because (null).dll was not found. Re-installing the application may fix this problem. All the tests throw this error, and fail. Once the warning is cleared the output from the test program is usually in the form of:


    I think I'm very close now...

      Try creating copies of I:/strawberry/perl/site/lib/auto/Gtk2/Gtk2.dll named I:/strawberry/perl/site/lib/auto/Gtk2/(null).dll and I:/strawberry/perl/site/lib/auto/Gtk2/.dll. (I don't know if that will have the desired effect. I'm surprised that, having got it compiled, 'dmake test' doesn't run for you pretty much the same as it ran for me. Did you build it by altering the generated Makefile as I had done ? ... or did you do something slightly different ?)

      This just gets more and more ridiculous, doesn't it :-)
      Is Gtk2 really worth the effort ?

      Cheers,
      Rob

        I'll give that a go. Yes I followed your instructions and they worked nicely. I've also revised that library renaming script, but I still had to append the other external libraries and include directories.

        Gtk2 is probably not worth all this effort, but I'm a stubborn and persistant son of a bitch. This meditation is as much about putting the solutions to these compiler problems up front on Perl Monks as it is about getting it done. Soon I think I'll be supressing my rage and writing up another article on the robustness, or lack therof, of makefiles. :)

        Well the dll isn't there yet, it's not been installed. It's still in the blib/arch/auto/Gtk2 directory in the build directory. Renaming it as per directed didn't help... wait no it set of a dmake clean and deleted everything. Grr. Trying again.

        Righty the error is now The procedure entry point cairo_object_from_sv could not be located in the dynamic link library  (null). :(

        ...I am also beginning to suspect that there is another Config.pm file that I need modification as per the config script I use to set up Strawberry Perl.

      Now that we've sorted the mistakes with dlltool I now have these errors with dmake test. Sigh:
      t/set-scroll-adjustments-signal......Gtk-ERROR **: Incompatible build! The code using GTK+ thinks GtkBox is of different size than it actually is in this build of GTK+. On Windows, this probably means that you have compiled your code with gcc without the -mms-bitfields switch, or that you are using an unsupported compiler. at I:\strawberry\cpan\b +uild\Gtk2- 1.164-jWrGfH\blib\lib/Gtk2/TestHelper.pm line 41. This application has requested the Runtime to terminate it in an unusu +al way. Please contact the application's support team for more information. t/set-scroll-adjustments-signal...... Dubious, test returned 3 (wstat +768, 0x300
      Which causes it to skip almost all of the tests.
        Dang, adding -mms-bitfields to CCFLAGS in the Makefile simply results in gcc: unrecognized option `-MMS-BITFIELDS'... hang on writing this makes me notice it might be case sensitive... yup. Case sentitive. Oooh some tests segfault :)
Re: RFC: Setting up a minGW compiling envronment for Perl 5.10
by spicyjack (Beadle) on Mar 14, 2008 at 22:46 UTC

    I've gotten this to work with Perl 5.8, and I just (5 minutes ago) finished a basic Cairo/Glib/Gtk2 Perl stack for Perl 5.10.0.

    My setup:

    • Windows XP SP2
    • MinGW 5.1.3
    • GTK Windows Libs (gtk+2.12.8, glib-2.14.6, cairo-1.4.14, pango-1.18.4, atk-1.20.0)
    • Gtk2-Perl Source (Cairo-1.045, Glib-1.180, Gtk2-1.180)
    • Perl 5.10.0 source
    • dmake 4.11-20080107 (from CPAN)
    • UnxUtilities (optional, but helpful)

    The order of building is Perl, Cairo, Glib, and then Gtk2. My notes from this latest build are posted here:

    http://wiki.antlinux.com/pmwiki.php?n=CamelBox.Feb2008Start

    It's 4 pages in sequence (follow the links at the bottom of the page to navigate). I have not updated the "Extra GTK Apps" page yet, but I don't expect any surprises. I don't know where you're at in your build (sounds like you got Cairo and Glib working, but not Gtk2), so maybe this will help you. If you do use my notes, at the very minimum, you'll have to change anything with a path of C:\camelbox to C:\strawberry, but I think that's about all you would need to do. For your Windows GTK libraries, are you also getting all of the support libraries that the GTK libraries are compiled against? The GTK libraries I used are on this page:

    http://www.gtk.org/download-windows.html

    I have all of the Binaries/dev packages under the GTK+ section, as well as all of the "Third Party Dependencies" that are marked with a checkbox image on the right hand side of that table.

    I don't even bother with running the tests, I'm on the gtk-perl mailing list and the developers have acknowledged that the tests *will* fail under Windows. But I can still get a working and current Gtk2-Perl stack on Windows, so I'm happy.

    PM/post here if you need any pointers.

    Thanks,

    Brian

    P.S. My demo app (outside of the Gtk2-Perl demos that come with the Gtk2-Perl source):

    http://cvs.antlinux.com/cvsweb.cgi/perl_scripts/gyroscope.pl... a web color-picker :)

      I'm on the gtk-perl mailing list and the developers have acknowledged that the tests *will* fail under Windows

      IIRC a number of the tests fail with errors along the lines of (eg, from gdk.t) Can't locate object method "get_xid" via package "Gtk2::Gdk::Window" at t/gdk.t line 108. Is that *really* a Windows-only error ?

      I also think that having pkg-config and glib be mutually dependent (but separate) is bad software design ... though really brilliant April Fool's Day prank design. And I feel the same away about the need for Gtk2 to link to the auto/Cairo/Cairo.dll and auto/Glib/Glib.dll.

      I have a question for you. I found I was able to link directly to either auto/Cairo/Cairo.dll or auto/Glib/Glib.dll ... but not both. Do you know why that is ? Presumably, it's a perl quirk/bug, since, if I build a C application using MinGW's gcc, I can link directly to as many dll's as is needed. But with the perl dll's, it seems that *one* is the limit.

      I even went to the trouble of building a Foo module and a FooBar module - where FooBar had to link directly to the Foo.dll. And that worked fine. I then built a Bar module, and changed FooBar so that it needed to link directly to both Foo.dll and Bar.dll. It couldn't do that - I could get it to link to one or the other, but not both. So I created import libs (which I think is what you have done, too) and linking to them worked fine. I still don't see why linking to *more than one* dll should fail.

      Cheers,
      Rob
Re: RFC: Setting up a minGW compiling envronment for Perl 5.10
by Bloodrage (Monk) on Mar 17, 2008 at 01:03 UTC

    Sorry y'all. I've been working on other things for a bit ($work/2, A fence, a LAN party, and a busted Lada Niva mostly).

    Using dlltool to create the library files from the definition files seems to have worked. Gtk2 seems to be working and I'm just about to start working through the Gtk2-perl tutorials and stuff.

    I've noticed that a lot of you also seem to have the Perl source available (which I don't) and I'm wondering if getting that set up to compile makes a difference. OTOH it's a bit out of the scope of this project which is specifically about using Strawberry Perl 5.10 and what it provides (which seems to be enough).

    My final conclusion is that Strawberry Perl does provide everything you need to compile modules, provided that they don't need more libraries. Almost everything I've done is telling the compiler where these external libraries are. I haven't yet figured out if this is a deficiency in the Makefiles, or in the way that minGW is set up in Strawberry Perl. Both seem equally likely.

    My next steps will be to rewrite this as two more meditations (one on setting up a complier, another on compiling Gtk2-perl) as this has kind of turned into a "How can I X with Y and Z?" kind of problem. I also think I need to set up minGW from scratch so that I can get a better understanding of that process, as I suspect as most of my problems could be made to go away with a reconfiguration script. I may also have to look into ExtUtils::MakeMaker.

      Sorry y'all. I've been working on other things for a bit ($work/2, A fence, a LAN party, and a busted Lada Niva mostly).

      Hahaha, Nivas are cool as hell. You ever seen where they took a Niva and totally submerged it, ran it around the bottom of a shallow lake in Russia wearing scuba gear, with nothing but snorkels for intake/exhaust.

      I've noticed that a lot of you also seem to have the Perl source available (which I don't) and I'm wondering if getting that set up to compile makes a difference. OTOH it's a bit out of the scope of this project which is specifically about using Strawberry Perl 5.10 and what it provides (which seems to be enough).

      I don't want to knock Strawberry/Vanilla Perl, I just find that if I do it myself, I understand the whole setup better, and have a somewhat better chance of figuring out what goes wrong when things do go wrong. After I get a Windows Installer built for my Perl/GTK build, I'd like to try to see if the same instructions work on Strawberry/Vanilla Perl as well as my own build. You can watch my Gtk2-Perl installer package progress here:

      http://code.google.com/p/camelbox/

      My final conclusion is that Strawberry Perl does provide everything you need to compile modules, provided that they don't need more libraries. Almost everything I've done is telling the compiler where these external libraries are. I haven't yet figured out if this is a deficiency in the Makefiles, or in the way that minGW is set up in Strawberry Perl. Both seem equally likely.

      From the way it was explained to me in a set of posts on the gtk2-perl mailing lists (my original post, a reply later on in the same thread), the ExtUtils::Maker module could not find the right libraries on Windows for whatever reason, hence the need to do the hand-hackery. ExtUtils::Maker is what generates the Makefile that the Gtk2-Perl modules are built from.

      Enjoy,

      Brian

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://671479]
Approved by Corion
Front-paged by Old_Gray_Bear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (10)
As of 2014-10-02 15:20 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    What is your favourite meta-syntactic variable name?














    Results (64 votes), past polls