Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Clean smoke-test install for Inline based modules using Inline::MakeMaker

by davido (Archbishop)
on Dec 14, 2011 at 15:34 UTC ( #943568=perlquestion: print w/ replies, xml ) Need Help??
davido has asked for the wisdom of the Perl Monks concerning the following question:

I recently took over Inline::CPP, which hadn't been maintained since 2003. When I took it over it was passing with only about 10% of the CPAN smoke testers. The points of failure traced back to ways in which C++ has standardized and evolved over the years, as well as how Inline has evolved. Once the points of failure were identified several individuals on the Inline mailing list (syphilis, Stefan, and Patrick) and I came together to work out some solutions that resulted in patches both to Inline and to Inline::CPP. Now Inline::CPP passes for a little over 93% of the smoke testers.

If you look in the documentation for Inline under the "Writing Modules with Inline" section, there is a list of steps needed to create a distribution that uses Inline::*. One of the steps is to substitute Inline::MakeMaker in place of ExtUtils::MakeMaker in the Makefile.PL file.

So far, so good, and that works as advertised. As a proof of concept I put Math::Prime::FastSieve on CPAN. This module is based on Inline::CPP. The intent was to demonstrate that basing a module on Inline was a reasonably robust solution, and if not, to identify and hopefully fix the points of failure.

When the smoke tests began trickling in I discovered another issue that is preventing some testers from getting a clean install. Several tests came back with a result of "UNKNOWN", and this is an example of the output from one tester. On the Inline email list we came up with the thought that these particular testers probably don't have Inline installed at all (which drags with it Inline::MakeMaker). So the solution I came up with was to add a 'BUILD_REQUIRES' => { 'Inline' => 0.49 }, directive in the Makefile.PL file. The thought was that at that point when I build the distribution Inline would be listed in the META.json and META.yml files, and the build dependency would be picked up by the cpan shell tool so that Inline would get installed prior to the target system invoking Makefile.PL.

So I bumped the version number to .03, and uploaded the changes. I seem to have achieved a marginal increase in smoke test passes, but I'm still getting some "UNKNOWN" results. And the test details for the remaining "UNKNOWN"s still seem to exhibit the same problem.

I'm wondering if any of you Monks wiser than I in the ways of building a bullet-proof distribution may come up with a suggestion on how to fix this issue.

Update: Fixed one of the module links.


Dave

Comment on Clean smoke-test install for Inline based modules using Inline::MakeMaker
Select or Download Code
Re: Clean smoke-test install for Inline based modules using Inline::MakeMaker
by BrowserUk (Pope) on Dec 14, 2011 at 15:51 UTC

    Could this be a part of the problem?

    C:\Perl64\packages\Math-Prime-FastSieve-0.03>makefile WARNING: BUILD_REQUIRES is not a known parameter. Checking if your kit is complete... Looks good Warning: prerequisite Inline::CPP 0.33 not found. 'BUILD_REQUIRES' is not a known MakeMaker parameter name. Writing Makefile for Math::Prime::FastSieve

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?

      Gah! I was afraid I was missing the obvious. In the absence of docs for Inline::MakeMaker I assumed it supported the same parameters as ExtUtils::MakeMaker, which apparently it doesn't. Back to the Inline list to work on that. :)


      Dave

      Hmm, on further inspection, it's not quite as simple as that, and I'm still confused. When I execute:

      perl Makefile.PL make make test make dist

      I get no such warning at any stage in the process. How is it that the target in some cases is coming up with that warning?

      In fact, when I examine the META.* files I see that the build requirement is getting listed therein when I make dist. So I'm still bewildered.

      I'm learning here too, and the whole concept of basing a module on Inline seems to be a road less traveled. In fact there are tools to automate the process of converting an Inline extension to plain old XS for the purposes of distribution. So this isn't really a critical adventure. But if we're not too far away from making Inline based modules distributable I'd like to see it through.

      ExtUtils::MakeMaker is always somewhat of a quagmire. I suspect even more-so when we're dealing with Inline::MakeMaker. But it seems like the goal isn't that far away.

      If I haven't said so before, thanks for your reply both here and to the many other threads where I've learned something from your answers.


      Dave

        I think it indicates that I have a down-level version of MakeMaker installed that doesn't have that keyword. The problem is, I cannot tell from teh error message whether I should upgrade ExtUtils::MakeMaker or Inline::MakeMaker?

        I guess I could try both one at a time to see. I'll get back to you after I've eaten.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?

        BUILD_REQUIRES was added in c8d5ea031 (June 17, 2009). It looks like this may be first introduced into a tagged release code at 6.55_03 (I am not certain on this, other than the mention in Changes -- haven't taken the time to track it down further than that). Perhaps the version on the testers machine is older.

        --MidLifeXis

        Grrr. I hate EU::MM. I have more trouble with that module than all the other modules I've used put together!

        Every other module I've ever 'gunzip & tar -xvf'd, unpacks all it contents into a subdirectory it creates with the same name as the package. The latest version of EU::MM just dumped it's entire contents into the current directory. Which means an hours work trying to clean up after it. What a crock!

        I'll get back to you to tomorrow.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The start of some sanity?

Re: Clean smoke-test install for Inline based modules using Inline::MakeMaker
by Anonymous Monk on Dec 14, 2011 at 19:18 UTC

    I think you might be the only one using Inline::MakeMaker for a cpan distribution :)

    "Inline::MakeMaker" site:search.cpan.org

    I think the solution might be to simply inline Inline::MakeMaker into Makefile.PL, and maybe add

    use ExtUtils::MakeMaker 6.49_01 ();

    Or, switch to http://search.cpan.org/perldoc/Module::Install#Module::Install::Inline

      Yup, unless you have Inline::MakeMaker, none of

      $ pmvers CPANPLUS CPAN CPANPLUS: 0.9111 CPAN: 1.9800

      will try to install Inline::MakeMaker after Makefile.PL dies. I doubt they consult META.json (cpanm might).

      So yeah, I think the inline approach is definitely the ticket :)

        I doubt they consult META.json

        What davido has presented here is a specific example of a more general problem.

        If a Makefile.PL contains:
        use Foo;
        how is the cpan shell then supposed to know (*before* executing that Makefile.PL) that it needs to check whether Foo.pm is installed (and immediately run off and install Foo.pm if it's not).
        I would expect that there should be *some* way of effecting this.

        Otherwise, any distro that use's a non-core module in its Makefile.PL runs the risk of being hammered in exactly the same way as davido's module.

        Cheers,
        Rob
Re: (solved) Clean smoke-test install for Inline based modules using Inline::MakeMaker
by davido (Archbishop) on Dec 15, 2011 at 16:16 UTC

    In Makefile.PL ensure that the following directive gets passed to WriteMakefile():

    CONFIGURE_REQUIRES => { 'Inline::MakeMaker' => 0.45, 'ExtUtils::MakeMaker' => 6.62, },

    This is a directive that only alters the way that make dist composes the META.yml and META.json files. In specific, those files will contain a configure_requires section. The various install tools (cpan, cpanm, and cpanp) will read the META.yml file, see what the "configure" requirements are, pull them in, and install them prior to executing Makefile.PL. At that point Makefile.PL will run cleanly as it will have access to the versions of ExtUtils::MakeMaker and Inline::MakeMaker that will allow a trouble-free install. As an added benefit, by causing EU::MM to upgrade (if necessary), the warning about not recognizing the CONFIGURE_REQUIRES directive goes away.

    The directive really has no effect on the target system. All the primary effects take place at the time the distribution is composed and packed up. The fact that everything will now work on the target system is because the distribution was built in a more failsafe way.

    As proof of concept refer to Math::Prime::FastSieve version 0.04. The smoke tests have begun trickling in, and with 18 of them I haven't yet seen an "UNKNOWN".

    To assist the next guy who comes along the docs for Inline should probably contain the following modification in the Writing Modules with Inline section:

    The following text:

    Finally, you need to modify the Makefile.PL. Simply change: use ExtUtils::MakeMaker; to use Inline::MakeMaker;

    Should say:

    Finally, you need to make the following modifications to Makefile. +PL. Change: use ExtUtils::MakeMaker; to use Inline::MakeMaker; And add the following directive to WriteMakefile(): CONFIGURE_REQUIRES => { 'Inline::MakeMaker' => 0.45, 'ExtUtils::MakeMaker' => 6.62, },

    My rationale for wanting to get a clean "one touch" install is this: If someone attempts to install a module and it fails, they may just move on to some other module, even if the one they chose initially would have been a very good choice had they gone through the extra steps of figuring out why it's not installing. I feel that part of the responsibility for quality insurance that an author takes on is that his module should install successfully using common conventions, without expecting the user to sift through a screen-full of error messages. It's sort of "putting your best foot forward."

    As for "Why base a module on Inline?" Well, it's documented in the Inline POD as being a capability. There are tools such as InlineX::CPP2XS that will convert a module with Inline::CPP dependency to an XS module. But that may not exist for Java, Python, etc. So by ironing out the details, all Inline-dependency authors benefit.

    On the down-side of requiring an Inline dependency in a module, there is yet another dependency for the user to deal with. Sort of like pulling in Moose for a trivial task, except the only performance penalty with Inline-based modules is paid at install time. Compiletime for code using an Inline-based module won't be impacted negatively in any measurable way. But on the other hand, making the Inline suite of tools more bullet proof could facilitate increased innovation in writing Perl extensions, as the barrier to entry is lower than writing pure XS.

    But all of this is just how I rationalize the endeavor in my own mind, when the real reason is that I've been having fun with it. That's how it should be, right? :)


    Dave

      Math::Prime::FastSieve still doesn't install for me. I used cpan this time and it seems to have done the right thing by all the dependencies, but the compileation of the module itself fails with:

      t/01basic.t ... Microsoft (R) Program Maintenance Utility Version 9.00.21022.08 Copyright (C) Microsoft Corporation. All rights reserved. C:\Perl64\bin\perl.exe C:\Perl64\lib\ExtUtils\xsubpp -typemap + "C:\Perl64\lib\ExtUtils\typemap" -typemap "C:\Perl64\cpan\build\Inli +ne-CPP-0.33-uracNj\_Inline\build\_01basic_t_5cd2\CPP.map" _01basic_t +_5cd2.xs > _01basic_t_5cd2.xsc && C:\Perl64\bin\perl.exe -MExtUtils:: +Command -e "mv" -- _01basic_t_5cd2.xsc _01basic_t_5cd2.c cl -TP -c -I"C:/Perl64/cpan/build/Inline-CPP-0.33-uracNj/t" +-nologo -GF -W3 -MD -Zi -DNDEBUG -Ox -GL -Wp64 -fp:precise -DWIN32 -D +_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DWIN64 -DCONSERVATIVE -DUSE_S +ITECUSTOMIZE -DPRIVLIB_LAST_IN_INC -DPERL_IMPLICIT_CONTEXT -DPERL_IMP +LICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX -MD -Zi -DNDEBUG -Ox -GL + -Wp64 -fp:precise -DVERSION=\"0.00\" -DXS_VERSION=\"0.00\" "-IC +:\Perl64\lib\CORE" _01basic_t_5cd2.c cl : Command line warning D9035 : option 'Wp64' has been deprecated an +d will be removed in a future release cl : Command line warning D9035 : option 'Wp64' has been deprecated an +d will be removed in a future release _01basic_t_5cd2.c _01basic_t_5cd2.xs(2) : fatal error C1083: Cannot open include file: ' +iostream.h': No such file or directory NMAKE : fatal error U1077: '"c:\Program Files (x86)\Microsoft Visual S +tudio 9.0\VC\Bin\amd64\cl.EXE"' : return code '0x2' Stop. A problem was encountered while attempting to compile and install your + Inline CPP code. The command that failed was: c:\PROGRA~2\MICROS~1.0\VC\Bin\amd64\nmake.exe > out.make 2>&1 The build directory was: C:\Perl64\cpan\build\Inline-CPP-0.33-uracNj\_Inline\build\_01basic_t_5 +cd2 To debug the problem, cd to the build directory, and inspect the outpu +t files. at t/01basic.t line 64 BEGIN failed--compilation aborted at t/01basic.t line 64. t/01basic.t ... Dubious, test returned 2 (wstat 512, 0x200) Failed 10/10 subtests ...

      From the error message: _01basic_t_5cd2.xs(2) : fatal error C1083: Cannot open include file: 'iostream.h': No such file or directory and a little googling, it looks like this is a common problem centred around using: #include <iostream.h>

      rather than the (MS?) preferred:<code>#include <iostream></c>


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

      The start of some sanity?

        Thanks for that report. That has to be an Inline::CPP issue, so a separate problem. syphilis and I thought we had solved it a couple weeks ago, but it looks like more attention is required. I'll get to work on it. You must be in the 7% that are still having problems with Inline::CPP. My goal there is to get that module to the point that it installs for all systems that Inline::C installs for (assuming they have a C++ compiler), which would be about 98%.

        More details: The problem is related to how C++ has evolved and standardized. As I'm sure you know better than I, before a standard was adopted C++ headers usually had the .h extension. More recently (which really isn't all that recent now) C++ standardized around headers without the .h extension. Inline::CPP automatically includes <iostream>, or <iostream.h>. It's supposed to detect which is required, and for the majority of systems it seems to get it right. It looks like in your case it's guessing wrong. Now that I know it's still an issue I can turn my sights on it again. It's enough of a sore-spot that I'm almost inclined to eliminate the automatic inclusion of <iostream>, but that wouldn't really solve the problem of distributing code based on Inline::CPP, as it just forces module authors to try to figure out how to decide between '.h' or no '.h' themselves.


        Dave

      As an added benefit, by causing EU::MM to upgrade (if necessary), the warning about not recognizing the CONFIGURE_REQUIRES directive goes away.

      I gather that's the *only* reason for adding the EU::MM requirement - I don't see that it could serve any other purpose.
      No problem with any of that but I'll specify '6.52' instead of '6.62'. According to the EU::MM Changes file 6.52 would be the earliest stable version to be aware of the CONFIGURE_REQUIRES option. It so happens that I have 6.52 on one of my builds of perl, and it doesn't complain about the option. (Version 6.48 *does* complain.)

      I'll wait for a week in case there are any more developments, then I'll upload your suggested amendment to github so that it's sitting there in readiness for the next Inline release.
      It's unlikely I'll forget about it, but please ping me if I do :-)

      Cheers,
      Rob

        Yes, that's the only reason. Perhaps it's not a very good reason, as a warning wouldn't cause an install failure anyway, right? Might not be worth forcing people to upgrade EU::MM just to squelch a warning.

        However, it would be worth mentioning in the POD for Inline that the CONFIGURE_REQUIRES directive requires that the person creating the distribution have EU::MM version 6.56 (or whatever it was) installed on his system.

        Opinions?


        Dave

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://943568]
Approved by marto
Front-paged by Old_Gray_Bear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (12)
As of 2014-12-22 09:16 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (113 votes), past polls