Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Perl6 - Distribution builder/installer

by dragonchild (Archbishop)
on May 30, 2005 at 01:49 UTC ( #461627=perlmeditation: print w/ replies, xml ) Need Help??

(This is a continuation of Perl6 - Module::Build.)

After reading those replies, talking on #perl6, and emailing perl6-language, I've come to the following conclusions:

  • There is a problem with how the problemspace is being divided up. There seems to be a call for having a separate distro builder and distro installer.
  • Actually, there seem to be six separate components in the whole distribution management space:
    1. Distro builder (ExtUtils::MakeMaker / Module::Build)
    2. Distro installer / uninstaller (EU::MM / M::B)
    3. Distro metadata inspector (No P5 analog)
    4. Distro distribution center (C6PAN)
    5. Distro up-/down-loader (Think CPAN/CPANPLUS and the cpan_upload script)
    6. The 'use' and 'require' keywords
    I include the latter because all 6 components need to be aware of S11.
  • Because of this, I suspect there's a seventh, shared component that controls how @*INC is searched and which of the competing options is selected.
  • That shared component needs to be mixin-able, with its Roles overridable. Because the way authoring/versioning metadata is stored will vary by OS, there needs to be OS-specific code in there.
  • A major issue, which apparently has not been discussed a lot, is how to implement the fact that Parrot (and any other VM P6 may target) will provide for multi-lingual distributions. The only comparison that exists is .NET, and that problem is solved by the fact that all the languages are under the same architecture team. We don't have that luxury.

Now, what I would like from the community:

  • What actions would you like to see from the various components? What would those actions entail? Would there be dependencies between actions? If so, could those dependencies be overriden (a la "perl -MCPAN -e 'force install Foo::Bar'")
  • Ideas regarding the Parrot dillema, particularly ideas crossposted to p6-language.
  • Comments on anything and everything I've posted so far.

  • In general, if you think something isn't in Perl, try it out, because it usually is. :-)
  • "What is the sound of Perl? Is it not the sound of a wall that people have stopped banging their heads against?"

Comment on Perl6 - Distribution builder/installer
Re: Perl6 - Distribution builder/installer
by Zaxo (Archbishop) on May 30, 2005 at 03:31 UTC

    I think that many of the problems shared by those components could be relieved by putting some teeth into Config.pm. Presently in P5, Config.pm has lots of potentially useful details about the local system, but many have the disclaimer in pod that they shouldn't be relied on.

    System utility paths are reset to some vanilla value after configuration (bash, bison, byacc, cat, . . .). Why not leave them intact? Why not have a localization script which can correct those entries?

    Config.pm seems quite accurate to me, but I'm on linux and I always build my own perl from source. That makes most things Just Work for me. With the fresh start of P6, there is the opportunity to tell vendors that the configuration must be accurate as installed and used on the local system.

    After Compline,
    Zaxo

      I do understand that it's easier to be a nay-sayer than to contribute positively, so I kinda regret doing this.

      For non-shared perls, that's great. For shared perls, it merely might be good. There is a lot of thought that needs to go into giving Config.pm the teeth it needs. Some of it is reasonable, some much more difficult.

      There are at least a few different uses for Config.pm that I'm aware of, and thought needs to be applied to all uses (whether I've thought of them or not). That doesn't mean that they all need to stay in Config.pm in Perl6, only that these different "users" of Config.pm still can do what they need to. For lack of better terms, they are:

      1. The IT guy. This is the guy whose task it is to compile a perl that is usable by one or more machines. This is the person who is creating the initial Config.pm (though s/he may not realise it) to compile perl itself, plus all the modules that come with Perl.
      2. The global module installer. This is the guy whose task it is to install more modules over the rest of the use of this copy of perl.
      3. The local module installer. This is someone who doesn't have full privileges to the system, and thus is installing some private modules to use with the global perl.
      4. The user. Someone who is running perl code, not necessarily caring that the code is in perl.

      Except for the first role, each role has a many-to-many relationship to people: more than one person may have sysadmin authority to install global modules, more than one person (possibly on more than one computer) will install local modules, and, hopefully, more than one actual end user, but even then there may be just one end user (and that user could be cron or apache).

      At the other end, for example at home, all these roles are filled by me. The configuration must work for all these roles, or as many roles as are solvable and realistic (the user role must always work for all machines sharing the same perl).

      That said, this may really work, with a good design. I just think it's a bit more complex than you're implying, and would look forward to some group coming up with a universal solution here.

      Two immediate reactions.

      The first is that you can insist all you want, but when vendors ship binary distributions they won't pay much attention to whether the configuration is accurate, and users won't be first to blame the vendor for those mistakes.

      The second is that I've done several network installs of Perl. That's where I install Perl to a file server that a bunch of Windows machines all see, instead of installing to each machine. Yes, I know all of the ways that this is fragile. But it has worked very well in practice, and you'll find that people will do this whether you like it or not.

      The shared item I was thinking of is the OS-dependent bit that navigates the modulename-specification that S11 lays out. I can imagine it being implemented with symlinks on *nix, but that's not going to work on VMS.

      As for Config ... I actually don't know how much is going to be needed, frankly. Especially given the idea floating around that documentation for a module should be within that module's install. Maybe the manpages for Perl shouldn't be in $Config{man1}, but $Config{perlmanpages} and let the understanding of that be OS-dependent.


      • In general, if you think something isn't in Perl, try it out, because it usually is. :-)
      • "What is the sound of Perl? Is it not the sound of a wall that people have stopped banging their heads against?"
Re: Perl6 - Distribution builder/installer
by BrowserUk (Pope) on May 30, 2005 at 06:05 UTC

    A few thoughts for consideration.

    • All files relating to a particular module(version/author) should exists in a directory tree below that module's place within the file system. Including, but not limited to:
      1. Distribution files (.tar.gz etc.)
      2. Build files (makefiles, source files)
      3. Built files including binary (dynamically linked) components.
      4. Test files.
      5. Build process log files.

      One exception might be generated docs like htmlized POD/man pages.

    • Parrot can save/load bytecode files.

      At use time, the first thing looked for should be a pre-compiled Parrot bytecode file. If found, this can be loaded, bypassing the need for compilation. Some check for updates should be possible.

      When a module is installed and tested, the bytecode is written out to the appropriate place (if tests pass?) and that module need never be recompiled until a new version is downloaded.

    • If I've been following things correctly, Parrot bytecode is platform independant?

      If so, it would sometimes be convenient to be able to download the bytecode only from CP6AN. Even better if this could be done (safetly) on the fly.

    • As well as use implicitly loading the latest version of module where the version is unspecified, and the specified version when requested, the ability to hold debug and/or development versions within the same libtree and request they be loaded would be handy.
    • A tool for taking a development version directly from the libtree and packaging it for distribution would be a nice convenience.
    • An tool equivalent to Module::Starter that created a development skeleton in-situ would complete the toolset.
    • Random thought. I wonder if there is any scope/merit in adding svn source control into the mix?

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
      A few responses I already have the answers to:
      • As well as use implicitly loading the latest version of module where the version is unspecified, and the specified version when requested, the ability to hold debug and/or development versions within the same libtree and request they be loaded would be handy.

        According to S11, there are three items that determine what gets loaded:

        1. Module name
        2. Module version range
        3. Module "authorizing authority URI" (usually author).
        So, you can have the C6PAN version and the debug version and it will "Just Work".
      • An tool equivalent to Module::Starter that created a development skeleton in-situ would complete the toolset.

        The distro builder would have this action. There's no reason why it should be separated.

      • Random thought. I wonder if there is any scope/merit in adding svn source control into the mix?

        I don't think so. For one thing, SourceForge is still CVS. Another is that some prefer svk over svn and (as I discoverd on feather) they're not identical.

      • Parrot loading bytecode

        This seems to indicate that Parrot is the thing we're installing modules for, not Perl6. So, we would have a module installer for each VM Perl6 targets? We have to reinstall eveyr time Parrot is upgraded? I'm not sure I like that ...

        Maybe, instead, we should be looking at caching the bytecode from the module and it's used unless either Perl6 or Parrot or the module has changed? That would seem to be something for "use"/"require" to do ...


      • In general, if you think something isn't in Perl, try it out, because it usually is. :-)
      • "What is the sound of Perl? Is it not the sound of a wall that people have stopped banging their heads against?"
        So, you can have the C6PAN version and the debug version and it will "Just Work".

        So, just make the debug version's author 'debug' or similar. Makes sense.

        I don't think so. For one thing, SourceForge is still CVS. Another is that some prefer svk over svn

        I guess I should have just said source control (omitting the 'svn'), but most everyone is seems to be settling upon svn recently. It would be kind of nice to have CP6AN be the master repository and then authors could just branch locally for the next version and commit to CP6AN when they're ready. Also, svn/svk will talk to each others repositories won't they? At least for the 'co' & 'up' commands?

        And the download tool could use the intelligent diffing to update installed packages rather than downloading the whole thing. And just checkout new modules? Not sure if that could work, but the thought crossed my mind so I mention it.

        I'm not sure where the sourceforge stuff comes in here? Is CP6AN likely to be on SF?

        This seems to indicate that Parrot is the thing we're installing modules for, not Perl6.

        I know that Parrot is intended to support multiple languages but I don't recall reading that Perl6 was intending to target multiple VMs?

        In any case, when the builder runs the test suite, it could (optionally) issue a

        perl6 --cache-bytecode -MNew::Module

        as a final action if the test passed. Or something...

        I agree that it is probably a function of use/require to actually load the bytecode, but I thing the Perl6 executable could be asked to load/save/igonore cahced bytecode and the builder module could support thiose options on a case-by-case basis as well as it being a configurable option for an installation?

        Anyway. Just thoughts for the pot.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
        "Science is about questioning the status quo. Questioning authority".
        The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
Re: Perl6 - Distribution builder/installer
by salva (Monsignor) on May 30, 2005 at 08:41 UTC
    modules should be easy to repack as deb, rpm, etc.
Re: Perl6 - Distribution builder/installer
by adrianh (Chancellor) on May 30, 2005 at 10:39 UTC
    Comments on anything and everything I've posted so far.

    Random comments:

    • I don't think separating out the module builder and installer will be a big issue. There's gonna be a whole lot of common code between the two so you'll probably end up with them in the same distribution anyway..
    • I wouldn't worry about C6PAN/CPANPLUS. These should be independent of the module builder/installler. People are always going to want to build/install modules outside of the context of CPAN. The only issue I would have is to make sure that there is a way to get at your META.yml equivalent without having to execute code.
    • Might be worth looking at Ruby Gems (one of Ruby's solution to this problem). The chapter on Gems from the Pickaxe book is available for download if you want a overview. There are some interesting design decisions.
    • Dig out the Module::Build mailing list archives :-)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://461627]
Approved by Zaxo
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (9)
As of 2014-09-01 12:07 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite cookbook is:










    Results (6 votes), past polls