Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

{Perl6} Macros and native compilation

by spurperl (Priest)
on Apr 10, 2005 at 17:28 UTC ( #446430=perlquestion: print w/ replies, xml ) Need Help??
spurperl has asked for the wisdom of the Perl Monks concerning the following question:

As time goes by I keep hearing things about Perl 6 that really incite my imagination. For example:

  • Macros - are the future macros of Perl 6 anything like Lisp/Scheme macros (very powerful) ? Like C macros (extremely weak) ? Something in between ?
  • Native compilation - all Perl 6 programs will compile to Parrot, which in turn can compile to native machine code on popular platforms. At last ! Has any steps in this direction been made ? How large (approximately) is the Parrot runtime that serves as the compulsory "rock on the neck" of every compiled Perl 6 script ?

Comment on {Perl6} Macros and native compilation
Re: {Perl6} Macros and native compilation
by chip (Curate) on Apr 10, 2005 at 18:51 UTC
    Parrot is a virtual machine. There's no need to go further in compiling down to native code.

        -- Chip Salzenberg, Free-Floating Agent of Chaos

      So all those Java JITs are just a big waste of time? If the virtual machine imposes no overhead then I guess you'd be right but that seems rather unlikely.

      HTML::Template is a VM is you squint and tilt your head but that doesn't make HTML::Template::JIT useless.

      -sam

        Err, actually, Parrot has this massively sane, cross platform, JIT system. It's just that we as compiler writers need not be aware of it.
      What you say is correct, but it's not enough for some cases.

      For example, at work we all have Windows machines and most people don't have Perl installed. From time to time, I create small Perl applications that could be useful for others, and it's a big problem to move them to other computers.

      Even if Perl is installed, all the modules the program need often are not, and some computers are not connected to the network so it's not easy to distribute.

      What I use is PAR, but it creates large executables (3 MB and north) and often induces a runtime penalty (even if it's only on startup - some apps are so small that a 20 second delay is unbearable)

      Compiling Perl to native code would be a bliss for me. I'd wish to work in a networked Unix environment (like at my other job), with Perl installed everywhere and CPAN distribution simple, but sometimes reality hurts.

      In fact this is the only downside of Perl for me, and the reason I still sometimes create pure-C++ (with Win32 API for GUI) apps - complete programs with GUIs that run autonomously on any Windows box disregarding of its application base and take only 100K of disk/memory are hard to beat.

        Your use case is an important one, and I intend for Parrot to be easily bundled with Parrot applications -- preferably with a smaller total size than that currently necessary with Perl 5.

            -- Chip Salzenberg, Free-Floating Agent of Chaos

        What I use is PAR, but it creates large executables (3 MB and north) and often induces a runtime penalty (even if it's only on startup - some apps are so small that a 20 second delay is unbearable)

        Compiling Perl to native code would be a bliss for me. I'd wish to work in a networked Unix environment (like at my other job), with Perl installed everywhere and CPAN distribution simple, but sometimes reality hurts.

        Most of that delay will likely disappear with Parrot without the need for compilation to native code. Most of that startup delay will be loading and compiling the Perl 5 code. With Perl 6/Parrot you'll be able to just ship the Parrot bytecode with the VM and save the compilation steps.

        What I use is PAR, but it creates large executables (3 MB and north) and often induces a runtime penalty (even if it's only on startup - some apps are so small that a 20 second delay is unbearable)

        Such long PAR startup penalty only occurs the very first time you run a (PARred) program (unless you have used the -C or --clean switch when creating the package), because PAR must extract all the files it needs to run your program, including perl itself, the modules used by your program and anything else you've packaged (the files are extracted into your system temporary directory).
        The next time you run your program, it starts up much faster (provided that these cached files have not been deleted of course), since PAR first looks for these previously cached files, and if it finds them it cleverly avoids to extract them again.

        Ciao,
        Emanuele.

      chip,
      Given your recent acceptance of the Parrot design chief hat, this answer worries me a bit. The question being asked was along the lines of has any work been done towards native executables, to which the answer I believe is "yes" having followed the project for some time now. The answer you gave leads me to believe that it is no longer a goal to support native executables on popular platforms. On the other hand, you might have just meant that while it is an option, it is not a necessary one.

      Can you please clarify?

      Cheers - L~R

        Well, I don't want to discourage anyone from working on native code export if they have good reason to believe it's worthwhile. (This is not to be confused with the JIT, which is native code as an in-memory optimization.) But as far as I can tell, it isn't worthwhile. Some good use cases might change my mind, of course.

        BTW, for those of you who want to create packaged binaries: I suggest that the best route will be to package the parrot executable along with your pbc.

            -- Chip Salzenberg, Free-Floating Agent of Chaos

      Erm... we already can do that. The JIT code has been leveraged to emit native object files, and there's a bytecode->C translator that generates compilable C. (That's been in there and working, on and off, since the first few weeks of the project)

      There's definitely a performance win to doing it, though how much depends on how much of the code uses the low-level types instead of PMCs, since the C optimizer has more to work with there.

Re: {Perl6} Macros and native compilation
by adamk (Chaplain) on Apr 11, 2005 at 00:20 UTC
    ... and more specifically, think for a moment on the implications of needing to support "eval" in a natively-compiled Perl program.

    Answer: Even if you do natively compile, you need the compiler/VM there anyway, to handle the evals, so why not just keep the bulk of the code as Parrot.
      Even if you do natively compile, you need the compiler/VM there anyway, to handle the evals, so why not just keep the bulk of the code as Parrot

      Because going all the way down to native code can give you a fair speed increase. You can get a very impressive turn of speed out of modern Lisp compilers.

        In a new twist on an old favorite: "Benchmarks welcome." I'm especially interested to know what a full-on native code compiler can do for you that a JIT can't.

            -- Chip Salzenberg, Free-Floating Agent of Chaos

Re: {Perl6} Macros and native compilation
by rg0now (Chaplain) on Apr 11, 2005 at 10:19 UTC
    Since I was just into A6 when I read your question, I wanted to write up a nice tutorial on Perl 6 macros for you. But I ended up just repeating the original text, so I decided to copy-paste here the relevant parts instead:
    Macros

    A macro is a function that is called immediately upon completion of the parsing of its arguments. Macros must be defined before they are used--there are no forward declarations of macros, and while a macro's name may be installed in either a package or a lexical scope, its syntactic effect can only be lexical, from the point of declaration (or importation) to the end of the current lexical scope.

    Every macro is associated (implicitly or explicitly) with a particular grammar rule that parses and reduces the arguments to the macro. The formal parameters of a macro are special in that they must be derived somehow from the results of that associated grammar rule. We treat macros as if they were methods on the parse object returned by the grammar rule, so the first argument is passed as if it were an invocant, and it is always bound to the current parse tree object, known as $0 in Apocalypse 5. (A macro is not a true method of that class, however, because its name is in your scope, not the class's.)

    Update: That's now the $/ object. $0 has been "demoted" to being the entire matched string.

    ...

    A macro can do anything it likes with the parse tree, but the return value is treated specially by the parser. You can return one of several kinds of values:

    • A parse tree (the same one, a modified one, or a synthetic one) to be passed up to the outer grammar rule that was doing pattern matching when we hit the macro.
    • A closure functioning as a generic routine that is to be immediately inlined, treating the closure as a template. Within the template, any variable referring back to one of the macro's parse parameters will interpolate that parameter's value at that point in the template. (It will be interpolated as a parse tree, a string, or a number depending on the declaration of the parameter.) Any variable not referring back to a parameter is left alone, so that your template can declare its own lexical variables, or refer to a package variable.
    • A string, to be shoved into the input stream and reparsed at the point the macro was found, starting in exactly the same grammar state we were before the macro. This is slightly different from returning the same string parsed into a parse tree, because a parse tree must represent a complete construct at some level, while the string could introduce a construct without terminating it. This is the most dangerous kind of return value, and the least likely to produce coherent error messages with decent line numbers for the end user. But it's also very powerful. Hee, hee.
    • An undef, indicating that the macro is only used for its side effects. Such a macro would be one way of introducing an alternate commenting mechanism, for instance. I suppose returning "" has the same effect, though.

    A macro by default parses any subsequent text using whatever macro rule is currently in effect. Generally this will be the standard Perl::macro rule, which parses subsequent arguments as a list operator would--that is, as a comma-separated list with the same policy on using or omitting parentheses as any other list operator. This default may be overridden with the "is parsed" trait.

    ...

    If there is no signature at all, macro defaults to using the null rule, meaning it looks for no argument at all. You can use it for simple word substitutions where no argument processing is needed. Instead of the long-winded:

    my macro this () is parsed(/<null>/) { "self" }
    you can just quietly turn your program into C++:
    my macro this { "self" }
    A lot of Perl is fun, and macros are fun, but in general, you should never use a macro just for the fun of it. It's far too easy to poke someone's eye out with a macro.

    Now, it's up to you to decide, how this compares to LISP et co.

    rg0now

      It sounds really good, just one thing is unclear...

      When the macro body is a string that is "shoved back into the input and reparsed", can I ask which variables to evaluate and which not ? I.e.

      my macro this {"self{$foo}"}

      Will $foo be interpolated/evaluated at compile-time, when the macro is expanded, or at runtime, in the context of the macro's usage ? In Lisp, using special syntax you can have both, which is very, *very* powerful.

        In my understanding, you absolutely can do this with Perl 6 macros. If you want $foo to be avaluated at compile time, you just return a string as you did. Contrariwise, if you return a closure (maybe using the simple my macro this { self{$foo} } syntax?), you get "a generic routine that is to be immediately inlined, treating the closure as a template", such that "any variable not referring back to a parameter is left alone, so that your template can declare its own lexical variables, or refer to a package variable". This way, $foo will be evaluated at runtime...

        But, as always with Perl 6, don't take my words for granted...:-)

        rg0now

        I'd say you'd have to use single quotes, e.g.:

        my macro this { 'self{$foo}' }

        I don't think rg0now's idea will work, because $foo would refer to some outer $foo. If my understanding is correct, you can do things like the following:

        my $compilation_time = BEGIN { time }; macro uptime { time - $compilation_time } # And then... say "$*PROGRAM_NAME has been running for {uptime} seconds.";

        (Update: Doing s/macro/sub/ here would work just fine, too.)

        But, as rg0now said, that may be wrong, of course.

        --Ingo

Re: {Perl6} Macros and native compilation
by adrianh (Chancellor) on Apr 11, 2005 at 13:30 UTC
    Macros - are the future macros of Perl 6 anything like Lisp/Scheme macros (very powerful) ? Like C macros (extremely weak) ? Something in between ?

    They're like Lisp macros - not the crippled things that C has.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://446430]
Approved by Joost
Front-paged by BrowserUk
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (11)
As of 2014-09-02 08:45 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite cookbook is:










    Results (20 votes), past polls