Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked

Writing a Filter::Simple -based filter for .vim

by Intrepid (Deacon)
on Aug 13, 2003 at 21:31 UTC ( #283697=CUFP: print w/ replies, xml ) Need Help??

Hello, Good Monks,

I find myself wanting to get advice on some preliminary code that employs the techniques described in Filter::Simple's documentation. Although the documentation is probably very good, I am finding myself having trouble mentally parsing it, and so I think I am probably working harder / dumber than I ought to be.

The goal here is to provide a devel support framework or "scaffolding" for writing the kind of Perl code that doesn't look like Perl code to the stand-alone perl parser, unaided: Vim scripts *.vim employing the embedded perl interpreter that can be compiled (as an option) into the Vim editor. An example fragment of such a script appears below.

""; use VimFilter; func! VerDataFmt() perl <<EOSCRIPT #NOVIM# use strict; #NOVIM# use EasyOpen; #NOVIM# my @Dummy = @{EasyOpen::SlurpIn ('/tmp/gvim-ver-output.tmp')}; my($regQ,$VregS) = VIM::Eval('@v'); my($reqR,$IreqR) = VIM::Eval('columns'); $IreqR =~/\d{2,3}/ || ($IreqR = 96); if($regQ) { my $ln = my $mlen = 0; my(@plus,@minu,@cummup,@cummum); my @lwise = split "\n",$VregS; VIM::Msg('There are '.@lwise." lines in \@v\n"); foreach my $lines (@lwise) { next unless $ln++>=2; if($lines =~/^(?:\-|\+)\w+/) { my @bopts = split /(?<=\w)\b/, $lines; VIM::Msg("Ele in picked-out: ".@bopts."\nin line:$lines"); push @plus, grep{ s/^\+// and ($mlen=$mlen<length($_) ? length($_):$mlen +) }@bopts; push @minu, grep{ s/^\-// and ($mlen=$mlen<length($_) ? length($_):$mlen +) }@bopts; } last if $lines =~/^\s+system/; } VIM::Msg("The size of arrays \@plus and \@minu is " . @plus ." , ". @minu ."\n"); $curbuf->Set(0, "Features compiled into this Vim:") ; my $slotspln = sprintf '%u', ($IreqR / (2+$mlen)) - 0.5; my $formatex = "%$mlen-s "x$slotspln; my$PY = my$YP = 0; while ( $PY < @plus ) { local $^W = 0; push @cummup, sprintf($formatex, @plus[$PY .. ($PY+=($slotspln - 1))]); } continue { $PY++; } while ( $YP < @minu ) { local $^W = 0; push @cummum, sprintf($formatex, @minu[$YP .. ($YP+=($slotspln - 1))]); } continue { $YP++; } $curbuf->Append(1,@cummup,''); $curbuf->Append(2+@cummup,'Features omitted from this Vim:'); $curbuf->Append(3+@cummup,@cummum,''); } else { VIM::Msg('The contents of register v could not be accessed'); } __DATA__ EOSCRIPT endfu

Note about code above:

The module and the EasyOpen module are my inventions, the former is shown below and the latter is just a trivial little piece of code that's not worth showing here.


Those unfamiliar with the experience of writing Perl script as part of Vim subroutines (functions) won't necessarily know, but there's a challenge here that is different from that encountered in writing ordinary Perl programs. In an average Perl script the system perl interpreter is invoked directly and except in the case of GUI programs, has access to STDOUT and STDERR for purposes of informing the user of what might have gone wrong during some stage of parsing, compilation or execution.

In the case of the perl interpreter embedded into GUI Vim (GVIM), there is typically no output at all if the interpreter encounters a syntax error. Program execution simply terminates silently in such a case, offering not a clue about what might have gone wrong. As can be seen, this can make the write-run-diagnose, write-run-diagnose cycle much more laborious if GVIM is the only environment the script can be tested in during development. Essentially it's like having to shoot an arrow into the bull's eye each time, but if the bull's eye is missed there's no peripheral targets zones that count as "partial points" for hitting in. In my opinion this factor alone has made developing Vim scripts using Perl sufficiently hard so that not that many exist.

As a side note it should be recognized that Vim has it's own internal scripting language, and it's not too bad at all, actually pretty capable, but in my experience rather difficult to learn. I already know perl and would much rather program using perl than vim (or "ex" as Vim's command language is called).

The thing I found I wanted to do was just to be able to concentrate on write-test-write 'ing the script using the normal stand-alone perl to execute, instead of the one inside Vim. Then when the syntax of the perl part of the work checks out as sound, I run it as a Vim function by source'ing the script and :call'ing the function.

Finally, we get to the work that supports developing a perl-Vim function this way. The module gets used only if we are running in the ordinary perl. This module transforms the source code of the Vim scriptlet so that it will parse under the ordinary (non-embedded) perl interpreter. This module is what I ask advice on from the Monks. First of all, I'd like to be able to have more than one perl program in a single Vim script. Secondly, I am opening the file manually because I am not sure I can get the @Dummy text extracted later. Thirdly, I have an awkward mess of regexen going on in the sub{ paramater.

The @Dummy is a crutch required because inside the embedded interpreter environment, the $curbuf and other objects are automatically provided, whereas they are not present outside it, of course. The price paid by the scripter using this my system, is that they'll need to provide a simulation in static form of the dynamic object, so that their code can really be tested outside Vim. At the very least this array must be defined, even if empty, to prevent syntax errors under use strict;.

package VimFilter; use FindBin qw($RealBin $RealScript); use Carp; BEGIN { $|++; #-------------------------------------------------------------------- # some code d/l from Perl Monks node: 106194 # View Original: +playcode #-------------------------------------------------------------------- use Term::Cap; use POSIX; my $termios = new POSIX::Termios; $termios->getattr; my $ospeed = $termios->getospeed; my $t = Tgetent Term::Cap { TERM => undef, OSPEED => $ospeed }; ($NO, $BO, $US, $SO) = map { $t->Tputs($_,1) } qw/me md us so/; # ended borrowed code. $BO = $US; our ($Supplied_dummy, $VFD); open US , "< $RealBin/$RealScript" or croak "Cannot open() \"$RealScript\" in \"$RealBin\", $!"; local $/; undef $/; my $whole = <US>; close US or die "Went wrong close() ing, $!"; print STDERR "Magnitude scalar of file is ",length $whole, "\n"; if( $whole =~m/^(?:#NOVIM#)?\s*my\s*\(?\s*\@(\w*Dummy\w*)\s*\)?\s*=. ++?(?=;\n)/msi ) { $Supplied_dummy = $&; $VFD = $1; $Supplied_dummy =~s/^#NOVIM#//; } else { croak "\nCouldn't find a ${BO}\@Dummy${NO} declaration in your f +ile:\n", "${RealBin}/${RealScript}\n"; } print STDERR "We found your ${BO}\$Dummy${NO} declaration:", <<EOBLK + ; ---------------------------------------------------------------------- +----- ${SO}$Supplied_dummy${NO} ; Symbol name of dummy array: ${BO}$VFD${NO} ---------------------------------------------------------------------- +----- EOBLK } my $dummy = "${Supplied_dummy};\n"; my $dumEv = 'my $dummyEval = sub { (0,0) };' ; # prevent Perl from seeing Vim wrapper tokens, so that we can run/deb +ug outside Vim. use Filter::Simple sub { if(!eval 'VIM::Buffers > 0x0;') { s'#NOVIM#''g; s'\A\s*fun?c?t?i?o?n?!?\s+.*''m ; s[^\s*\Qperl <<\E.*] [$dumEv]m ; s'VIM::Msg'print STDERR 'mg ; s'VIM::Eval\s*\('&$dummyEval('mg ; s/\$ curbuf->(?:G|S)et\( ([^\)]+) \)/\$${VFD}[$1] /xg ; s/\$ curbuf->Count\(\) /scalar(\@${VFD})/xg ; s'\$ curbuf-> \w+ \(\) 'qq/DUMMY/'exg ; } } ; 1; =pod =head1 SYNOPSIS use VimFilter; =head1 NOTES The entire source file is slurped in as one long string. The sub{} is +not applied line-wise (looping through each line) but to the whole file, once. Trying to define a token to "ignore from here down" as suggestion in t +he docs for Filter::Simple didn't work. I probably didn't understand something abo +ut how to do that. The biggest challenge is to provide some kind of dummy array for the b +uilt-in automatic $curbuf object; we need to tailor that simulation to each application +of this module. A real refinement would be to treat each substitution as a replaceable + parameter, perhaps read in from a .conf file. One can even imagine a repository f +or those subregexen because the possible permutations are infinite! Using an external file + would be the only rational approach. =cut


use PerlMonk::Tye qw(:wisely);

Replies are listed 'Best First'.
Re: Writing a Filter::Simple -based filter for .vim
by bbfu (Curate) on Aug 15, 2003 at 19:19 UTC

    Thinking about it some more, I believe that you're actually trying to do two things here: 1) Emulate the interface provided by VIM, and 2) Filter out VIM code from the Perl code, so that you can run .vim file directly under perl. These two are separate, and should be made into two distinct modules.

    It should be fairly straight-forward to create a VIM::Emulate module. There's an overview of the VIM interface features you'd need to implement at :help perl-overview in VIM. The only one that I think wouldn't be feasable is the VIM::Eval() function, since you'd have to emulate every VIM command. If you felt ambitious, you could implement a "most commonly used" subset of the VIM commands, and detail it in the documentation for your module. If you do create this module, be sure to upload it to CPAN.

    Filtering out the non-Perl code is a little more tricky. I think, however, that you'd be better off not using a Filter::Simple-based solution, and instead leverage perl's -x command-line option. Simply add a she-bang line to the start of the Perl code, and a __END__ line to the end of it, and pass the -x option to perl. You won't be able to have multiple Perl functions in one .vim file but it would be a lot less work on your part. Also, one way around that limitation would be to have one big Perl block in your .vim file that declares a bunch of functions, and then declare your VIM functions separately that just call the appropriate Perl function, like so:

    perl <<EOP #!perl sub foo { VIM::Msg("Called foo"); } sub bar { VIM::Msg("Called bar"); } __END__ EOP func! TestFoo() " Note, you will need to pass along any " parameters your VIM function is called with. perl foo() endfu func! TestBar() perl bar() endfu

    Anyway, I that's how I would approach this problem. Let me know if you decide to start working on a VIM::Emulate module. I would be willing to assist, whenever I have time.

    Black flowers blossom
    Fearless on my breath

      Anyway, I that's how I would approach this problem. Let me know if you decide to start working on a VIM::Emulate module. I would be willing to assist, whenever I have time.

      Excellent! Thank you. I am quite short on time myself right now, so I'll have to keep this reply brief.

      One thing I'd like to mention is how great the insight you've provided is, regarding one issue. Your lattermost suggestions hinge on the understanding that Vim is creating a persistent perl interpreter that is lasting the lifetime of the Vim script: as far as I knew, it was not, and so it never occured to me to try to do this. I had thought that Vim creates an interpreter for each block of Perl code, it does its work and then expires. I was wrong!

      The example you posted above in fact proves out nicely, so thanks again for the very helpful insights. I'll be in touch!


      use PerlMonk::Tye qw(:wisely);

        Vim is creating a persistent perl interpreter that is lasting the lifetime of the Vim script

        Actually, the perl interpreter is lasting for the entire life of the VIM instance, as evidenced by the fact that you can declare Perl functions in one .vim script and call them in a different .vim script.

        This, of course, leads to the logical solution for the parsing-via-stand-alone-perl issue: move your functions into a module (or even just a required .pl script), and use the module in you .vim script. This would probably work out better in the long run than even in-lining Perl code in the .vim script and using the -x option, as you could then re-use the Perl code. Then, you'd need only create the VIM emulation module.

        Black flowers blossom
        Fearless on my breath

Re: Writing a Filter::Simple -based filter for .vim
by pimlott (Initiate) on Oct 11, 2004 at 06:19 UTC
    I just ran across this page and thought I should mention how I do vim extensions in perl.

    First, I put all the code in a module, which I load from the vim script. All of the mappings and functions I want to define are in the vim script, but they are just stubs for a perl functions, eg

    function! s:rowwise_dd() range perl VIM::CSV::rowwise_dd endfunction nnoremap <buffer> <LocalLeader><Left> :perl VIM::CSV::prev_col<CR>
    By putting only trivial perl in the vim script, I minimize difficult-to-debug problems that come from mixing perl and vim. Also, things like syntax highlighting work perfectly on the module.

    Second, I want the module to syntax-check outside of vim (the major pay-off of step one). This is a simple matter of duplicating the prototypes that vim implicitly sets up, eg

    sub VIM::DoCommand ($); sub VIM::Eval ($); sub VIM::Msg ($;$);
    I have not made any effort to make my module run outside of vim, as it is quite intimately tied to vim.

    Third (my favorite part), some syntactic sugar to make vim commands look like vim commands (this should be its own module, but I'm lazy):

    use Filter::Util::Call; BEGIN { filter_add( sub { my $status = filter_read; return $status unless /^\s*:/; s/"/\\"/g; s/:(.*)/VIM::DoCommand "$1";/; return $status; }) } ... :w

    By the way, the problem that you don't get any output from syntax errors is really a perl bug.

      Well done!

      If I could upvote you more than once I would, for replying to a year+ -old node as well as just having good tips to share. There're good ideas in your approach, and the only downside I can see is that it just slightly complicates the task of distributing your Vim code, since you now have to get modules installed in a correct manner, so that the perl embedded in Vim will find them. Not a huge problem but a bit more fuss than just placing a single plugin file.

      Thanks for the ideas!

          Soren A / somian / perlspinr / Intrepid

      Now, 2004: The 3 least meaningful terms in online jargon are:
        troll   flame   rant
      These used to mean something; but then they were highjacked by the kind of
      inferior intellects who, when faced with a more erudite opponent employing
      superior arguments (or simply hanging in there with a disagreeable
      contention), abuse these terms as merely another form of name-calling. ;-)
Re: Writing a Filter::Simple -based filter for .vim
by bbfu (Curate) on Aug 15, 2003 at 08:07 UTC

    My thoughts on your approach (though, I've never actually used any of the Filter:: modules, so I'm by no means an authority :) are that filters should generally be reserved for cases where the desired syntax is not valid Perl or there is some extreme change to the way the valid syntax would work that can't be accomplished any other way.

    The only things in your example .vim file are the function declaration line and the corresponding endfu (update: er, and the perl <<EOSCRIPT line pair). You ought to be able to accomplish everything else with normal Perl code. For example, you may well be better off simply installing a Msg function in the VIM:: namespace that prints to STDERR. You'd need to define a few functions, and export a few variables/objects, but all-in-all, I think it would come out cleaner and more straight-forward.

    Also, I think your #NOVIM# construct is reduntant, as you acomplish the same thing earlier in the file using "";. Since you'll have to do that at least once for the line that includes the filter module, you might as well comment it well and use it consistantly. (As a corollary to that, I think that you should probably move the use strict; et al. lines up with the use VimFilter; line, as they conceptually belong together anyway.)

    Then, I think that the way you're defining the @Dummy array is a bit convoluted. It seems like it would be easier to just pass that information to the VimFilter module as you use it. Then you shouldn't need to scan-and-parse the original source file looking for the declaration line. If I'm correct, this ought to obviate your entire convoluted BEGIN block.

    Lastly (and it's entirely a matter of preference), since your working with VIM functions anyway, it makes more conceptual sense to translate the func blah and endfu lines into subroutine definitions, instead of just deleting them. It should make no difference in the end, though. *shrug*

    Black flowers blossom
    Fearless on my breath

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: CUFP [id://283697]
Approved by broquaint
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (3)
As of 2016-06-26 22:42 GMT
Find Nodes?
    Voting Booth?
    My preferred method of making French fries (chips) is in a ...

    Results (331 votes). Check out past polls.