CUFP
gmax
<p>Syntax highlighting has always been one of my favorite features in every
editor I have used. After becoming addicted to it, I have always regretted
not being able to share colored code with other people.</p>
<p>If we talk about Perl code alone, there are a few available tools to
create colored code in HTML and publish it on the web.</p>
<p>When you need to show code in more than one language, though, things become
more difficult. Think about presenting the installation and customization
of a complex system. You'd need to show Perl code, Apache configuration files,
HTML code, SQL queries, and perhaps some XML. </p>
<p>When you publish this code on the web, what was clearly highlighted and easily
understandable in your editor screen becomes a flat sequence of black on white text.</p>
<h3>Introducing Text::VimColor</h3>
<p>Two wonderful features of <a href="http://www.vim.org">Vim</a> are its ability
of highlighting different languages (384 as of today) and producing an HTML page
with the same layout of the code on screen.</p>
<p>(And don't forget Vim's ability to highlight nested syntax, such as [id://270014|Perl embedded in HTML] or <a href="http://jeffa.perlmonk.org/vi/example.html">SQL embedded in Perl</a>.)</p>
<p>Producing code manually with vim is not user friendly and it is quite slow.
If you need to publish code on a regular basis, producing HTML pages manually from
Vim is a hassle.</p>
<p>Enter Geoff Richards' [http://search.cpan.org/perldoc?Text::VimColor|Text::VimColor], a module
that removes your need to remember difficult commands and to cut and paste your
code snippets.</p>
<readmore>
<p>Given these code fragments:</p>
<code>
# CodeSamples.pm
package CodeSamples;
our $ctext = <<'CTEXT';
#include <stdio.h>
int main()
{
printf("hello world\n");
return 0;
}
CTEXT
our $perltext = <<'PTEXT';
my $query = qq{SELECT mycol, COUNT(*)
FROM mytable WHERE mycol <= 10 GROUP BY mycol};
print $$_,$/ for @{ $dbh->selectcol_arrayref($query) };
# Notice that $query has nested SQL syntax
PTEXT
1;
</code>
<p>This script will produce nicely highlighted code (HTML + CSS).</p>
<code>
#!/usr/bin/perl -w
# test_vimcolor.pl
use strict;
use CGI qw/:standard/;
use Text::VimColor;
use CodeSamples; # contains code samples in C and Perl
my $csyntax = Text::VimColor->new(
string => $CodeSamples::ctext,
filetype => 'c'
)or die("can't create C object ($!)\n");
my $perlsyntax = Text::VimColor->new(
string => $CodeSamples::perltext,
filetype => 'perl'
)or die("can't create perl object ($!)\n");
my $fperlsyntax = Text::VimColor->new(
file => $0,
filetype => 'perl'
)or die("can't create perl object ($!)\n");
print
start_html(-title=>"Text::VimColor test",
-style=>{'src'=>'light.css'}
),
h2("C"),
pre( $csyntax->html),
hr,
h2("Perl"),
pre( $perlsyntax->html),
hr,
h2("Perl (file)"),
pre( $fperlsyntax->html),
hr,
h2("CSS"),
pre(Text::VimColor->new(file =>'light.css', filetype => 'css')->html),
hr,
h2("Perl (package)"),
pre(Text::VimColor->new(file =>'CodeSamples.pm', filetype => 'perl')->html),
hr,
h2("Perl (another package)"),
pre(Text::VimColor->new(file =>'VimColorCache.pm', filetype => 'perl')->html);
</code>
<p>See <a href="http://gmax.oltrelinux.com/test_vimcolor.html">the colorful result</a>.</p>
</readmore>
<h3>Highlighting systems overview</h3>
<p>Before continuing, let me show the alternatives. I have tried all of them, and
I have good and bad feelings for each one of them. I am currently in favor of
[CPAN://Text::VimColor] for the reason given before, and more.</p>
<table border="1" cellpadding="3" cellspacing="0">
<tr> <th>Application</th> <th>Pro</th> <th>Con</th> </tr>
<tr> <td>[http://search.cpan.org/search?query=perltidy&mode=all|perltidy]</td> <td>Fast and accurate</td> <td> Only Perl</td> </tr>
<tr> <td><a href="http://www.gnu.org/software/src-highlite/source-highlight.html">GNU source highlight</a></td> <td>Very fast</td> <td>Hard to customize. <br>Only a few languages</td> </tr>
<tr> <td>[CPAN://Syntax::Highlight::Perl]</td> <td>Fast and customizable</td> <td>Only Perl</td> </tr>
<tr> <td>[CPAN://Text::VimColor]</td> <td>All languages.<br> Easily customizable</td> <td>Slower than other modules.<br> Works only on Unix (as of today)</td> </tr>
</table>
<h3>Improving performance</h3>
<p>As I said, [CPAN://Text::VimColor] main deficiency is its poor performance compared to other
modules. Although the latest version (0.07) is twice as fast as the previous one, it is still
way too slow for any sensible web usage.</p>
<p>Therefore, I decided to create a caching object, to improve Text::VimColor basic performance.</p>
<readmore>
<p>The simplest way I could think of was a tied hash with [CPAN://DB_File]. I have also simplified
the object interface, to make it easier to use.</p>
<code>
package VimColorCache;
use strict;
use warnings;
use Text::VimColor;
use Digest::MD5 qw/md5_hex/;
use DB_File;
our $VERSION = '0.1';
sub new {
my $class = shift;
my $filename = shift || 'VimColorCache.db';
my %code_items;
tie %code_items, 'DB_File', $filename
or return undef;
my $self = bless {
code_items => \%code_items
}, $class;
return $self;
}
sub _get_text {
my $filename = shift;
my $text = undef;
open IN, $filename or return undef;
local $/;
$text = <IN>;
close IN;
return $text;
}
sub draw {
my $self = shift;
my $text = shift; # either the code or the file name
my $input = shift; # file or string
my $syntax_type = shift; # syntax type (perl, c, sql, html, xml, etc)
my $output = shift; # output mode
return undef unless $output =~ /^(?:html|xml)$/;
return undef unless $input =~ /^(?:file|string)$/;
my $code = $text;
if ($input eq 'file') {
$code = _get_text($text) or return undef;
}
$code =~ s/\t/ /g; # turns tabs into 4 spaces
my $signature = md5_hex($code);
if (exists $self->{code_items}->{$output.$signature}) {
return $self->{code_items}->{$output.$signature}
}
else {
my $syntax = Text::VimColor->new (
$input => $text,
filetype => $syntax_type
) or return $code;
my $out = $syntax->$output;
$self->{code_items}->{$output.$signature} = $out;
return $out;
}
}
sub remove {
my $self = shift;
my $text = shift; # either the code or the file name
my $input = shift; # file or string
my $output = shift;
my $code = $text;
if ($input eq 'file') {
$code = _get_text($text) or return undef;
}
my $signature = md5_hex($code);
delete $self->{code_items}->{$output.$signature};
}
1;
__END__
=head1 NAME
VimColorCache - caches the result of Text::VimColor
=head1 SYNOPSIS
use VimColorCache;
my $filename = 'syntax.db';
my $vcc = VimColorCache->new($filename);
print $vcc->draw('print $_,$/ unless m/^\s*$/g',
'string', 'perl', 'html');
print $vcc->draw('hello.c', 'file', 'c', 'html';
=head1 class methods
=over 4
=item new()
The constructor accepts an optional filename where to store previously
highlighted code snippets.
The default file name is VimColorCache.db
=item draw()
Returns a properly highlighted text.
It needs some parameters:
- text either a filename or a string containing
code to be formatted
- input 'file' or 'string'
- syntax_type language to use (perl, c, php, xml, SQL)
The same as Vim's 'filetype'
- output either 'html' or 'xml'
If the code passed as string or file has already been processed, then
the corresponding formatted text is returned, otherwise a Text::VimColor
object is created and the highlighted syntax is processed from scratch.
print $vcc->draw('hello.c','file', 'c', 'html');
open IN, "hello.c" or die "can't open\n";
my $c_text = do { local $/; <IN> };
close IN;
print $vcc->draw($c_text,'string', 'c', 'html');
These two instructions will print the same output.
Only, the first one will be slow, the second one will be
extremely fast.
=item remove()
Removes an item from the repository.
It needs the same parameters as draw(), except "syntax_type"
$vcc->remove('hello.c','file', 'html');
=back
=head1 AUTHOR
Giuseppe Maxia, a.k.a. gmax
(gmax_at_cpan.org)
=head1 COPYRIGHT
Same as Perl itself.
=cut
</code>
<p>VimColorCache is a layer between the application and the highlighting module.
It works on the assumption that, in most cases, code is published once and shown
many times. Sometimes it is modified, but mostly it is just published and then left
on the page for public consumption. In this scenario, the first poster has to
wait one or two seconds for the highlighting engine to do its job, but every
further request of the same code is resolved instantly.</p>
<p>The first example shown in this node could be rewritten using VimColorCache
as follows:</p>
<code>
#!/usr/bin/perl -w
# test_vimcolor_cache.pl
use strict;
use CGI qw/:standard/;
use VimColorCache;
use CodeSamples;
my $vcc = VimColorCache->new or die("can't create object ($!)\n");
print
start_html(-title=>"VimColorcache test",
-style=>{'src'=>'light.css'}
),
h2("C"),
pre( $vcc->draw($CodeSamples::ctext, 'string', 'c', 'html')),
hr,
h2("Perl"),
pre($vcc->draw($CodeSamples::perltext, 'string', 'perl', 'html')),
hr,
h2("Perl (file)"),
pre($vcc->draw($0, 'file', 'perl', 'html')),
hr,
h2("CSS"),
pre($vcc->draw('light.css', 'file', 'css', 'html')),
hr,
h2("Perl (package)"),
pre($vcc->draw('CodeSamples.pm', 'file', 'perl', 'html')),
hr,
h2("Perl (another package)"),
pre($vcc->draw('VimColorCache.pm', 'file', 'perl', 'html'));
</code>
<p>And <a href="http://gmax.oltrelinux.com/test_vimcolor_cache.html">the second colorful result</a>
shows exactly the same output as the previous one, except for the page title.</p>
<p><b>update (1)</b><br>
<u>CAVEAT</u>. If you pass a file to [CPAN://Text::VimColor] and at the same time you are editing the same file with Vim, it will return an error. You should either ensure that your file is not currently in use by Vim before passing it to the class constructor, or slurp it into a scalar and pass it as a string.
</p>
<p><b>Update (2)</b><br>
In case you are wondering just how slow is Text::VimColor without a cache, here is an example.<br>
Processing times are acceptable for small scripts, but become unbearable for large ones.</p>
<table border=1 cellpadding=3 cellspacing=0>
<tr>
<th rowspan="2">Application</th>
<th colspan="4">Time to highlight</th>
</tr>
<tr>
<th>strict.pm<br> (2.6 KB)</th>
<th>Benchmark.pm<br> (22 KB)</th>
<th>CGI.pm<br> (221 KB)</th>
<th>DBI.pm<br> (226 KB)</th>
</tr>
<tr>
<td>perltidy</td>
<td align="right">0.30</td>
<td align="right">0.86</td>
<td align="right">1.76</td>
<td align="right">2.77</td>
</tr>
<tr>
<td>source-highlight</td>
<td align="right">0.00</td>
<td align="right">0.03</td>
<td align="right">0.19</td>
<td align="right">0.21</td>
</tr>
<tr>
<td>Text::VimColor</td>
<td align="right">0.34</td>
<td align="right">2.35</td>
<td align="right">16.62</td>
<td align="right">17.23</td>
</tr>
<tr>
<td>VimColorCache</td>
<td align="right">0.01</td>
<td align="right">0.01</td>
<td align="right">0.01</td>
<td align="right">0.02</td>
</tr>
</table>
</readmore>
<p>Enjoy!</p>
<div class="pmsig"><div class="pmsig-127116">
<pre>
_ _ _ _
(_|| | |(_|><
_|
</pre>
</div></div>