Removing javascript comments

[Anonymous Monk]

Hi monks, I have a really quick question... I'm a Perl newbie... just started a couple days ago: :- Here is my question: How would I remove all comment strings (double slashes and comment text) in an arbitrary javascript function similar to the below comment without removing things like http://... strings? // comment string I know it would be something like $fileString=~s/ not sure what the pattern replace would be after the s/ Thanks very much for any assistance you may provide. John

[Joost]

this is actually fairly complex to do correctly, since javascript has 2 different comment delimiters, and 'comment-like' constructs in string and regex literals (which are especially tricky to recognize) should not be mistaken for comments. You may want to use a full javascript grammar/parser instead of starting from scratch.

If you feel like experimenting, take a look at JE::Parser. Probably not for newbies, though...

Otherwise, you can take advantage of a few facts:

javascript string and regex literals are always single line. This means if you go trough the code line by line, ignoring string and regex literals, you can safely assume** any /* .. */ and // ... constructs left are comments (javascript does not allow for an empty regex literal). As I mentioned before, regex literals will be tricky, since the JS grammar for where regexes are valid (instead of divide operators) are fairly tricky, IOW, you will need to distinguish between

 var res = a / 4 /* whatever */
                 ^^^^^^^^^^^^^^ comment

and

 var res = /bla \/*  .*/.exec("stuff");
                 ^^^^^^^ not a comment

for instance.

** in fact you can't, but it's /probably/ good enough.

update: fixed js code.

[renodino]

You might cheat from Dean Edward's packer implemented in Perl (and several other languages). It does a great deal more than just strip comments, but you can always run the output back thru a beautifier to reconstruct code layout.

Otherwise, I'd suggest looking into Text::Balanced, since parsing JS has many of the same gotchas as parsing Perl, notably, regex syntax that can contain stuff that looks like comments.

[tachyon-II]

As noted to do this properly you really need an HTML parser to extract the javascript and then a javascript parser to parse the javascript. A regex solution will never be 100% reliable BUT that said it is a great way to learn about regexes as you get to write a regex to do a task and then find out it does unexpected stuff!

perlre is the reference to regexes. Although s/this/that/ is the usual format ie using / to delimit the regex you can in fact use just about anything. When you have / symbols to match you either use a different delimiter of you have to escape the / symbols in the regex with a \ so \/ means match /. Compare the readability of:

$str =~ s/http:\/\///g;

# and

$str =~ s|http://||g;
$str =~ s#http://##g;
$str =~ s!http://!!g;

The first example is a bit harder to read due to the escapes. At least it is to me. Unfortunately the * char is a regex wildcard so you will still need to escape that. To get you started try:

$str =~ s!(/\*[^/]+\*/)!>>>> \1 <<<<!g;  # /* comments */
$str =~ s!(\s//[^/\n]+)!>>>> \1 <<<<!g;  # // comments
print $str;

There will be plenty of edge cases not handled by this code but you can have a great learning experience trying to fix them without breaking other things! The () in the match section just allow you to explicity see what has happened as we capture the match and do some funky ascii highlighting in the replace section. To strip you just replace with nothing and don't need to capture but first you need to see what is happening. For debugging you may like to use:

$str =~ s!(fancy re here)! print "$1\n"; "" !ge;

That way when you run the code on a file you get the comments stripped but also printed to STDOUT so you can watch and make sure all the stuff that is going looks like comments!

Welcome to Perl, hope you have fun and it helps you to get the job done.

[Anonymous Monk]

Thank you all very much for your valuable feedback... I almost have my code finished with the exception of the removal of javascript comments, so I will try out all your suggestions and see what happens... the best way to learn of course is to dabble in different methodologies to try and find the optimum solution for the given problem... looking forward to getting in to some cool Perl projects in the future... I'll post again once I have the solution in place... thanks again!

[Anonymous Monk]

Hi, I have a similar task and was wondering if you were able to accomplish this?

[deibyz]

Maybe something in Regexp::Common, like Regexp::Common::comment could help? You can use ECMAScript comment parsing.

[cparker]

John Crockford's JSMin removes comments as part of the minifcation process. There's a Perl implementation of JSMin at CPAN. You could study how this implementation removes comments and try to duplicate it. (Of course, if your license is compatible with Perl's license, you could just lift the relevant portions from the module, but then you wouldn't be learning anything. :)