Beefy Boxes and Bandwidth Generously Provided by pair Networks httptech
Perl: the Markov chain saw
 
PerlMonks  

Programmatically Updating Code with PPI

by Tanktalus (Canon)
on Jan 04, 2010 at 21:55 UTC ( #815650=perlquestion: print w/ replies, xml ) Need Help??
Tanktalus has asked for the wisdom of the Perl Monks concerning the following question:

I've been toying with Locale::Maketext for our next project at work. The biggest problem (remaining - it solves a lot of problems) I have is in populating the files that need to be sent to translation. On one hand, it's awesome that I don't need to go and update 10-30 different language files every time I want to add another string. The downside is that I still have to do that eventually, so that the translators know what to translate. So the first thing popping into my head is that every call to Locale::Maketext::maketext could be found automatically, its first parameter pulled out, and we instantly have the list of strings that need translating.

With PPI, this seems to have gone fairly smoothly. Getting the first Expression from the maketext call seems to work, and it conveniently keeps the quotes, which allows me to propagate them if there are special characters (\n, \t, etc.).

The next obvious part is to automatically insert the text into the language-specific module files. One possibility is to use a templating solution that builds each file. Unfortunately, that comes with a couple downsides:

  • Existing translations (if some of the text has already been translated) can get lost
  • Code that someone puts in to handle things like quant or numf for specific languages in specific ways will be done in a non-standard way: updating the template rather than the module directly
The most obvious answer, then, to me, is to continue using PPI, this time to insert additional code. If it doesn't format well, I can always throw perltidy at it afterwards, so that's a nice-to-have but still quite secondary.

Going down this road, I got confused as to the proper incantation. There's a mental block between me and success on this, so I was hoping that someone else has gone down this road more successfully than I and could provide the missing piece. Here is what I have (ok, not really - this is just boiled down to the salient points):

#! /usr/bin/perl use strict; use warnings; use PPI; my $perlcode = << "END_PERLCODE"; package My::L10N::en; use base 'My::L10N'; our %Lexicon = ( 'Some [_1] text' => 'Some [_1] text', ); END_PERLCODE my $doc = PPI::Document->new(\$perlcode); my $lexicon = $doc->find(sub { $_[1]->content eq '%Lexicon' }); $lexicon = $lexicon->[0]; # returns an array ref, but in my case, ther +e's only ever one. my $newmsg = q{'The [_1] is in the [_2]'}; my $newtext = sprintf ' %s =>\n %s,', $newmsg, $newmsg; my $insert_this = PPI::Document->new(\$newtext); # insert $newtext into %Lexicon? $lexicon->insert_after($insert_this->tokens()); print $doc->serialize();
It outputs the input unchanged:
package My::L10N::en; use base 'My::L10N'; our %Lexicon = ( 'Some [_1] text' => 'Some [_1] text', );
What I was hoping for is more like:
package My::L10N::en; use base 'My::L10N'; our %Lexicon = ( 'Some [_1] text' => 'Some [_1] text', 'The [_1] is in the [_2]' => 'The [_1] is in the [_2]', );
(Yes, I know the example is missing the "1;" at the bottom, but it's just an example.) Thanks,

Update: ARGH. After getting Khen1950fx's "solution", I realised that I could simply mandate something like "### NEW TEXT GOES HERE" as a marker to find and insert on. But, because I still wanted to figure this out with PPI (so that people couldn't clobber my nice insert marker), I kept hacking. And then I got it.

#! /usr/bin/perl use strict; use warnings; use PPI; use PPI::Dumper; sub mydump { PPI::Dumper->new(shift)->print() } my $perlcode = << "END_PERLCODE"; package My::L10N::en; use base 'My::L10N'; our %Lexicon = ( 'Some [_1] text' => 'Some [_1] text', ); 1; END_PERLCODE my $doc = PPI::Document->new(\$perlcode); my $lexicon = $doc->find(sub { $_[1]->content eq '%Lexicon' }); $lexicon = $lexicon->[0]; # returns an array ref, but in my case, ther +e's only ever one. my $newmsg = q{'The [_1] is in the [_2]'}; my $newtext = sprintf " %s =>\n %s,\n", $newmsg, $newmsg; my $insert_this = PPI::Document->new(\$newtext); # insert $newtext into %Lexicon? my $stmt = $lexicon->statement(); my $list = $stmt->find('PPI::Structure::List')->[0]; for ($insert_this->tokens()) { $_->remove(); my $rc = $list->add_element($_); } print $doc->serialize();
Apparently, insert_after doesn't take multiple elements (yet?). So I have to do one after another. After trying a few things, I got to the point where I was re-finding each last-child, and inserting after that. But that got ugly fast, and didn't seem to work in my original code. And that's when I happened upon add_element. However, in that case, I found I had to remove the element from its existing doc first (which is fine, it's just the code snippet I created dynamically anyway), reminiscent of using XML::Twig. So, I simply remove and add. I also found I had to find the list - because it was the list's children where I was adding the element. Now off to "create" %Lexicon if it doesn't exist (should be much easier now that I have this working...)...

Comment on Programmatically Updating Code with PPI
Select or Download Code
Re: Programmatically Updating Code with PPI
by Khen1950fx (Canon) on Jan 04, 2010 at 22:36 UTC
    I tried this:
    #! /usr/bin/perl use strict; use warnings; use diagnostics -verbose; use PPI; my $perlcode = << "END_PERLCODE"; package My::L10N::en; use base 'My::L10N'; our %Lexicon = ( 'Some [_1] text' => 'Some [_1] text' 'The [_1] is in the [_2]' => 'The [_1] is in the [_2]' ); END_PERLCODE my $doc = PPI::Document->new(\$perlcode); my $lexicon = $doc->find(sub { $_[1]->content eq '%Lexicon' }); $lexicon = $lexicon->[0]; # returns an array ref, but in my case, ther +e's only ever one. my $newmsg = q['The [_1] is in the [_2]']; my $newtext = sprintf(' %s =>\\n %s,', $newmsg, $newmsg, $newmsg, $newmsg); my $insert_this = PPI::Document->new(\$newtext); $lexicon->insert_after($insert_this->tokens()); print $doc->serialize; 1;

      You cheated ;-) The $perlcode you started with was modified, which isn't the problem I'm having. You presumed the work was already done to insert that code, whereas I want to write the code to do that insertion. Thanks, though.

Re: Programmatically Updating Code with PPI
by JadeNB (Chaplain) on Jan 04, 2010 at 22:51 UTC
    I don't have PPI installed, so can't test; but I hope you don't mind a few quick questions:
    1. Wouldn't
      my $lexicon = $doc->find(sub { $_[1]->content eq '%Lexicon' }); $lexicon = $lexicon->[0]; # returns an array ref, but in my case, ther +e's only ever one.
      be better as
      my $lexicon = $doc->find_first( ... )
      ?
    2. Does the insert_after call return true?
    3. When $lexicon is returned by find, is it still “part of” $doc? That is, could it be that you're modifying only $lexicon, not $doc?
Re: Programmatically Updating Code with PPI
by Anonymous Monk on Jan 04, 2010 at 23:15 UTC
    I see a two problems 1) you're trying to insert at the wrong place (before assignment operator =), and 2) you're inserting the wrong thing (whitespace)
    tokens PPI::Token::Whitespace ' ' insert_this PPI::Document PPI::Token::Whitespace ' ' PPI::Statement PPI::Token::Quote::Single ''The [_1] is in the [_2]'' PPI::Token::Whitespace ' ' PPI::Token::Operator '=>' PPI::Token::Cast '\' PPI::Token::Word 'n' PPI::Token::Whitespace ' ' PPI::Token::Quote::Single ''The [_1] is in the [_2]'' PPI::Token::Operator ',' lexicon PPI::Token::Symbol '%Lexicon' lexicon PPI::Token::Symbol '%Lexicon' package My::L10N::en; use base 'My::L10N'; our %Lexicon 'The [_1] is in the [_2]' =>\n 'The [_1] is in the +[_2]', = ( 'Some [_1] text' => 'Some [_1] text', );
      Both insert_after and tokens/children seem to be broken methods (or unfinished), but I've managed to get it done using __insert_after
      #!/usr/bin/perl -- use strict; use warnings; use PPI; my $perlcode = << "END_PERLCODE"; package My::L10N::en; use base 'My::L10N'; our %Lexicon = ( 'Some [_1] text' => 'Some [_1] text', ); END_PERLCODE my $doc = PPI::Document->new(\$perlcode); my $lexicon = $doc->find_first(sub { $_[1]->content eq '%Lexicon' }); my $newmsg = q{'The [_1] is in the [_2]'}; my $newtext = sprintf qq' %s =>\n %s,\n', $newmsg, $newmsg; my $insert_this = PPI::Document->new(\$newtext); #~ use PPI::Dumper; #~ print __LINE__,"\n"; PPI::Dumper->new( $lexicon->parent )->print; #~ print __LINE__,"\n"; PPI::Dumper->new( $lexicon->parent->last_token + )->print; #~ print __LINE__,"\n"; PPI::Dumper->new( $lexicon->parent->last_token +->previous_token )->print; #~ print __LINE__,"\n"; PPI::Dumper->new( $lexicon->parent->last_token +->previous_token->previous_token )->print; #~ print __LINE__,"\n"; PPI::Dumper->new( $insert_this->find_first('St +atement') )->print; #~ print __LINE__,"\n"; PPI::Dumper->new( $insert_this->find_first('St +atement')->tokens )->print; #~ print __LINE__,"\n"; PPI::Dumper->new( $insert_this->find_first('St +atement')->children )->print; $lexicon->parent->last_token->previous_token->previous_token ->__insert_after( $insert_this); print $doc->serialize(); __END__ package My::L10N::en; use base 'My::L10N'; our %Lexicon = ( 'Some [_1] text' => 'Some [_1] text', 'The [_1] is in the [_2]' => 'The [_1] is in the [_2]', );

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://815650]
Approved by broomduster
Front-paged by ikegami
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (6)
As of 2014-04-24 10:42 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (565 votes), past polls