Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Speeding up Apache RewriteMap script.

by graq (Curate)
on Apr 25, 2007 at 11:12 UTC ( #611968=perlquestion: print w/ replies, xml ) Need Help??
graq has asked for the wisdom of the Perl Monks concerning the following question:

I have the requirement that I need to map the start of (what is effectively) PATH_INFO to an int (PK on some table). I had a look at Apache RewriteMap and chose to go down that route. I'm not sure what go-faster techniques are valid in this environment, or whether they would have any impact (e.g. memoize).

Using files to lookup the values, RewriteMap only handles simple string comparisons, which does not cater for my need.

These are my test URLs:
http://myurl.com/aa/index.html http://myurl.com/aa/bb/index.html http://myurl.com/aa/foo/index.html http://myurl.com/cc/index.html http://myurl.com/cc/dd/index.html
This is my mapfile:
aa 1 bb 2 cc/dd 3 cc/ee 4
Consider using the following mapping:
RewriteMap mapping txt:mapfile.txt RewriteRule ^/(.*)/ - [E=TEST_VAL:$1,E=TEST_RESULT:${mapping:$1},NS]

Because Apache uses a simple text match, the '/'s (slashes) in the URLs break the mapping. Which led me to use a program.

Apache Config: RewriteMap mapping prg:/my/dir/mapper.pl

mapper.pl:

#!/usr/bin/perl my $CONFIG_FILE = 'mapfile.txt'; my $config = &get_config; $| = 1; while(<STDIN>){ my $pk = lookup($_); print "$pk\n"; } sub get_config { my $apache_boot_time_message = "$0: Setting up Apache RewriteMap [$C +ONFIG_FILE]"; print STDERR "$apache_boot_time_message\n"; my %config; open( CONFIG, '<', $CONFIG_FILE ) or die ("Could not open $CONFIG_FI +LE: $!"); while(<CONFIG>){ chomp; my $line = $_; my @line = split('\s+', $line); $config{$line[0]} = $line[1] unless $line =~ /^#/; } close CONFIG; return \%config; } sub lookup { chomp; my $DIR = shift; my @parts = split( '/', $DIR ); for( 0..$#parts ){ my $dir = join('/', @parts[0..$_]); return $config->{$dir} if $config->{$dir}; } return "NULL"; }

As this script is going to be hit so many times by the webserver, I wondered if anyone had ideas on speeding it up?

-=( Graq )=-

Comment on Speeding up Apache RewriteMap script.
Select or Download Code
Re: Speeding up Apache RewriteMap script.
by merlyn (Sage) on Apr 25, 2007 at 11:13 UTC
    If you can use mod_perl, a PerlTransHandler is far more flexible (and easier to write/read for me) than any mod_rewrite configuration mish-mash ever was.

      I can happily accept that a custom piece of mod_perl is going to be more flexible. However, my requirements are fairly simple, and speed is the main issue.

      Are you saying writing a mod_perl handler will be a better performance boost than anything that can be done to the mapper script?

      -=( Graq )=-

        Are you saying writing a mod_perl handler will be a better performance boost than anything that can be done to the mapper script?
        Yes, I believe that to be the case, simply by looking at all the work mod_rewrite has to go through to talk to your script, compared to just executing a simple bit of code already compiled into the on-board Perl interpreter.
Re: Speeding up Apache RewriteMap script.
by Moron (Curate) on Apr 25, 2007 at 11:39 UTC
    I agree with merlyn, but if for some reason you are stuck with this approach, then the most obvious optimisation that stares back at me would be to put configuration values as assignments in httpd.conf (loaded once per server startup) instead of having a full cycle of file i/o to read that separate config file occurring once per rewrite.
    __________________________________________________________________________________

    ^M Free your mind!

      The mapper script only reads the file once for each VirtualHost (when Apache starts). It then sits in memory.

      configuration values as assignments in httpd.conf

      I'm not sure how this will solve my problem (perhaps I have not explained the problem well enough). Do you have a snippet example?

      -=( Graq )=-

        In your OP you said "As this script is going to be hit so many times by the webserver, I wondered if anyone had ideas on speeding it up?"

        So if that is no longer perceived to be the case, I guess we can close this issue now?

        __________________________________________________________________________________

        ^M Free your mind!

        Key to hats: ^I=white ^B=black ^P=yellow ^E=red ^C=green ^M=blue - see Moron's scratchpad for fuller explanation.

Re: Speeding up Apache RewriteMap script.
by awgtek (Initiate) on Mar 22, 2009 at 04:08 UTC
    perhaps strip forward slashes or replace with another matchable character in a preceding rewrite rule? Then you could use the text file approach rather than the script.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://611968]
Approved by shigetsu
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (6)
As of 2014-09-16 22:52 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (51 votes), past polls