Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re: Pass through perl http proxy server (no code, just advice)

by BooK (Curate)
on Mar 01, 2003 at 08:51 UTC ( #239675=note: print w/ replies, xml ) Need Help??


in reply to Pass through perl http proxy server (no code, just advice)

Well, uh, I'm just writing a module that's called HTTP::Proxy. ;-)

The goal is to be able to pass every request and response though a set of configurable Perl "filters".

Here's an example script that creates a anonymizing proxy, just like in merlyn's column (his script has been one of my inspirations, along with two others scritps by Abigail and Rafael Garcia-Suarez):

#!/usr/bin/perl -w use HTTP::Proxy qw( :log ); use strict; # a very simple anonymizing proxy my $proxy = HTTP::Proxy->new; $proxy->logmask( shift || NONE ); # log configuration # the anonymising filter $proxy->push_headers_filter( mime => undef, # apply this on any type of content request => sub { $_[0]->remove_header(qw( User-Agent From Referer Cookie )); }, response => sub { $_[0]->remove_header(qw( Set-Cookie )),; }, ); $proxy->start;

(This code works with the development version.)

The proxy is not fully functional yet, though. One of my goal is to create a set of filters that would log the whole session, so that yet another script could (automatically) create a web robot from the session transcript.

If you are interested, there is a web site (older versions and CVS snapshot) and a mailing-list at http://http-proxy.mongueurs.net/. I am interested in help, and comments on the API (which is subject to change).


Comment on Re: Pass through perl http proxy server (no code, just advice)
Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://239675]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (15)
As of 2014-10-01 13:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    What is your favourite meta-syntactic variable name?














    Results (20 votes), past polls