Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things

Pass through perl http proxy server (no code, just advice)

by tachyon (Chancellor)
on Feb 28, 2003 at 21:38 UTC ( #239558=perlquestion: print w/replies, xml ) Need Help??

tachyon has asked for the wisdom of the Perl Monks concerning the following question:

I have a need to implement an http filter ie Net Nanny/Surf Control style to protect innocent students from the evils of the internet at large. OK so I want to write a proxy server to accept requests, validate the URL and then do pass through/deny. I know how to do all of this with sockets, fork, redirects and a DBMS. The question is boring. Although I really love reinventing wheels I wish to learn from those who have gone before so any suggestions of what is currently available for me to R&D* would be appreciated.



* R&D == Rip off && Duplicate. Open source == R&D && include attribution && share


  • Comment on Pass through perl http proxy server (no code, just advice)

Replies are listed 'Best First'.
•Re: Pass through perl http proxy server (no code, just advice)
by merlyn (Sage) on Feb 28, 2003 at 22:15 UTC

      I just love getting you to say it! Kinda like Pavlov's dog and ringing that bell. Of course I have to grant it to you, been there done that, sometimes done it twice! Might make a good slogan for stonehenge....




Re: Pass through perl http proxy server (no code, just advice)
by BooK (Curate) on Mar 01, 2003 at 08:51 UTC

    Well, uh, I'm just writing a module that's called HTTP::Proxy. ;-)

    The goal is to be able to pass every request and response though a set of configurable Perl "filters".

    Here's an example script that creates a anonymizing proxy, just like in merlyn's column (his script has been one of my inspirations, along with two others scritps by Abigail and Rafael Garcia-Suarez):

    #!/usr/bin/perl -w use HTTP::Proxy qw( :log ); use strict; # a very simple anonymizing proxy my $proxy = HTTP::Proxy->new; $proxy->logmask( shift || NONE ); # log configuration # the anonymising filter $proxy->push_headers_filter( mime => undef, # apply this on any type of content request => sub { $_[0]->remove_header(qw( User-Agent From Referer Cookie )); }, response => sub { $_[0]->remove_header(qw( Set-Cookie )),; }, ); $proxy->start;

    (This code works with the development version.)

    The proxy is not fully functional yet, though. One of my goal is to create a set of filters that would log the whole session, so that yet another script could (automatically) create a web robot from the session transcript.

    If you are interested, there is a web site (older versions and CVS snapshot) and a mailing-list at I am interested in help, and comments on the API (which is subject to change).

Re: Pass through perl http proxy server (no code, just advice)
by zengargoyle (Deacon) on Feb 28, 2003 at 21:56 UTC

    not Perl and a heavyweight (functionwise, not necessarily slow/bloated) is Squid Web Proxy Cache which you could probably make do whatever you can imagine.

Re: Pass through perl http proxy server (no code, just advice)
by IlyaM (Parson) on Mar 01, 2003 at 13:06 UTC
    In my attempt to learn POE I wrote this simple single-process proxy:
    #!/usr/bin/perl use warnings; use strict; use POE qw(Component::Server::TCP Component::Client::HTTP Filter::HTTPD Filter::Stream); POE::Component::Client::HTTP->spawn(Alias => 'user_agent', Streaming => 1024); POE::Component::Server::TCP->new ( Alias => 'web_server', Port => 8000, ClientFilter => 'POE::Filter::HTTPD', ClientInput => \&client_input, InlineStates => { proxy_response => \&proxy_response } ); $poe_kernel->run(); sub client_input { my ($kernel, $heap, $request) = @_[KERNEL, HEAP, ARG0]; # if request is actually response it just means that we've got an # error if($request->isa('HTTP::Response')) { $heap->{client}->put($request); $kernel->yield('shutdown'); return; } $request->header(Connection => 'close'); $kernel->post(user_agent => request => proxy_response => $request) +; } sub proxy_response { my ($kernel, $heap, $request_packet, $response_packet) = @_[KERNEL, HEAP, ARG0, ARG1]; my $request = $request_packet->[0]; my ($response, $data) = @$response_packet; unless($heap->{got_headers}) { $response->header(Connection => 'close'); $heap->{client}->put($response); $heap->{got_headers}++; } if(defined $data) { $heap->{client}->set_output_filter(POE::Filter::Stream->new()) +; $heap->{client}->put($data); } else { $kernel->yield("shutdown"); } }
    Note that at the time of writing it requires patched POE::Component::Client::HTTP.

    Ilya Martynov,
    CTO IPonWEB (UK) Ltd
    Quality Perl Programming and Unix Support UK managed @ offshore prices -
    Personal website -

Re: Pass through perl http proxy server (no code, just advice)
by traveler (Parson) on Feb 28, 2003 at 22:51 UTC
    Not perl, but a famous solution is dansguardian. It is GPL.

    HTH, --traveler

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://239558]
Approved by pfaut
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (4)
As of 2023-06-10 07:20 GMT
Find Nodes?
    Voting Booth?
    How often do you go to conferences?

    Results (37 votes). Check out past polls.