Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

In need of VERY simple perl proxy

by perlpreben (Beadle)
on Aug 21, 2009 at 21:59 UTC ( #790487=perlquestion: print w/ replies, xml ) Need Help??
perlpreben has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

I have tried a lot of perl proxies that are on the net, anything from perl modules like HTTP::Proxy, and socket module. But they all fail on a different levels.. either that the webpage sends some content that is not understood and it crashes etc.

Does anyone have a quick sample on how I can use sockets, to listen to a port (say 8080), and just forward that out onto the internet. My aim is to have a super simple proxy that will only print out the urls that goes through. Nothing more, nothing less.

Comment on In need of VERY simple perl proxy
Re: In need of VERY simple perl proxy
by jethro (Monsignor) on Aug 22, 2009 at 00:38 UTC
    That all those proxies fail might indicate that proxying isn't such an easy task and to program a super simple proxy isn't so super simple.

    Maybe a full-fledged proxy like squid would work as well. It might provide hooks where you can plug in your perl script. Or your script could just look at the logfile or get the log lines piped from syslog, all the urls can be logged if you instruct squid to do so.

      I've been using squid for several weeks and found that it has problems in accessing pages at times for strange reasons. For example, sometimes I am forced to type google.com into the URL window instead of entering my search straight into the search window of the browser (which links to google). At other times it will work flawlessly--so it's been somewhat hit and miss, and I have been unable to find meaningful errors or reasons why.

      I hope you can find a simple proxy, but I'm unaware of any. I would be happy to hear of it too when you find one.

      Blessings,

      ~Polyglot~

        It is probably DNS related
Re: In need of VERY simple perl proxy
by carlin (Beadle) on Aug 22, 2009 at 13:48 UTC
    Your best bet would be to look through the source of each proxy you have tried to see how they do it and, if you can, try and see what causes the failure that particular one suffers from. The problem is though, with all the websites out there you can never predict what weird concoctions someone might have come up with, which results in the issues that you have encountered.
Re: In need of VERY simple perl proxy
by afoken (Parson) on Aug 22, 2009 at 17:32 UTC
    My aim is to have a super simple proxy that will only print out the urls that goes through. Nothing more, nothing less.

    What about using squid and look at its access.log?

    You could also use wireshark to log all HTTP traffic.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://790487]
Approved by AnomalousMonk
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (4)
As of 2014-07-14 00:45 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    When choosing user names for websites, I prefer to use:








    Results (253 votes), past polls