Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

How to implement a fourth protocol

by Moron (Curate)
on Mar 27, 2007 at 16:26 UTC ( [id://606789]=perlquestion: print w/replies, xml ) Need Help??

Moron has asked for the wisdom of the Perl Monks concerning the following question:

I notice a lot of recent posts here relate to denial of service attacks on HTTP and HTTPS servers and how to use Perl to fight them. It occurred to me that even having detected a bad bot spinning your wheels, blocking them at the firewall is still going to cost you bandwidth and some CPU cycles just for spinning the firewall.

So one idea I want to experiment with is to make a custom protocol that could be used between a system and its proper clients that avoids spinning the firewall by disabling all the usual protocols and only accepting a custom one. My first idea was to use the tty protocol, which badbots no longer bother with these days, but when playing around with this from the browser to see what would happen (entering a URL of tty://somewhere.com it got translated into ftp://somewhere.com whereas xxx:// did not default to that. What I presume is happening is that because tty is a real protocol, something has to translate it to stop it spinning the login shell.

So the initial questions are:

(1) is there a way to make tty:// really log into a system from a browser (useful for a testing phase of this)

(2) Is there some other protocol than FTP or HTTP(S) that could be used for client/server sessions across the net without having to butcher a potentially needed mail protocol or something? If not can a new one be defined?

(3) Are there any Perl modules that could help me? I looked at NetServer::Generic and that might be very useful but it seems to leave uncharted territory for me to cover such as what about the client end.

tyvm in advance!

-M

Free your mind

Replies are listed 'Best First'.
Re: How to implement a fourth protocol
by almut (Canon) on Mar 27, 2007 at 17:02 UTC

    Not directly answering your questions, but maybe you are looking for something like port knocking (google for other references, there are plenty...). The idea is to generally keep all ports closed, and only open specific ones (HTTP, SSH, whatever...) temporarily, if a magic pattern of requests is made to selected closed ports (typically only known to the initiated clients).

    This isn't supported by any browsers (AFAIK), but there is some prototype code in Perl, in case you want to play with it... Have fun!

      I like it. But looking through, I wonder if it is scalable. i.e. I imagine that it would itself deny adequate service above a certain number of concurrent client systems trying to connect.

      -M

      Free your mind

        Sure, depending on the implementation, there can be considerable overhead. So, it's generally more intended for the occasional use of some non-advertised service by a few initiated people...

        I think it will be very hard to run a regular public service, without at the same time opening up your resources to misuse of some kind, even if it's "only" bad folks clogging up your network bandwidth by sending tons of packets that get rejected at the firewall level...

Re: How to implement a fourth protocol
by gloryhack (Deacon) on Mar 27, 2007 at 17:04 UTC
    While it may technically be doable if you modify the browser to understand your "fourth protocol", in the final analysis you'll have to solve all of the same problems that have already been solved by things like mod_dos_evasive and firewalls, and then go on to dream up something more effective than they are.

    Other than being an interesting exercise, what would be the point? In what way would your "fourth protocol" be better/faster/more secure?

      The lower the network modelling layer at which you defend against a bot, the earlier in the cycle you can stop them in their tracks. The earlier their advances are cut off, the less ballast such a measure will weigh on your system resources. The server is one such layer, the protocol such as http is another, the firewall is at another, TCP/IP at another.

      -M

      Free your mind

        And in what way does having a "new" protocol help? At worst you've now got yet another point of ingress (through a new, untested protocol and its implementations no less) in addition to the existing ones (because in all likelihood you're not going to supplant them and someone will insist on still using them (c.f. the installed base of Netscape 4.76 browsers just now disappearing)). At best you've pushed the problem around . . . to a new, untested protocol and its new untested implementations; you might see some benefit in that not many black hats know about it enough to start jiggling the doorknobs, but security through obscurity isn't.

        I guess one person can make a difference, but most of the time, they probably shouldn't.Marge Simpson

        Update: Not that you're incorrect that you want to stop someone as far out as you can, but a new protocol's not necessarily the best way to do that. Things like port knocking or running on non-standard ports (or better yet port knocking to connect to a non-standard port :) reduce the visibility of the service, but if the underlying protocol (say) uses rot13 or its analogue to hide user credentials (HTTP Basic authentication, I'm looking at you . . .) it doesn't gain you much.

        If all you're doing is moving a service using an existing protocol to a different port you haven't gained much. If you're using a new protocol, you're tossing out years of live field testing. Things like SSH and TLS have been gone over by experts (white and black hatted both) and are at this point pretty much algorithmically sound and most of the exploits are against implementation bugs not protocol flaws.

        And an aside, one of the laments I remember seeing a while back was the problem that putting everything over HTTP makes it harder to lock down access at the network layer since everything is going through one port rather than separate ports for separate protocols. Coming full circle I guess.

        Hmmm ... generally you're right but your layers are wonky ... almost like you're confusing network model layer with what is sometimes called the security onion model. I would agree stopping them earliest is the best but sometimes you don't really have enough info in the lower layers of the network model to make that determination, that's why we have things like firewalls and dmzs -- make the decision as soon as possible in the network stack but also as far away as possible from critical areas.

        -derby
        Yep, and the firewall is at a pretty darn low level already since it's working at the packet level, isn't it? You cannot stop J. Random Cracker from spewing packets at your network unless you unplug from it. Or go to his house and unplug him from the internet :-) Short of that, his packets are coming at you even if they're just bouncing off of ports with no listeners.

        I'm assuming that we're talking about a public service here, and not something that can be very easily protected by something akin to VPN. In order to detect a bot, you have to allow it some initial degree of access so you can discern its intent. There's really no other way to determine the intent of a previously unseen client. After something at the server level determines that the client is malicious, then it has to work to defend itself. The lowest level available to us is the TCP/IP layer, where we can decide at the packet level whether to accept, reject, or drop the packets without the overhead of reassembling them into messages. This is the level where the firewall lives, very efficiently sieving bits. So, the easy way to implement a defensive measure is to give the server or some lightweight middleware the smarts to detect malicious activity and the means to communicate to the firewall "I don't want to hear from IP address ww.xx.yy.zz on my port nn any more". Bam, problem solved. From that point forward you don't analyze payloads, and the firewall just sieves bits. The best efficiency comes if you just drop those packets without bothering to tell the client that you don't want them. It's a bit rude, but it's efficient.

        So, again, how would the certain to introduce vulnerability new protocol be faster/easier/better than the existing tools which are readily available today?

Re: How to implement a fourth protocol
by derby (Abbot) on Mar 27, 2007 at 17:14 UTC

    Well ... tty is not an internet protocol (ala http, smtp, ftp, telnet, ssh, and ohh about 300 more). So yes, you could create your new protocol, name it tty (much to the chagrin of terminal coders everywhere) but you haven't really solved anything - just shifted the problem from well known, well tested services to your home-brew service --- all the spinning problems you've identified are still there.

    -derby
      I don't see any justification to your view being supplied.

      I do see contrary arguments such as: there are more Windows viruses for the simple reason that there are more Windows systems out there. So there should be more similarly http bots out there because there are plenty of http servers but no xyz bots because there are no xyz servers.

      As to the problem shifting slowly and reforming, perhaps it would eventually. But the weakness of the existing protocols is that they ARE protocols, leaving little room for flexible problem-solving of the bad-bot problem. The advantage of a clean slate is that you can build more room for anti-bot measures into the protocol itself and have more choice of layers of action.

      -M

      Free your mind

        It certainly would be nice to have a new protocol designed to deal with the myriad problems we are seeing today but the reality is it will take a group of technology and industry leaders to design a new protocol. And it will take them years to do it. Not to mention the RIAA and MPAA will likely try to get involved which will drag things out even further. And then how fast will it be implemented? It might take another year or two for Microsoft to implement it in Internet Explorer.

        Quite frankly, five years for design and implementation sounds optimistic to me which makes me think wide spread adoption of a new protocol would likely be a decade or more away.

        And that's why you are getting some of the feedback you are and why it is hard to take your "advantage of a clean slate" theory seriously. It is certainly a worthy goal and your optimism is admirable but it will be very, very hard to pull it off.

Re: How to implement a fourth protocol
by shmem (Chancellor) on Mar 27, 2007 at 21:05 UTC
    The usefulness of your ideas depend on the kind of service you are offering through the real protocol in terms of publicity/privacy. You can run an open webserver, or you can tunnel services through some sort of VPN to accredited clients, e.g. running SOCKS through an OpenSSH connection.

    That's much like the difference between a shopping mall and a private bar, but there's more to it. In a public mall, all kinds that make a world are allowed, but there might be cops around with a watchful eye to arrest evildoers. For a private bar, you could have door keepers, biometrics checking, ad-hoc-passwords or the like. Still those admitted could wreck havoc any time according to drinks absorbed and/or personal disposition.

    So behaviour of clients is crucial to detect mischief, regardless of any protocol involved.

    Bottom line is: you can't solve social problems with technology. (You can use technology to some degree of success as an aid to defend yourself, but only if you don't attack. Failure to observe that rule will render your technology worthless).

    (Hm. Now what does this have to do with your post?)

    --shmem

    _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                  /\_¯/(q    /
    ----------------------------  \__(m.====·.(_("always off the crowd"))."·
    ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
      Given that one would separate customer traffic from public traffic, there still needs to be scalability that could not be achieved by e.g. VPN and that is why the new protocol idea, so that customers can get on board by downloading a protocol installation kit as soon as they have a formal relationship (e.g. a financial account) whose registration can be automated rather than needing lower level network modifications or hardware.

      -M

      Free your mind

        Protocols are ways for computers to communicate. This is fairly fundamentally in opposition to your aim of security through obscurity - they don't know what language you're speaking, therefore they cannot hack it.

        You'll filter out _most_ of your automated attacks, by simply moving services onto another port. For everything else, you'll need the same security/authentication layers that you'll need anyway.

        What are you really trying to achieve? Reduced bot traffic? Just move ports. Even if you use a new protocol (which for a lot of reasons, isn't a good idea) then there will still be packets from the bots, bouncing off your firewall. So you might as well just run your webserver on a different port, and block all traffic on port 80. It'll achieve pretty much the same result - firewall can easily reject the traffic based on the port number.

        If you're thinking your new protocol will improve security, then ... well, there's an old adage: Security through obscurity, isn't. It's all well and good to work on the basis that if people don't know that you're using some unusual protocol, then they can't get in, but the fact that this protocol is going 'out onto the net' means it is impossible to guarantee that. And therefore, you still need to authenticate etc. everyone, and have an additional layer of complexity for your end users.

Or spin the bot back?
by Moron (Curate) on Mar 28, 2007 at 16:58 UTC
    One related idea I had at lunchtime is this: suppose the bot attempts to crawl your directory structure. Having detected it's a bad bot. Is there a way to make its client machine cache a recursive directory structure? (or a false picture of one induced by artificial response). I am not sure how caching works at a lower level - it's just a functional idea. But if so, then the bot would be crawling its own cache forever, rather than bothering your real machine with it.

    -M

    Free your mind

      Spiders don't revisit pages they have already visited. That's necessary because web sites are naturally full of loops. For example, the home page links to the contact page, and the contact page links to the home page. Since it doesn't make the same request twice, a client-side cache will never get hit. You can make the client spin, but it will involve a request to your server.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://606789]
Front-paged by andyford
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others pondering the Monastery: (5)
As of 2024-03-28 20:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found