"The First Rule of Distributed Objects is..."

[perrin]

"... don't distribute your objects!" -- Martin Fowler, author of the book "Refactoring."

Can you use multiple physical tiers? Yes. You can use XML-RPC or SOAP to do it, although neither of those really offer object-oriented access (but neither does EJB, when used with popular approaches like "session facade"). You could probably get better performance by using straight HTTP and Storable for serialization, but then you'd have to write it yourself.

More importantly, should you? As you already realized, there is no need to have multiple physical tiers just to get the abstraction benefits. Adding physical tiers will add tons of overhead in the form of RPC calls. Don't let anyone tell you that this overhead is insignifcant. On a fast platform like mod_perl, this sort of RPC stuff is likely to be your biggest bottleneck. It could easilly cut your performance in half, or worse, and would probably force you to change your nice fine-grained OO interfaces into ugly corase-grained ones just to reduce the number of calls it takes to do something (J2EE people do this all the time with the aforementioned session facade stuff).

So why would anyone do this? The standard response is "scalability." The theory here is that it will be helpful to be able to add more machines to your application server tier without adding any to your web servers, because your application servers need the extra power and your web servers don't. If you have both of these in a single physical tier, you can't separate them like that. When you think about it though, what exactly is the problem with your web servers getting more power than they need? Having these in the same process makes you use fewer resources (because of the missing RPC overhead), not more. I don't think you could reasonably argue that it takes more machines to do the same work because they are together. It will probably take fewer machines. You also get to use standard approaches for load-balancing, like hardware HTTP load-balancers.

Some people will say "What about when you have really uneven business objects, and a few of them take 10 times as long to run as others? That will mess up your HTTP load-balancing." Well, most load-balancers can keep track of how many requests they have open to each web server and use that to send requests to the ones that have the lightest load, but it is possible that you could have a component that did some really heavy calculation and sucked up lots of CPU, making it hard to load-balance because it is so unlike the other requests. You can deal with this either by using asynchronous processing (a queue), or by dedicating some specific servers to handle these requests. This avoids penalizing every single request with RPC overhead. It's easy to route requests based on URL when you are using a front-end proxy or a good load-balancer.

The real joke here is that the people who shout about this loudest are typically the EJB vendors, and they are selling snake oil. EJB performance is so bad when remote calls are involved that the EJB containers were all written to make the calls local whenever they can. That's right: the remote calls are actually turned into locals calls into the same Java process whenever possible. In EJB 2, an explicit local interface was added so that developers can force this behavior on the few EJB containers that weren't already doing it!

There is one thing that is good to separate, and that's your static web content. No need to tie up a big mod_perl process serving your JPEGs. There are several ways you can handle this, discussed here.

You might also be interested in the article I wrote about how we used multiple logical tiers and in some cases physical tiers (for separate search servers) at eToys.com. That article is here.

[exussum0]

So why would anyone do this? The standard response is "scalability." The theory here is that it will be helpful to be able to add more machines to your application server tier without adding any to your web servers, because your application servers need the extra power and your web servers don't. If you have both of these in a single physical tier, you can't separate them like that. When you think about it though, what exactly is the problem with your web servers getting more power than they need? Having these in the same process makes you use fewer resources (because of the missing RPC overhead), not more. I don't think you could reasonably argue that it takes more machines to do the same work because they are together. It will probably take fewer machines. You also get to use standard approaches for load-balancing, like hardware HTTP load-balancers.

I'll try and refute this. Imagine a site that keeps it simple and they need to keep the fast parts to stay fast. Well, things that take up more speed need more resources. Yes, throwing more hardware, complete webservers, does solve something, but in the long run, the entire site will slow down again as it gets more popular.

So fine, now you create a seperate farm for one part of your site. Tons of links to change over etc etc..

It'd a big pain in the ass. So why not keep the web interface running uber fast, the actual display layer, but move the bottle necks to a seperate area. This of course, as we both know, is how n-tiered would start to evolve. The advantage is, that if I hit one interface that is dog slow, it may not mean the others will slow down. That might mean tweaking the aplication layer or db layer.

As for your RPC argument, you are right, but then again, the same problems happen in PHP or Perl. That's why you have connection pools and the likes. You keep everything on high speed connections, and things will run well. You and i both know that sending 1k of data over RJ45 is very quick to move that type of stuff over. And we all know you don't do repeditive rpc calls to get all your data.. .just one fell-swoop with one biz objects.

Also, doing things via proxy server would be fast, but as the scenario gets more complex with a url proxy, things slow down in a way you can't speed back up. Load balancing a bunch of proxy servers is just ugly. 'cause then you have something that is just as bad as a regualr web farm. As things slow down, the entire service slows down and not just the parts that are costly.

[perrin]

Imagine a site that keeps it simple and they need to the fast parts to stay fast. Well, things that take up more speed need more resources. Yes, throwing more hardware, complete webservers, does solve something, but in the long run, the entire site will slow down again as it gets more popular.

I'm not sure what your point is here. Yes, scaling typically involves adding hardware as well as tuning your code. What does this have to do with physical tiers vs. logical tiers?

So fine, now you create a seperate farm for one part of your site. Tons of links to change over etc etc..

First, you only separate things if they are so uneven in terms of the resources they require that normal load-balancing (treating all requests for dynamic pages equally) doesn't work. This is very rare. Second, you NEVER change links! There is no reason to change your outside URLs just because the internal ones changed. If you want to change where you route the requests internally, use your load-balancer or mod_rewrite/mod_proxy to do it.

So why not keep the web interface running uber fast, the actual display layer, but move the bottle necks to a seperate area.

How does this help anything? The amount of work is still the same, except you have now added some extra RPC overhead.

The advantage is, that if I hit one interface that is dog slow, it may not mean the others will slow down. That might mean tweaking the aplication layer or db layer.

Can you explain what you're talking about here? Are you saying that some of your requests will not actually need to use the db layer or application layer?

You keep everything on high speed connections, and things will run well. You and i both know that sending 1k of data over RJ45 is very quick to move that type of stuff over.

Running well is a relative term. They will not run anywhere near as fast as they would without the RPC overhead. I'm not making this stuff up; this is coming from real applications in multiple languages that I have profiled over the years. Socket communication is fast, but it's much slower than most of the other things we do in a program that don't require inter-process communication.

And we all know you don't do repeditive rpc calls to get all your data.. .just one fell-swoop with one biz objects.

And that's the other problem: the RPC overhead forces you to replace a clean OO interface of fine-grained methods with "one fell-swoop..." This is one of Fowler's biggest complaints about distrubuted designs.

Also, doing things via proxy server would be fast, but as the scenario gets more complex with a url proxy, things slow down in a way you can't speed back up.

I'm not sure where you're getting this from. mod_rewrite (the most common choice for doing things based on URL) is very fast, and the hardware load-balancers are very very fast.

Load balancing a bunch of proxy servers is just ugly. 'cause then you have something that is just as bad as a regualr web farm. As things slow down, the entire service slows down and not just the parts that are costly.

What's so bad about a web farm? Every large site requires multiple web servers. And which parts are costly?

I think you are imagining that it would take less hardware to achieve the same level of service if you could restrict what is running on each box so that some only run "business objects" while others run templating and form parsing code. I don't see any evidence to support this idea. If the load is distributed pretty evenly among the machines, I would expect the same amount of throughput, except that with the separate physical tiers you are adding additional load in the form of RPC overhead.

Think of it like this: you have a request that takes 10 resource units to handle -- 2 for the display and 8 for the business logic. You have 2 servers that can each do 500 resource units per second for a total of 1000. If you don't split the display and business logic across multiple servers and you distribute the load evenly across them, you will be able to handle 100 requests per second on these servers. If you split things up so that one server handles display and the other handles business logic, you will max out the business logic one at 62 requests per second (496 units on that one box). So you buy another server to add to your business logic pool and now you can handle 125 requests per second, but your display logic box is only half utilized, and if you had left these all together and split them evenly across three boxes you could have been handling 150 at this point. And this doesn't even take the RPC overhead into account!

Distributed objects sound really cool, and there is a place for remote cross-language protocols like SOAP and XML-RPC, but the scalability argument is a red herring kept alive by vendors.

[exussum0]

I'm not sure what your point is here. Yes, scaling typically involves adding hardware as well as tuning your code. What does this have to do with physical tiers vs. logical tiers?

Just reving up :)

First, you only separate things if they are so uneven in terms of the resources they require that normal load-balancing (treating all requests for dynamic pages equally) doesn't work. This is very rare. Second, you NEVER change links! There is no reason to change your outside URLs just because the internal ones changed. If you want to change where you route the requests internally, use your load-balancer or mod_rewrite/mod_proxy to do it.

Sometimes, you don't know ahead of time what gets used more than naught. Usually products evolve. And using mod_proxy or the likes is a cludge. What happens when you move something more than once? Regardless, doing the extra work for moving stuff around is ugly anyway. It's unnecessary work if you seperate your "hard work" away.

How does this help anything? The amount of work is still the same, except you have now added some extra RPC overhead.

Yeah, but you've moved your hard work away to pools dedicated to resources that can handle it. Things like static stuff doesn't need to hit db resources ever. Things that need to do intensive stuff goes to a pool that does that. It breaks down more and more.

Can you explain what you're talking about here? Are you saying that some of your requests will not actually need to use the db layer or application layer?

Ok, for instance, let's say your login page is realy fast, and so is your page after auth. Now let's say your preferences page is REALLY slow. It takes up lots of resources since it gets pegged a lot. Seperating out the logic that is so slow because it gets hit so much can be moved to it's own pool. Now you have one set { login, homepage} and another {preferences} which can be in two different pools. The poools don't have to be server farms of machines, but when things get more complex or not, you can allocate or take away resources for them.

Running well is a relative term. They will not run anywhere near as fast as they would without the RPC overhead. I'm not making this stuff up; this is coming from real applications in multiple languages that I have profiled over the years. Socket communication is fast, but it's much slower than most of the other things we do in a program that don't require inter-process communication.

You are 100% right, but the time difference is insignificant. A pooled connection vs an in-machine IPC call's speed is a magnatitude faster, but in terms of user experience, it is so small, that you can hardly notice.

And that's the other problem: the RPC overhead forces you to replace a clean OO interface of fine-grained methods with "one fell-swoop..." This is one of Fowler's biggest complaints about distrubuted designs.

No it doesn't. They are called transfer objects. Just a basket where you say, I want NN and it returns back in one request. Nothing particularly messy about it. If you do it generic enough, I'm sure that you could abstract it out to many many uses and not 1 dedicated object transfer.

I'm not sure where you're getting this from. mod_rewrite (the most common choice for doing things based on URL) is very fast, and the hardware load-balancers are very very fast.

Yup, but then again, so are RPC calls :)

What's so bad about a web farm? Every large site requires multiple web servers. And which parts are costly? I think you are imagining that it would take less hardware to achieve the same level of service if you could restrict what is running on each box so that some only run "business objects" while others run templating and form parsing code. I don't see any evidence to support this idea. If the load is distributed pretty evenly among the machines, I would expect the same amount of throughput, except that with the separate physical tiers you are adding additional load in the form of RPC overhead.

Ah.. that's the thing. evenly. You don't want everything running evenly. If slashdot could seperate out say, it's front page logic from its comment logic, then the front page will always be speedy and the comments section be its relative speed. As more people do commenty stuff, the home page stays right quick.

Think of it like this: you have a request that takes 10 resource units to handle -- 2 for the display and 8 for the business logic. You have 2 servers that can each do 500 resource units per second for a total of 1000. If you don't split the display and business logic across multiple servers and you distribute the load evenly across them, you will be able to handle 100 requests per second on these servers. If you split things up so that one server handles display and the other handles business logic, you will max out the business logic one at 62 requests per second (496 units on that one box). So you buy another server to add to your business logic pool and now you can handle 125 requests per second, but your display logic box is only half utilized, and if you had left these all together and split them evenly across three boxes you could have been handling 150 at this point. And this doesn't even take the RPC overhead into account!

Distributed objects sound really cool, and there is a place for remote cross-language protocols like SOAP and XML-RPC, but the scalability argument is a red herring kept alive by vendors.

Or not. Say the cost of rendering a page is small, s. You have 1 server that can deal with 10 connectiosn really well. On that same server, you have b, a big process that takes a lot of time. and 10 tiny processes t. b bogs down t to the point of "slow". You add another server. Things get "better" but imagine if you tier'ed it. You have three machines. One that handles s, one that handles b and one that handles t. the t-machine will alwyas run fast. And as more people use b, you add more resources for b. But as b continuously gets more and more poeple, T NEVER slows down. THAT is what you want to avoid.

You don't want to add to the entire pool and have to speed up everything in one fell-swoop. It's the same reason you have a 3d video card and a cpu completely seperate. Totally seperate purposes for different things. If your cpu gets pegged for whatever reason, your 3d video doesn't. You can tweak the 3d card or even replace it w/o having to go through hell.

Btw, there's always statistics, but we can always skew them in various ways. I can quote numbers any way i want, even to refute my own argument. But you can't refute that if T stays simple and fast, and B gets more complex, that T would be unaffected. :)

Btw, my GF is ticked from all the typing you are making me doing. She was on the phone and kept thinking i was doing something more important, from the clakity clak i was making :)

[freddo411]