|Just another Perl shrine|
I'm puzzled by something in your description.
You seem to be thinking in terms of many independent applications that exist seperately and are started by huge copy and pastes. If that's accurate, then that's a terrible way to operate. I think in terms of one application that has many parts to it, many of which share components. So the second developer just checks out the application and starts adding on where needed.
If their development will take a while, then they should branch and then merge that branch back into HEAD when the project is finished.
Another red flag is that you're talking about having each developer check their stuff out on production. And then wonder who owns the files. That's a big red flag.
Instead have a regular release process to production (we aim for weekly) and have the actual installation be done using specific production user accounts. Developers don't even have personal logins on production machines. That problem is gone, and several with it.
Incidentally the release process should be scripted and automated. Both pushing to QA, and then pushing from QA to production, should be a matter of pushing a button and watching it work. That way you guarantee that important steps (eg tagging the release and running your test suite) happen. A note on the production release. A good strategy is to take half your machines out of the load balancer, install there, tell the load balancer to switch which machines are online, install on the rest, then bring the rest back online. That way at all points all webservers are consistent.
About production machines. Let me just say that scripting is a good thing. Script how to install version X of Foo on machine Y. Then do that on every production machine. And make that part of your install process.
Personally I don't like having /usr/local/ on a shared mount, because that mount can become a single point of failure. Plus see the binary incompatibility issue that I had before - you're now forced to do "big bang" OS upgrades, all machines at once. However to whatever extent possible, you want to make your machines cookie cutter copies. The details are a matter of system administration.