Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Re: Total speculation?

by liz (Monsignor)
on Oct 02, 2003 at 10:30 UTC ( [id://295876]=note: print w/replies, xml ) Need Help??


in reply to Total speculation?

Sometimes I wonder whether it wouldn't be a better idea to look at the functionality that the Monastery currently offers and implement a new engine that would allow the same functionality, using todays knowhow and available software. And then at one point, close the Monastery for renovation for a few hours at most while the database is being converted.

And then I realize somebody/ies would have to do it in their spare time. And I wake up ;-(

Liz

Replies are listed 'Best First'.
Re: Re: Total speculation?
by BrowserUk (Patriarch) on Oct 02, 2003 at 10:54 UTC

    Personally, I don't think that there is that much wrong with PM.

    There are some things I would like to see fixed and there's no doubt that any codebase that has been subjected to 'live maintainance' over an extended period is surely due for some remedial refactoring, and there undoubtably some good ideas (tye has mentioned a few in the past) that could be implemented to improve the performance etc.

    I don't think I would throw the baby out with the bathwater. Too many times, essential fixes to the original codebase get overlooked in "rewrites from scratch", and then have to be re-invented when the new system goes operational and you rapidly end up with a new, but heavily patched system that exhibites all the same flaws as the original.

    It comes back to the thoughts I expressed a while ago of always starting from a working base. Essential to the philosophy is the ability to have ideas and try them out without compromising the working system. A test system is pretty much essential.

    It breifly crossed my mind that maybe something like thttpd -- a lightweight, throttleable server might be used to provide this, but I seriously doubt that it has a mod_perl environment available:(


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
    If I understand your problem, I can solve it! Of course, the same can be said for you.

      Perhaps an Apache virtual host would do it. I think a bare-bones throttling system could be set up with a filter. It could certainly be done with an Apache 2.0 PreConnectionFilter, though I doubt anybody is willing to upgrade PM to Apache 2 just yet.

      ----
      I wanted to explore how Perl's closures can be manipulated, and ended up creating an object system by accident.
      -- Schemer

      Note: All code is untested, unless otherwise stated

        FWIW, I did discover that there is a version of THTTPD available with a built-in perl5 interpreter. Whether that would be enough to run mod_perl I have no idea, but if it is, it might be worth considering. At 200k, it's a single-threaded, single-process, select model server with every conceivable throttling option built-in, and more than up to the task of acting as a test server.


        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
        If I understand your problem, I can solve it! Of course, the same can be said for you.

      Personally, I don't think that there is that much wrong with PM.

      You're talking about features. Feature-wise, there is not much wrong except the performance. However, contributing code to PM is much harder than for most open source projects, and the site suffers because of it.

      If PM had all of its code in CVS, and a clear installation procedure, and a test suite (with fake data) to tell you if you broke something, there would be a lot more people helping out with the code. I'm not sure there's any way to get there without a complete rewrite, as Liz suggested, and I share her doubts that anyone will manage to find time to do that. A migration to a later version of Everything might be easier, but I'm not certain of that.

        I think that we are mostly in agreement.

        I agree that a complete re-write is not on the cards, for the reason you gave of time, but also because I think that starting from scratch would be throwing away a lot of good code which is just wasteful. Given this isn't going to happen, what are the alternatives?

        I'm not sure that migration to Everything 2 would be helpful either. Whilst it would make it easier for individuals or groups to set up replications of PM, there would still be a whole lot of customisation in the core, and sensitive data that would be impossible to share openly and very difficult to mock up.

        Even if this were done, it would still mean that contributions from outside would need to be submitted to PM with a promise of "I've tested it thoroughly and it's fine!", which isn't going to work. The gods would still need to inspect for backdoors and malicious failures and test to their satisfaction. The same bottleneck would exist.

        The only way I could see of alleviating the bottlenecks in the testing and approval mechanism -- beyond recruiting 100 new gods (Maybe from the Indian subcontinent, they seem to have more than their fair share:) -- is to make it possible for PMdevers to test their own code in a realistic, but non-critical environment, and in a way that allows the gods to verify the varacity of their testing (by inspecting the logs of the test system to check for errors, the number of times the modification has been exercised etc.). The only way I can see of doing that is for the test environment to be in the same box and sharing the same (live) data.

        Obviously, ad-hoc changes to the live system aren't desirable, so logic led me to suggest a test server with limited bandwidth/cpu accessing the same data except for updates.

        From my, very limited, external viewpoint, this is the only possibility that addresses the problems. The alternative is to stick with the status quo, which while an option, and currently the only game in town, is the reason for the disquite in the first place.

        The idea is far from unique. Having test systems that referenced live databases read-only and wrote updates to a different database or a seperate table within the database was once common practice when disc storage was too expensive to replicate whole databases willy-nilly. This is just an extension of that idea attempting to work around the specific PM peculiarities. It cost nothing, except a little of my time to write it up, and a little of your time to read.

        I don't have a good enough view to know if it is feasible or practical, but I thought it worth mentioning anyway.


        Examine what is said, not who speaks.
        "Efficiency is intelligent laziness." -David Dunham
        "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
        If I understand your problem, I can solve it! Of course, the same can be said for you.

        One thing I find interesting is the idea that moving code out of the database would improve things. I dont think it would. In fact my tendency is to go totally the other way. Consider that in order to deploy changes to PM that arent in the DB we need to ssh into at least two boxes, upload the perl modules and then force a server restart. Wheras we can make on the fly changes to in-db code and have it automatically deployed seamlessly.

        I really dont think having the code in CVS or equivelent would particularly helpful nor do i think it would increase the number of contributors. On the contrary in fact. One aspect of the design of PM (and Everything) is that nodes are both suboutines and objects. CVS'ing the code would severly impact on the objectness of the code. Consider something like patch display page. Out of context from the monastery that code means pretty much nothing. In context its a window into the soul of the system. Putting it in CVS would lose the important part. It would be kinda like taking a window with a fine view and sticking in a warehouse and then wondering why it didnt look as good.

        A bunch of us in pmdev have even discussed moving the entire Everything code base into the DB and the boot strapping from that. We wont ever do it of course but the fact that we even think its a good idea suggest that there is something to this point.

        As a last aspect, PM itself is a stones throw away from CVS anyway. We can currently cross diff patches both on a single site or between the two. We can view a nodes patch history, and etc. We can and will expand these features as well. pmdev is very much alive and functional these days, and i dont think it would be if it was that horrible to work with.


        ---
        demerphq

          First they ignore you, then they laugh at you, then they fight you, then you win.
          -- Gandhi


Re: Re: Total speculation?
by bart (Canon) on Oct 03, 2003 at 08:58 UTC
    Well the prerequirement would be that the people rewriting the new engine should be intimitely familiar with the current implementation... Otherwise they'd surely make the same mistakes all over again. If you can't learn from history, you're bound to repeat it. Which brings us back to the original problem as brought up by theorbtwo... (here)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://295876]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (5)
As of 2024-03-29 08:49 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found