Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Having a 'Lab Rat' User Setting to minimize the impact of live development

by dws (Chancellor)
on Apr 26, 2002 at 18:35 UTC ( [id://162360]=monkdiscuss: print w/replies, xml ) Need Help??

Lately we've seen some site features being debugging in public, with mandatory, no-prior-warning audience participation. Often, there is no indication in the development wikis of who is doing what. This results in a bit of grumbling the the CB.

I'm wonding if the pain might be eased if there were a "Volunteer Lab Rat" user setting. The setting would be queried when forming pages -- only volunteers would get to see unfolding experiments. Once development is believed to be stable, all users get it. This would require a bit more coding on the part of developers, but should minimize site-wide breakage.

Causual users, what do you think?

Developers, what do you think? Would such a thing be an undue burden on development?

Replies are listed 'Best First'.
(crazyinsomniac) Re: Having a 'Lab Rat' User Setting to minimize the impact of live development
by crazyinsomniac (Prior) on Apr 26, 2002 at 20:30 UTC
    It's usually god's who debug in public, and where I am god, I usually debug on my special restricted little play node, where I do whatever crazy thing I want.

    Your idea is logical, but not practical. Way too much overhead which a few gods could make obsolete by simply debugging stuff on their private nodes ... I wonder why they haven't done so already (like when tye tested out his new ticker, it worked great for about a minute, but he, as a god, could've kept the rest of the pm out of the test) bad example, couldn't think of anything recent and applicable (this wasn't, so i don't know )

    Anyway, the latest versions of everything2 have something like this kinda built in (there are "workspaces" to which nodes can belong to, and usually only one is accessible by regular users ~ like everyone except root), but that's besides the point (JavaJunkies is taking the e2 engine down, cause, well, of performance issues ~ probably related to the gigantic OO architecture of e2).

    update: added stuff above, and below

    as you know, pmdev'ers can only submit patches. only gods have the ability to actually change stuff on the server. pmdev'ers should not rely upon the gods to catch syntaxt errors and such. I do miss the point sometimes (quite sometimes some might argue), and everything tye says below is true.

     
    ______crazyinsomniac_____________________________
    Of all the things I've lost, I miss my mind the most.
    perl -e "$q=$_;map({chr unpack qq;H*;,$_}split(q;;,q*H*));print;$q/$q;"

      like when tye tested out his new ticker, it worked great for about a minute, but he, as a god, could've kept the rest of the pm out of the test

      Interesting choice of examples.

      by simply debugging stuff on their private nodes

      We do that all the time! It doesn't solve the whole problem.

      The fact is that the new chatterbox xml ticker was developed for several weeks as a separate node and was tested in a number of different ways. Eventually, you have to move beyond unit testing, integration testing, and system testing and go to alpha/beta testing. As this thread highlights, we don't have a set of alpha- nor beta-test users. But the biggest problem with the chatterbox xml ticker was that it used too much CPU and this probably would not have been noticed if it were only used by a small set of 'lab rats'.

      So in this particular case, your suggestion wouldn't have done any good -- the problem would still not have been found until it was put onto the live site. Sure, you can develop performance testing scenarios. And after that problem I did write and use some performance unit tests for that particular feature. But I have no plans for developing a performance system test with similuated visitors using a separate group of three machines that I expect pair.com to provide for free. Sorry, "we" don't have the budget for that.

      I've done quite a bit of work to extended the options for testing PM changes. I've set up a test environment for system testing for changes to the *.pm files that PM uses. We are currently testing several new features (for example, new perl monks user search -- this link will break and then probably go away when the testing is finished and we roll it into production).

      ar0n is working on setting up a backup database. Sounds easy, and it should be, but I won't go into the many problems we've already run into trying to get this simple tool working. If we got paid to do this full time, we'd probably have worked this all out in a day. Be we aren't and we don't so we didn't.

      I have plans for changing how "bad" HTML is stripped that will address several current problems (it won't be stripped any more, it will be escaped and appear as text). For this I'll need to have it take effect for only some users so that I can properly test it. I'll be making myself a 'lab rat' by adding the 'lab rat' idea for just this specific feature. I'm doing this because it is worth the effort in this particular case.

      Making the 'lab rat' idea (creating a group of beta-test users) work in a more general way is quite a bit of work and, in my opinion, isn't nearly worth the effort at this point, dws. It may be implemented slowly, in pieces when motivated by certain changes.

      Yes, we need more testing options. We know this, and that is why we've been working on creating them. We need a good super search that doesn't make the site totally unstable. We've been working on that as well. We've even made some progress on several fronts there. We've tried several things that didn't pan out. We've started working on several other things that aren't finished yet.

      I feel bad because I recently made a very small change that had no effect on most of the site (to improve the appearance of some parts while improving support for some less-common browsers). It also broke the appearance of several parts of the site that I went about fixing. I didn't expect this to take too long or I wouldn't have made the changes in the way that I did. I was quite surprised when this small change tickled a rather unfortunate bug in NetScape 4.something which caused severe disruption in a few areas of PM (if you were using NS4.x). I ended up spending quite a long time resolving issues related to this and often wondered if it wouldn't be less work to back out what I'd done so far than continue forward.

      If I'd had any idea that the change would have lead to such problems, or would have realized the magnatude my mistake sooner in the process, then I would have done things much differently. I try to do the level of testing, isolation, announcement, etc. that seems appropriate for the type and size of change being made. I try to be conservative. But there are a lot of things that are still very difficult to test. We're working to reduce them, but not to the exclusion of all other activities.

      As I recall, the last patch you submitted, crazyinsomniac, contained a syntax error and this was very soon after you'd scolded some of us about how trivial it is to test for those. In that case, I actually did that work for you and tested your patch before I applied it.

      So it is easy to look from the outside and wonder why we don't do a better job. We will continue to make mistakes. A lot of the work we are doing now is the kind of background work that is needed so that we can provide real improvements with an appropriate amount of risk, so you probably haven't notice much of the "improvements". I'm sorry, I wish I had more time to create more summaries of what is going on behind the scenes. I fight the fires that look the most important, look the easiest to put out, look like the most fun, etc. And public announcements is just another fire (along with my real job, my family, etc.).

      Finally, thanks for the ideas and suggestions. I read everything in Perl Monks Discussion. I'm sorry but, if an idea comes up that I can't use (or that we've already tried), then I often won't take the time to explain why I can't use it. For one thing, it often isn't easy (for me, anyway) to explain. But I don't want people to think I'm ignoring the input. I appreciate it.

              - tye (but my friends call me "Tye")
Re: Having a 'Lab Rat' User Setting to minimize the impact of live development
by talexb (Chancellor) on Apr 26, 2002 at 18:58 UTC
    Great idea -- if it's feasible.

    That way, if there's an hour or more to stroll around the site, nibbling on various nodes, the occasional Odd Thing won't intrude too much. OTOH if you need to look something up fast, an Old, but it works setting would be preferable.

    --t. alex

    "Nyahhh (munch, munch) What's up, Doc?" --Bugs Bunny

    The Old, but it works setting was for the web site and :) wasn't directed at anyone. (Like me, who graduated from high school in '75 and has been fiddling with computers ever since. Happily, so it seems.)

Re: Having a 'Lab Rat' User Setting to minimize the impact of live development
by sparkyichi (Deacon) on Apr 26, 2002 at 20:04 UTC
    I would be willing to donate some funds and maybe some equipment to supporting a test server. I don't think that we are setting a good example by doing "code tests" in a production environment, whatever the precautions that have been taken. I am sure that there are some others here that would support such an endeavor.

    Sparky
    FMTEYEWTK
Re: newrisedesigns Having a 'Lab Rat' User Setting to minimize the impact of live development
by newrisedesigns (Curate) on Apr 26, 2002 at 19:45 UTC

    I would sign up to be a lab rat, considering I probably have little to contribute to hardcore coding of a new feature.

    With a 'lab rat' setting, all users, regardless of Perl hacking skill can be active in the development process.

    John J Reiser
    newrisedesigns.com
Re: Having a 'Lab Rat' User Setting to minimize the impact of live development
by ignatz (Vicar) on Apr 30, 2002 at 20:04 UTC
    For me part of the value that I get for the price of admission to PM is the entertaining floor show when a "GOD" somehow skrews up something on the site. Suddenly the chatterbox lights up with everyone chipping in to put out the fire. Kinda like driving a quirky sportscar. Everyone bending over the hood helping to fix it is part of the experience.

    Not the kosher way to do things for a commercial site, but this is a community run site where everyone should lend a hand cleaning up when needed.

    ()-()
     \"/
      `                                                     
    
Re: Having a 'Lab Rat' User Setting to minimize the impact of live development
by dthacker (Deacon) on Apr 30, 2002 at 21:51 UTC
    As a casual user, I think it's pretty interesting when the site behaves in unexpected ways. I don't see the need to be warned. It's not like I'm paying for my fun here.

    "Ready! Fire! Aim!"-Tom Peters

    Dave


    Code On!

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: monkdiscuss [id://162360]
Approved by coreolyn
Front-paged by particle
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others sharing their wisdom with the Monastery: (4)
As of 2024-03-28 21:18 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found