Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Recursive locks:killer application. Do they have one?

by BrowserUk (Pope)
on Feb 02, 2012 at 04:01 UTC ( #951340=perlquestion: print w/ replies, xml ) Need Help??
BrowserUk has asked for the wisdom of the Perl Monks concerning the following question:

There is a school of thought, to which I think I am rapidly coming, that suggests that recursive locks are not just unnecessary, but actually dangerous.

My question is, does anyone know of a particular algorithm or use-case that would be impossible or difficult to implement without recursive locks?

Opinions; examples; references; all sought.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

The start of some sanity?

Comment on Recursive locks:killer application. Do they have one?
Re: Recursive locks:killer application. Do they have one?
by mbethke (Hermit) on Feb 02, 2012 at 08:14 UTC

    I agree, although I find the following comment on StackOverflow quoting Dave Butenhof more convincing, the one that basically goes "if you think you need this, you're doing it wrong".

    That said, the same goes for the whole multithreading paradigm, at least in scripting languages. Separated address spaces were invented for a reason, namely to keep people from shooting themselves in the foot in one of the many ways possible without them---I've used an Amiga for a long time and probably succeeded at most of these. I know a few people that I believe know what they're talking about when they say they need multithreading because everything else would be even more complicated, but the vast majority of programs and everything written by people who think multithreading was easy would be better served by a decent fork() (OK, sucks to be on Windows ...) or a state machine.

      I find the following comment on StackOverflow quoting Dave Butenhof more convincing, the one that basically goes "if you think you need this, you're doing it wrong".

      That said, the same goes for the whole multithreading paradigm ...

      I'm guessing you didn't take the time to look up Dave Butenhof's main claim to fame?


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

      The start of some sanity?

        I'm guessing you didn't take the time to look up Dave Butenhof's main claim to fame?
        No, I didn't have to. "The one quoting David Butenhof" was just meant to help identify the comment, not to say "I agree with it because Butenhof said it". I happen to find this particular argument of his convincing, no more, no less.
Re: Recursive locks:killer application. Do they have one?
by roboticus (Canon) on Feb 02, 2012 at 11:34 UTC

    BrowserUk:

    I used to do a good bit of real-time programming (motion control systems) and haven't used recursive locks. I once thought to use them, but a more-experienced co-worker gave a similar argument against them, and suggested an algorithm change. Since then, I hadn't come up with any need for them. It's pretty convincing to me.

    ...roboticus

    When your only tool is a hammer, all problems look like your thumb.

Re: Recursive locks:killer application. Do they have one?
by sundialsvc4 (Monsignor) on Feb 02, 2012 at 14:00 UTC

    I have never found it necessary to use such a mechanism either.   Also, I think that such niceities, well-intentioned though they might be, might actually tend to create less-debuggable code simply because they are more permissive than they strictly “need” to be.   One of my mentors referred to it as (and I always loved this expression...) “Coding for WTF?!?!”   The notion being that, the more things that are allowed to occur, the more possibilities to have to sift through when you are faced with an evident bug but don’t yet know where.   Multiprocess code is already notoriously hard to debug, without “help” from the OS.   :-)

    It is appealing, maybe, to somehow pawn “doing the right thing” off to the operating system ... but “a general case” is hard to consider here, which means that the OS code starts filling up with decision heuristics, which might match your case or simply be an unwanted degree-of-freedom in the mechanism that you are constructing.   If you need a mechanism like this one, you can construct it using simple atomic primitives.   (And if you do, you’d probably want to have an “already locked by this process” error-code from the OS primitive.)

Re: Recursive locks:killer application. Do they have one? (mu)
by tye (Cardinal) on Feb 02, 2012 at 21:28 UTC

    I guess I'm in more of the opposite camp. The example problem given tells me that they are "doing it wrong", but not for using a re-entrant mutex. If you have a class where "A::foo() acquires the lock. It then calls B::bar()" then you are already holding the lock too long. The mutex being non-reentrant isn't going to point this out to you. B::bar() might decide to do something that blocks or that tries to acquire some other lock and then you've got lock-acquisition order to worry about which leads to deadlock problems.

    I've seen tons of code that uses re-entrant mutexes and isn't "doing it wrong". That example is more like: you have a class that mostly just deals with the bits that need to be under a specific mutex. So the code to be run under the mutex is kept very small and cohesive by being its own class that just concentrates on doing the locking right.

    And the re-entrant mutex comes in because you have methods that mostly can tell that they don't need to grab the mutex and so don't most of the time. So, since you are only grabbing the mutex in the rare cases when you need it, you can easily end up with a simple and clear utility method that might be called in a context where the mutex isn't held and also in contexts where the mutex is held and the utility function might (even indirectly) only rarely decide that it needs to hold the mutex.

    You can get around that by splitting any such method into two methods, say doFooUnlocked() that just does the work and doFooLocked() that just holds the mutex and then calls doFooUnlocked(). Then doFooUnlocked() might be declared such that it can only be called from within the class. Then, if you already are holding the mutex, you need to call doFooUnlocked().

    But, that solution requires the bifurcation of all methods that might call doFoo*() which can lead to quite a mess.

    But this type of concern mostly only pops up when doing the style of threading + locking that Java pretty much encourages and I find that that is an approach that is just way too easy to end up becoming an unreliable mess after it tries to scale in the feature set supported. So I don't do anything like that these days.

    I wish I had a much more concrete example handy but it has been too many years since I was doing that type of work (in C++).

    - tye        

      But this type of concern mostly only pops up when doing the style of threading + locking that Java pretty much encourages ...

      Pre-1.5 Java is certainly the poster bot for recursive locks -- synchronized blocks -- which also makes it the poster bot for all that is wrong with them.

      Hence the 1.5 moves to add finer grained locks. Though I think they went too far the other way with the need to explicitly unlock.

      I like perl's current mechanism -- locking data rather than code blocks -- combined with it's semantics -- automatic unlocking at the end of the encompassing block. What I dislike is the overhead of current implementation with its need to count and no timeout.

      Then, if you already are holding the mutex, you need to call doFooUnlocked(). But, that solution requires the bifurcation of all methods that might call doFoo*() which can lead to quite a mess.... I wish I had a much more concrete example handy.

      Ditto the last part. Whilst appreciating that your example is abstract, it doesn't sound right to me.

      IME, the whole idea of needing to retain a lock long enough to call out to another method just seems wrong to me. If what you need to do with the thing you are protecting is sufficiently complex to warrant calling a subroutine to do it, then your design strategy is all wrong. Like declaring variables, locks should be taken as late a possible and in as small a scope as possible. And that means they should not be held across function/method call boundaries.

      I'm still open to the idea that there exist algorithms that require recursive locks, but until I find a concrete example that I can't re-write to not use them -- without having to jump through hoops to do so -- I'm pretty settled on the notion that they should only be used as a last resort rather than a first.


      For the most part, I think most of the bad press surrounding locking comes simply from bad programming. And most of that comes down to the belt & braces conservatism of applying locks to everything; too often; and fundamentally, for too long.

      There is also a lot of bad academic research floating around. So called 'classical concurrency problems'. Things like this. There are reams and reams of academic treatise attempting to come with provably correct algorithms to deal with this, but not one of them suggests the obvious solutions. Buy five more forks. Or, eat with their fingers.

      By far the easiest way of avoiding locking problems, is to avoid locking. Not always possible, but (I'm pretty sure) it is always possible to confine locking to very short pieces of code in very small scopes.

      But the proof will be in the counter example.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

      The start of some sanity?

        What I dislike is the overhead of current implementation with its need to count and no timeout.

        I find it hard to imagine how "need to count" can have more than the most trivial of impacts on the efficiency of a mutex.

        Ah, Java switching locking schemes explains why this is such a political football, then.

        I agree with many of your points above.

        IME, the whole idea of needing to retain a lock long enough to call out to another method just seems wrong to me.

        True. But the methods I'm talking about were tiny bits of utility code. Think something like "length" or "isReserved".

        The only reason the locking would get that complicated for these things was due to being careful to only lock when locking was required. So, a huge fraction of invocations of some method would never even need to lock. When moving the "window" where the lock was held to the smallest possible scopes, those scopes would fairly often move down inside some internal utility method. This was C++ so there was more call for tiny utility methods compared to writing in Perl.

        Something like a "move" operation wouldn't have to lock unless either the source or destination was "shared". And a "clear" operation would boil down to a bunch of "move" operations with no outer lock while a "shutdown" operation would lock and then "clear".

        But these days I don't program by writing a class and then trying to insert the locks where required so that the class becomes "thread safe". I design the system to not need locks except the minimal number of key places. It is closer to "multiple processes" coding over "multiple threads" coding.

        So, instead of some object that might be shared between threads, I'd have a mechanism for transferring responsibilities between threads that would transfer simple data and end up with either two similar, separate objects or one object being destroyed and another created.

        So, when I try to put my "multiple threads" programming hat back on, I would want re-entrant mutexes (and requiring an explicit unlock sounds like a really horrid idea). But, stepping back, I'd rather just not go back to that way of thinking and instead do design that can be implemented with "multiple processes" even if the expected implementation is "multiple threads", and that makes "re-entrant or not" mostly a moot question.

        - tye        

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://951340]
Approved by davido
Front-paged by MidLifeXis
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others wandering the Monastery: (10)
As of 2014-08-20 15:45 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (118 votes), past polls