Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses

Re: Multithreaded process on AIX slow

by talexb (Canon)
on Nov 14, 2013 at 13:42 UTC ( #1062577=note: print w/replies, xml ) Need Help??

in reply to Multithreaded process on AIX slow

Interesting problem. The application without the REST piece works OK -- and the REST piece works OK, but put them together and the whole thing slows down. That's my understanding of the problem.

I wonder if there's an issue in the middle -- is there a way to limit how many REST requests are in flight? If so, you might see how things go by limiting that number to see if you get better performance. Alternatively, is there some resource that a REST request uses in the application that's in short supply?

Alex / talexb / Toronto

Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

Replies are listed 'Best First'.
Re^2: Multithreaded process on AIX slow
by scunacc (Acolyte) on Nov 14, 2013 at 14:59 UTC

    Hi Alex.

    Thanks for the thoughts. Essentially you are correct in your 2nd sentence.

    I do have a standalone version of the very simple REST piece I could try hammering away with as fast as possible in parallel I guess. I should do that to eliminate the cross-fertilization with the threading on AIX I suppose, but, it needs to work in this server, otherwise I'll have to use the Linux port instead which is not ideal in this situation but might be workable. (The server communicates with another similar (but-non-REST-call-making) server to store the results from the REST calls in a database among many other tasks, but that second server currently runs on the AIX box where it needs to sit - and does so very smoothly presently. However, they *are* on the same net and it is all interoperable - just that it means more net interactions between the two machines instead of having both servers on the same machine.)

    And, yes, if I reduce the # of REST calls by batching as I've mentioned, it works very quickly - quicker even than the Linux version.

    The REST resource is not in contention. + I have short-fuse alarm timeout handlers using eval built around the calls so if they do fail it bombs gracefully and moves on to the next call.

    Kind regards


Re^2: Multithreaded process on AIX slow
by Anonymous Monk on Nov 14, 2013 at 14:20 UTC
    Is the REST code using some kind of semaphore internally? You've been running this same code, with REST, on other machines and it's only slowing-down on one?

      Appreciate the further thought. All good to help the juices flow :-)

      Identical code - yes. I guess I need to go look at the actual REST::Client and HTTP::Request CPAN module sources to see what they are doing to answer that.

      I know when I developed the surrounding code back in 2008 I discovered a bug/feature in the Perl threading mechanism and variable sharing (reported back on it) that had me going in circles for a while, but led to some interesting workarounds that ended up making the code more efficient as a result. That isn't related to this though. I suppose I'll just have to dig through those net modules and see as well on this. Oh well :-) All good fun.

      Kind regards


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1062577]
[corenth]: Good morning (in my neck of the woods)
[Cosmic37]: hello perly kings and queens
[jedikaiti]: 'ello Monks
[Cosmic37]: and monkeys too...
[Cosmic37]: is anyone doing machine learning with Perl?
[Cosmic37]: I was using gneural code in C and it was interesting but did not have back propagation for neural networks
[corenth]: Cosmic, poking at it, but not really.
[Cosmic37]: I read that its possible to use the FANN library with Perl so I might try that now...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (10)
As of 2018-02-20 18:11 GMT
Find Nodes?
    Voting Booth?
    When it is dark outside I am happiest to see ...

    Results (274 votes). Check out past polls.