Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?

Re^6: Perl threading stability?

by guice (Scribe)
on Jul 26, 2005 at 21:13 UTC ( #478369=note: print w/replies, xml ) Need Help??

in reply to Re^5: Perl threading stability?
in thread Perl threading stability?

Not only does it sound doable, it sounds pretty straight forward.

I'm pretty sure it is pretty straight forward. My only concern is taking up too much resources. The current script (using max if 5 fork()'d children) already takes ~4-5 hours to run... Albeit some things aren't at it's most efficient, I just don't want it to take longer.

From what I'm seeing with Perl threads is that it's really CPU/Memory intensive vs general fork(). That's a lot to ask for just for the ability to share data, my primary concern. Not to mention Perl's history of being flakey with Threads (although a post below has stated it's stablalized now).

In the future I might see about writing this in Java, but at this time it's just not feasiable due to current data imports (standard Dumper dump of variables into a file). But that's not for at least 6+ months, until I can get XML inputs and data use stablized.

If you want to chat more about specs, feel free to email me I don't really want to clutter this post up with "my problem" per say. More use it to discusse perl threading in general.

-- philip
We put the 'K' in kwality!

Replies are listed 'Best First'.
Re^7: Perl threading stability?
by BrowserUk (Pope) on Jul 26, 2005 at 21:27 UTC
    My only concern is taking up too much resources.

    With appropriate care, spawning a thread can take as little as 300/400k. It's just a case of ensuring that each thread only loads what it needs. Your main concern (reading between the lines still) is that sharing a large hash between many threads will cause large amounts of duplication--and your right, if you allow the default behaviour to take it's course, it will--but there are some fairly simple techniques for getting around that in most cases.

    It really comes down to only sharing what is required a given thread with that thread. Whilst your overall application may have need to retain a large volume of data in memory, for most applications, each thread only needs access to some small part of the overall dataset, at any given time.

    As for the spec, I'm really interested in seeing how iThreads can be used. I'll drop you a note and we can move on from there.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://478369]
[Corion]: marto: Heh - currently they seem promising, but I think I'll stay with nVidia+Intel for the time being, as I've been bitten too often by bad AMD drivers
Discipulus is (still?) not a big fan of notepad++
[marto]: Corion, I think in the past this was a big problem for them. GPU driver wise the improve all the time. I use the open source drivers on my machine (old R9 270, 2GB) and had no problems
[Discipulus]: is not the problem with new hardware?
[marto]: notepad++ ticks all the boxes for me, it's a fine line between just enough, and bloat, in editors for me :P
[marto]: Discipulus]: less so than the modern intel CPUs
[marto]: also, AMD are faster at fixing the CPU microcode it would seem

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (8)
As of 2017-07-27 08:58 GMT
Find Nodes?
    Voting Booth?
    I came, I saw, I ...

    Results (407 votes). Check out past polls.