Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re: My code randomly stops working

by sundialsvc4 (Abbot)
on Mar 27, 2013 at 14:58 UTC ( #1025732=note: print w/replies, xml ) Need Help??


in reply to My code randomly stops working

Here are a couple of CPAN modules that you should look at just for their descriptions:   Test::Memory::Cycle, and Scalar::Util, the latter specifically in reference to weaken.

It certainly appears to us that the root cause of the problem is that the program is exhausting memory.   Most likely, the system begins to “thrash” as a result of this issue, and that’s physically-speaking what actually brings it to its knees.   One subtle problem that I am thinking of, hence the first paragraph, is that of circular memory-references which prevent storage from being cleaned up successfully after it has been released.   But, another possible problem is that storage structures such as hashes aren’t emptied before the start of the next run ... e.g. by use of undef.

One fairly easy-to-do structural rehab that I have sometimes tried, with programs that need to process a lot of files in sequence, is to put the file-scanning loop as the main program, and have it call a subroutine for each file.   All of the data structures needed are made local to that one subroutine, so that everything goes away “in one swell foop” when the subroutine ends ... provided that there are no circular storage-references in play.   (If any globals are needed, the subroutine in its first few statements explicitly undefs each and every one.)   This is a fairly simple re-factoring to do with most existing, hairy programs, because it doesn’t attempt to restructure the main-line logic ... it merely puts it into a box.

Amazing!   A down-vote, not five seconds after this posting was made!   Is anyone out there, like, running a “bot” that will do this?   Just askin’ ...   I find it quite entertaining, really.   Pathetic, but laughable ...

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1025732]
help
Chatterbox?
[ambrus]: Corion: push_write is in the higher level abstraction of AnyEvent::Handle, not in the watcher
[Corion]: ambrus: Hmm - rereading Prima::File, that merrily coincides with what Prima does - it tells you "you can read", and you're supposed to read from the fh yourself. I thought it called you with the data already read, which would've been harder to integrate
[ambrus]: you just need an io watcher, created by &AnyEvent::Impl:: Whatever::io(...)
[Corion]: So after talking it through with you even while I'm still not entirely clear on where AE ends and my implementation begins, I think I understand that I only need to implement some smaller parts for each functionality I want to support.
[Corion]: Yeah... and you might even be able to mix and match additional functionality if you have additional async suppliers, like from a separate thread
[ambrus]: You hvae to be careful with the timer, because apparently Prima::Timer insists on being periodic, wheras AnyEvent::Impl:: Whatever::timer should give a one-shot timer watcher

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (6)
As of 2016-12-08 12:23 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    On a regular basis, I'm most likely to spy upon:













    Results (141 votes). Check out past polls.