Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Memory leak

by aplonis (Pilgrim)
on Jan 13, 2005 at 20:13 UTC ( #422088=perlquestion: print w/replies, xml ) Need Help??
aplonis has asked for the wisdom of the Perl Monks concerning the following question:

I have a huge, gigantic, 5,000-line Perl/Tk program which edits even more monstrous data files. When I call it up and run the first file it goes okay. If I close Perl/Tk and open again, no matter now many times, no problem. But if I leave it open and edit successive files, the memory dips ever lower. So somewhere I'm holding onto data, I'd guess. I'd like to know where.

Are there any tools, perhaps in the debugger, that will show me how many vars and arrays I have and what size each is so I'll know where my problem lay?

Thanks,

Gan

Replies are listed 'Best First'.
Re: Memory leak
by Joost (Canon) on Jan 13, 2005 at 20:33 UTC
Re: Memory leak
by zentara (Archbishop) on Jan 13, 2005 at 21:34 UTC
    The first thing to look for is trying to empty widgets by undef'ing them. When you edit successive files, are you undef'ing the old text widget, and creating a new one for each file? That would be my first guess without seeing your code. If it is correct, you should reuse the same text widget, just empty it and delete all tags and text, before loading the next file.

    I'm not really a human, but I play one on earth. flash japh

      The editor is for automotive test equipment such as you see rattling cars in TV commercials. They run on road load data files, recordings from the test track. My editor is described and downloadable via this link.

      I do open and close a fair number of windows. To close, I destroy them at top-level like so...

      $mw->destroy() if Tk::Exists($mw);

      ...with the assumption that all said top-level's child widgets are likewise destroyed. Then later I rebuild them when needed. Does destroying a widget not return all memory from them and their children? Is that my problem, or something else?

      Thanks,
      Gan

        I downloaded and ran your program and it produced a bunch of warnings. Most of them are probably benign, at leats with respect to your memory problem, but one group caught my eye and tweaked somthing in my memory:

        Variable "@perl_widgets" will not stay shared at P:\test\422088.pl lin +e 3475. Variable "@checks_antiflats" will not stay shared at P:\test\422088.pl + line 4999. Variable "@antiflats_cmd" will not stay shared at P:\test\422088.pl li +ne 5005. Variable "@antiflats_nb" will not stay shared at P:\test\422088.pl lin +e 5035. Variable "@antiflats_ms" will not stay shared at P:\test\422088.pl lin +e 5056. Variable "@checks_antiflats" will not stay shared at P:\test\422088.pl + line 5076. Variable "$frame_ps_options" will not stay shared at P:\test\422088.pl + line 5087. Variable "$frame_ps_options" will not stay shared at P:\test\422088.pl + line 5104. Variable "$frame_ps_options" will not stay shared at P:\test\422088.pl + line 5122. Variable "$frame_ps_options" will not stay shared at P:\test\422088.pl + line 5138. Variable "$pythag_flag_last_time" will not stay shared at P:\test\4220 +88.pl line 5163. Variable "$frame_ps_options" will not stay shared at P:\test\422088.pl + line 5169. Variable "$pythag_flag_last_time" will not stay shared at P:\test\4220 +88.pl line 5192. Variable "$frame_ps_options" will not stay shared at P:\test\422088.pl + line 5218. Variable "$frame_ps_options" will not stay shared at P:\test\422088.pl + line 5318. Variable "@antiflats_nb" will not stay shared at P:\test\422088.pl lin +e 5324. Variable "@antiflats_ms" will not stay shared at P:\test\422088.pl lin +e 5325. Variable "$frame_ps_options" will not stay shared at P:\test\422088.pl + line 5349. Variable "@antiflats_nb" will not stay shared at P:\test\422088.pl lin +e 5366. Variable "@antiflats_ms" will not stay shared at P:\test\422088.pl lin +e 5370. Variable "$frame_ps_options" will not stay shared at P:\test\422088.pl + line 5422. Variable "@antiflats_nb" will not stay shared at P:\test\422088.pl lin +e 5428. Variable "@antiflats_ms" will not stay shared at P:\test\422088.pl lin +e 5429.

        The diagnostics for this warning reads:

        Variable "@perl_widgets" will not stay shared at P:\test\422088.pl lin +e 3475 (#4) (W closure) An inner (nested) named subroutine is referencing a lexical variable defined in an outer subroutine. When the inner subroutine is called, it will probably see the valu +e of the outer subroutine's variable as it was before and during the *f +irst* call to the outer subroutine; in this case, after the first call t +o the outer subroutine is complete, the inner and outer subroutines will + no longer share a common value for the variable. In other words, the variable will no longer be shared. Furthermore, if the outer subroutine is anonymous and references a lexical variable outside itself, then the outer and inner subrouti +nes will never share the given variable. This problem can usually be solved by making the inner subroutine anonymous, using the sub {} syntax. When inner anonymous subs tha +t reference variables in outer subroutines are called or referenced, + they are automatically rebound to the current values of such variables.

        I wonder, without being able to find anything to confirm it, if these unshared closures are causing (part of) your memory leak.

        As I say, I'm not sure if this could cause such a problem as I have never really understood the diagnostic, but it might be worth your further investigation.


        Examine what is said, not who speaks.
        Silence betokens consent.
        Love the truth but pardon error.
        Does destroying a widget not return all memory from them and their children? Is that my problem, or something else?

        That's your problem. I was burned by the same thing when I built my first "big-long-running" script. If there is one rule I can summarize...you have to REUSE all widgets in Tk, or there will be leaks. Somewhere deep down in the innards of Tk, references to all objects are kept, and not released until the end of the program. I know it is a pain to deal with, but once you learn the tricks, it isn't too bad.

        For instance, you say you open a fair number of windows, then destroy them....that is memory use going up. What you want to do, is pre-create a set of reusable top-level windows, each packed with the widgets they need (which will also be reused), and "withdraw" them. Now, when you need to load them with data, loop through them all, and "configure" each of them with the new data, then "raise" the toplevel. When you are done, don't destroy the toplevel, just "withdraw" it. When you need the next toplevel, just loop through the withdrawn toplevel(and it's widgets) and "reconfigure" them with the new data, then "raise" them again.

        I have made some big programs run this way, loading, unloading, and reloading data, without memory leaks. The memory will increase to the size of the largest "instance" of the toplevel, but will be reused, and generally will level out at some average value.

        Photo objects can be "emptied" with $photo->blank, text widgets can be blanked with $text->delete('1.0', 'end'); etc, etc.

        I've found almost all widgets can be reused, without leaking, with the exception of the Progressbar.

        Most Tk apps are small, and tend to do something then exit; so the small memory gains, created by using "destroy", can be ignored. But long running scripts have to be caarefully planned.


        I'm not really a human, but I play one on earth. flash japh
Re: Memory leak
by perrin (Chancellor) on Jan 13, 2005 at 20:54 UTC
    If you edit the exact same file again and again and it chews up more memory each time, that's a problem. If you keep opening bigger ones and it chews up more memory, that's normal.
Re: Memory leak
by dragonchild (Archbishop) on Jan 13, 2005 at 20:29 UTC
    I would suggest code inspection for improperly-scoped variables. Seriously, 5000 lines isn't that much.

    Being right, does not endow the right to be rude; politeness costs nothing.
    Being unknowing, is not the same as being stupid.
    Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence.
    Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://422088]
Approved by Paladin
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (8)
As of 2018-07-17 15:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    It has been suggested to rename Perl 6 in order to boost its marketing potential. Which name would you prefer?















    Results (372 votes). Check out past polls.

    Notices?