|
Speedfreak has asked for the wisdom of the Perl Monks concerning the following question:
Hej all,
For the past 5 weeks I've been writing a Perl module and its getting big, REALLY big - like 3000 lines so far.
One things that comes to mind is that there are maybe 100 object variables you can access from the module, but internally, every function has its own "my" declared variables and constants for use within that function.
Many of these are arrays and hashes that store data that is being sorted or maniputlatied before being pushed into object variables.
I've reused variable names in different functions and I'm
wondering, do variable names get reused? I mean, if I have a function A with variable my $name = "Jed"; and then in function B I declare my $name = "Ted"; then do two variables called $name exist in memory?
Guestimating, I said that my 3000 line module probably
holds maybe 128k of data in local variables and produces 512K - 2048k in object variables and I'm just a little concerned about speed and memory usage as this is called via CGI which will spawn a new instance.
Anyone got any tips or info on ways to make clean, efficient modules?
- Jed |
RE: Memory usage in Perl.
by eduardo (Curate) on Aug 06, 2000 at 19:46 UTC
|
| [reply] |
|
|
Well, to justify some of my points.
This module is huge because the data it processes is really complex. Its handling strings from another program which report up to 200 different enviromental conditions in a working environment. The string can report 0-200 of these conditions at any one time and all of the values need to be available.
Therefore, say someone wanted to know the oxygen content of the environment, they would take the environmental data string from the test equipment and send it to the module, which decodes it and returns all the factors as object variables.
If there was no oxygen data, then the variable containing its content value would be undef and there program using the module would take appropriate action to report that.
I have looked and looked again and this module and made
two or three small programs, optimised to do certain tasks and then combined.
The big problems is that the whole problem requires the decoding and handling of various text strings, something I find Perl unmatched for, however the nature of the data is what is forcing the size of the module!
I'd like to use mod_perl, but never used it before and cant find any clear details on implementing it.
- Jed
| [reply] |
|
|
whenever I hear about complex parsing, and I think about module size, performance, scalability, maintainability... I always come to the same exact conclusion... someone should check out Parse::RecDescent
There have been some awsome comments on how to use Parse::RecDescent here in the monestary, look at some of merlyn's posts... however, the quick overview is simple (if you've never worked with parsers...) Parse::RecDescent will allow you to write a grammar that describes what it is that the data should look like, and the mechanisms by which it should be dissected and manipulated. If you are having to do a lot of conditional matching (eg If this match this else match that) then it may be a perfect job for a recursive descept parser... Give it a look, the docs are awsome, and if you have questions on it, I am sure some of the more knowledgable monks (talk to maverick) will be able to give you a hand on how to dissect Parse::RecDescent. Hope it goes well!
| [reply] |
Re: Memory usage in Perl.
by tilly (Archbishop) on Aug 06, 2000 at 19:38 UTC
|
| [reply] |
RE: Memory usage in Perl.
by mikfire (Deacon) on Aug 07, 2000 at 08:10 UTC
|
I will attempt to answer your question about memory usage. In general,
perl is seriously memory hungry - in a quest for speed, it will
burn memory. That is a very general statement.
In more specific context, there are not two variables called
"name" in memory when your module is used. Perl uses garbage
collection, which is (some say) simplistically implemented
by a reference count. When a variable's reference count
goes to 0, it is "destroyed" and the memory it used is
given back to perl - not to the system - to be reused.
Assuming you haven't returned a reference to your $name,
that instance of the variable was destroyed when
function A exited.
It may be possible perl will reuse that memory space for
Function B, or it may allocate more. I do not know how
aggressive perl is in its memory recycling and it still really
depends on the flow of your code and how it is used.
You can also see perldoc perlobj ( search on garbage ) for
a better description of the collection algorithm.
mikfire | [reply] |
|
|
One detail. Lexical variables do not hand their memory
back. Instead perl assumes that it will need to repopulate
them again and keeps the memory in use to make it faster
to allocate on the next pass through. Unfortunately this
can be seen in current versions of Perl in an obscure bug
when you declare a my variable inside a one-line if or
unless. In particular this sort of thing is seriously
unwise:
sub my_func {
# Stuff
my $foo = $bar unless $cond;
# More stuff
}
Perl's behaviour in this case is IMO seriously broken,
intentionally undocumented, and as Larry Wall put it, "Use
of this feature would be erroneous." It is also far more
complex than a simple test would indicate.
I would expect it to get fixed, probably for 5.6.2. In
the meantime make sure that my statements are guaranteed
to be executed.
EDIT
If anyone wants to know more about this, here is an
explanation
of what exactly is going on, and the
original
bug report. (*ahem* Yup, that is my bug.) | [reply]
[d/l] |
(crazyinsomniac) Re: Memory usage in Perl.
by crazyinsomniac (Prior) on Aug 07, 2000 at 10:17 UTC
|
Why don't you post the specifications and your source, and maybe we can help you out a little better?
______________________________________________
|_____¸.·ooO--(> cRaZy is co01. <)--Ooo·.¸_____|
ŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻ | [reply] |
|
|
| [reply] |
|
|
