Re: Re: Re: Re: The costs of packages

[BrowserUk]

With the scheme I outlined, the packages would only come into existance when they are used, generated (eval'd) by the one small AUTOLOAD routine that gets loaded by the original use statement with the detail of each individual type being created based on information from a lookup table (hash) filled from the __DATA__ file.

The information in the DATA section will itself be autogenerated from several other very large files (think OS C header files) and reduced to the minimum info I require.

My only reason for creating the minimalist packages for each type were

a) To route the initial calls to new AsYetUndefinedType; back to the single Superclass AUTOLOAD generator.

b) To ensure that only known types would be auto-generated. Unknown types, typos, etc. wouldn't have a stub package in existance, so they would cause the usual

Can't locate object method "new" via package "XTYPE" (perhaps you forg
+ot to load "XTYPE"?)

Abigail's neat use if the UNIVERSAL package (which I was only vaguely aware of) neatly side steps the whole issue by removing the need for the hundreds of minimal (stub) packages whilst retaining the simplicity of syntax and avoiding the need for predeclaration. The only caveat I can see is that it places the burden of verifying that the types being autovivified are known types on my code rather than the compiler. Ie. run-time verification, rather than compile-time, but that is a small price to pay for the benefits it produces.

The original question was just trying to clarify what performance impact having hundreds of unused minimal packages floating around would have, and whether any such impact would be reduced by moving those minimal packages below a single, top level namespace, rather than each existing in the top-level namespace.

Every time you call a subroutine, perl has to lookup the name of the sub to find the address to transfer control to. Remembering that goto &subname; is not itself the quickest code on the block, I was concerned that maybe adding large numbers of unused names to the top-level namespace would have a detrimental performance impact on the whole program and wondered (aloud) if this might be alleviated by keeping most of them in a different namespace.

Each package also uses memory. It's easy to demonstrate that creating 10,000 hashes with 1 key/value pair each requires (on my machine with 5.8.1) around 5 MB of memory, whilst creating 1 hash with 10,000 key/value pairs only requires 1 MB of ram. From my inspections with the debugger, it would appear that there are several hashes, each containing several nested hashes created for each package one loads. Each hash appears to contain very little, but as shown above, each hash carries an overhead.

If I can reduce the overall memory consumption by nesting the generated packages one level down in the top level hash by putting many keys/value pairs within a single hash rather than creating lots of small hashes, I consider this to be worthwhile doing. If that has the knock-on effect of reducing the namespace pollution at the top level, and with it, the lookup time for every every function call in the program, then I consider it even more worthwhile.

However, from my inspections and experiements, it is not at all clear to me whether my concerns are real or not. Hence my questions and the hope that someone browsing by with a better understanding of these things might be able to clarify them for me.

I never consider it premature to think about the overall impact of my designs upon the programs that will use them. I beleive it to be good design to do so. Only by thinking about such things do you avoid the 'bloat' syndrome that we all love to hate in Word and other similar programs.

Abigail's neat solution with the UNIVERSAL package bypasses these concerns and results in a solution that from my perspective has almost no downsides. Having this brought to my attention completely vindicates both my having thought about the impact and asking the question.

---

Examine what is said, not who speaks.

"Efficiency is intelligent laziness." -David Dunham
"When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
If I understand your problem, I can solve it! Of course, the same can be said for you.

[demerphq]

Ie. run-time verification, rather than compile-time, but that is a small price to pay for the benefits it produces.

But an entirely perlish approach :-)

The original question was just trying to clarify what performance impact having hundreds of unused minimal packages floating around would have, and whether any such impact would be reduced by moving those minimal packages below a single, top level namespace, rather than each existing in the top-level namespace.

And its an interesting one that Im glad you raised as I, for one, have never thought about this. Of course theres an overhead but it never ocurred to me to think about it.

I never consider it premature to think about the overall impact of my designs upon the programs that will use them. I beleive it to be good design to do so. Only by thinking about such things do you avoid the 'bloat' syndrome that we all love to hate in Word and other similar programs.

No I agree. But there is taking it too far, and for me I think my counter-question was just as valid as yours, and still not entirely resolved. :-) A lot of times a person with a completely fresh view is able to spot the "flaw" in a plan. Having had people ask me to justify some interesting optimisation on occasion and not being able to come up with a good answer more than once says to me that its always fair to second guess such thinking. However this does not mean that I think your question shouldnt have been asked, just that this was a fair response.

Abigail's neat solution with the UNIVERSAL package bypasses these concerns and results in a solution that from my perspective has almost no downsides. Having this brought to my attention completely vindicates both my having thought about the impact and asking the question.

Agreed, its been a thought provoking thread. However, I still stand by the comment that any time you start getting obsessed with optimization, its worth a deep breath and to ask yourself if its really needed... You just have to admit that on occasion it is... :-)

As somebody once said, theres a fine line between premature optimisation and mature optimisation. :-)


---
demerphq

_{<Elian> And I do take a kind of perverse pleasure in having an OO assembly language...}