### Re: Archive::Zip performance question

by Albannach (Prior)
 on Feb 16, 2005 at 16:24 UTC ( #431626=note: print w/replies, xml ) Need Help??

in reply to Archive::Zip performance question

Well one thing that will clearly not scale well is that you are writing the entire ZIP file after each new member is added to the archive, and that ZIP just gets bigger each time. Perhaps try it with the writeToFileNamed outside the loop?

Update: I just noticed you didn't escape the period in your grep, but I imagine you probably wanted to.

--
I'd like to be able to assign to an luser

Replies are listed 'Best First'.
Re^2: Archive::Zip performance question
by holli (Monsignor) on Feb 16, 2005 at 17:08 UTC
My programming teacher in school (ahh, long ago) told us this story once:

One day Schlemihl got a new job at a company that paints the white markings on the street. On the first day Schlemihl tagged 3 miles of the street with markings. The Boss was very pleased and thought about paying him a bonus. On the second day Schlemihl did 1,2 miles and on the third day he only made 300 yards.
Whatīs up?, asked the boss. You made 1.2 miles on the first day and now you come back with wimpy 300 yards. Donīt you want to keep your job?

Well., said Schlemihl, The way to the paint bucket gets longer and longer.
Since then I know the term Schlemihl Algorithm for a algorithm that does not scale well.

holli, /regexed monk/
Just for fun, I tired to reason out how far Schlemihl walked that first day, with the following assumptions:
Each stripe is 24 inches long, and separated from the next stripe by 24 inches
The paint brush holds enough paint to paint 1 stripe
There are 1311 stripes per block (1 block = 1 mile, with 36 foot streets separating the blocks that are unpainted)

Where T = trip number, S = stripe length, I = interval between stripes
That makes each trip = T*(S*2)+I(2(T-1))
or Trip 1 = 48"
Trip 2 = 144"
T3 = 240" etc.

total trip = T1+T2+T3...

adding in the width of the streets crossed between blocks, Schlemihl walked an amazing 11,772.18 MILES to paint 3933 stripes on 3 miles of road. At a speed of no less than 1471.5 MPH! (assuming an 8 hour shift)

Assuming he could carry a bucket that held enough paint to paint 3 miles of stripes, he could paint that 3 miles in about 6 seconds.

On the other hand, his performace at a more human speed of 2.5 MPH, he would only be able to paint about 162 stripes in 8 hours (using the old method), or about 12% of a block, and his performace drops even faster than in the example given. He ends up taking 2.5 years to paint stripes on a 3 mile road.

Thanks for the diversion! :-)
Re^2: Archive::Zip performance question
by cowboy (Friar) on Feb 16, 2005 at 17:43 UTC
I agree, move the writeToFileNamed outside the loop. Instead of adding each file to 1 zip, you are doing something like the following:
• Zip up file 1
• Zip up file 1 and 2
• Zip up file 1, 2, and 3
• Zip up file 1, 2, 3, and 4
You should add all the files, then writeToFileNamed at the end.

Create A New User
Node Status?
node history
Node Type: note [id://431626]
help
Chatterbox?
 [LanX]: two utf8 strings from different sources are base64 encoded, but after joining both the umlauts in teh second get deleted [Corion]: LanX: You can't just join two base64 strings together [LanX]: (not a high priority bug because I can use some HTML entities in the second string) [Corion]: base64 is padded to a multiple of 4 chars (or something) [LanX]: misunderstanding, I joined them before converting to base64 [Corion]: Also, I would be wary of encodings and try to make really sure that both input strings are UTF-8. Maybe join the input strings from one source together to see whether they decode as bad or not [Corion]: LanX: Then the problem should persist without encoding to base64 too ;) [LanX]: I think it's a flag problem ... I'll produce a reprodocable example for SOPW [Corion]: "flag problem" to me sounds like "contains UTF-8 bytes but was never properly decoded to an UTF-8 string"

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (11)
As of 2017-01-16 13:55 GMT
Sections?
Information?
Find Nodes?
Leftovers?
Voting Booth?
Do you watch meteor showers?

Results (150 votes). Check out past polls.