Re: Archive::Zip performance question

[Albannach]

Well one thing that will clearly not scale well is that you are writing the entire ZIP file after each new member is added to the archive, and that ZIP just gets bigger each time. Perhaps try it with the writeToFileNamed outside the loop?

Update: I just noticed you didn't escape the period in your grep, but I imagine you probably wanted to.

[holli]

My programming teacher in school (ahh, long ago) told us this story once:



One day Schlemihl got a new job at a company that paints the white markings on the street. On the first day Schlemihl tagged 3 miles of the street with markings. The Boss was very pleased and thought about paying him a bonus. On the second day Schlemihl did 1,2 miles and on the third day he only made 300 yards.
What´s up?, asked the boss. You made 1.2 miles on the first day and now you come back with wimpy 300 yards. Don´t you want to keep your job?

Well., said Schlemihl, The way to the paint bucket gets longer and longer.

Since then I know the term Schlemihl Algorithm for a algorithm that does not scale well.

[poqui]

Just for fun, I tired to reason out how far Schlemihl walked that first day, with the following assumptions:
Each stripe is 24 inches long, and separated from the next stripe by 24 inches
The paint brush holds enough paint to paint 1 stripe
There are 1311 stripes per block (1 block = 1 mile, with 36 foot streets separating the blocks that are unpainted)

Where T = trip number, S = stripe length, I = interval between stripes
That makes each trip = T*(S*2)+I(2(T-1))
or Trip 1 = 48"
Trip 2 = 144"
T3 = 240" etc.

total trip = T1+T2+T3...

adding in the width of the streets crossed between blocks, Schlemihl walked an amazing 11,772.18 MILES to paint 3933 stripes on 3 miles of road. At a speed of no less than 1471.5 MPH! (assuming an 8 hour shift)

Assuming he could carry a bucket that held enough paint to paint 3 miles of stripes, he could paint that 3 miles in about 6 seconds.

On the other hand, his performace at a more human speed of 2.5 MPH, he would only be able to paint about 162 stripes in 8 hours (using the old method), or about 12% of a block, and his performace drops even faster than in the example given. He ends up taking 2.5 years to paint stripes on a 3 mile road.

Thanks for the diversion! :-)

[cowboy]

I agree, move the writeToFileNamed outside the loop. Instead of adding each file to 1 zip, you are doing something like the following:
• Zip up file 1
• Zip up file 1 and 2
• Zip up file 1, 2, and 3
• Zip up file 1, 2, 3, and 4
You should add all the files, then writeToFileNamed at the end.