 Syntactic Confectionery Delight PerlMonks

Re^4: Algorithm for cancelling common factors between two lists of multiplicands

by BrowserUk (Pope)
 on Aug 10, 2005 at 10:35 UTC ( #482557=note: print w/replies, xml ) Need Help??

These are the results I have currently for the inputs [989 9400 43300 2400]:

tmoertel's haskell implementation of the cancelling code ( in 2.6 seconds):

[11:20:43.54] C:\ghc\ghc-6.4\code>FET < ex1.dat +8.070604647867604097576877675243109941729476950993498765160880e-7030 [11:20:46.34] C:\ghc\ghc-6.4\code>

Using Math::Pari (in 26 ms):

P:\test>MP-FET.pl 8.070604647867604097576877668E-7030 1 trial of _default ( 24.817ms total), 24.817ms/trial

Using Math::BigInt (Yes. That 4 hrs 38 minutes!)

P:\test>MBF-FET.pl 8.070604647867604097576877675243109941729e-7030 1 trial of _default ( 16,699s total), 16,699s/trial

The whole point of the cancelling code is that you do not have to calculate the huge factorials in infinite precison (slow) math, because you reduce the factorials to products of sets of much smaller, prime factors and then cancel most of those.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

Replies are listed 'Best First'.
Re^5: Algorithm for cancelling common factors between two lists of multiplicands
by sk (Curate) on Aug 10, 2005 at 16:38 UTC
Thanks for the update

I cannot read Haskell so I was not sure how the factoring was done. Going through the example it seems like tmoertel was just factoring like terms. If the code does further factoring (school days division) then I am surprised that it can loop through that fast!

If you take a look at my scratchpad I tried ikegami's logic on gcd. I added a condition to check for values == 1 to save an extra GCD. I also did not have the Math module he was using so picked up another one.

This one takes about 9 minutes. It would be interesting to see what will happen if we apply BigInt from here onwards!

I am also curious to see the runtime of the prime factor approach. I shall try hv's code to see how fast it runs. If I remember correctly Euclidean algo for GCD is O(n+log n) and prime factorization is NP-complete. For small values, prime fact is a breeze but the complexity does not change so i shall let you know when i get around testing the runtime for it!

cheers SK

I cannot read Haskell so I was not sure how the factoring was done. Going through the example it seems like tmoertel was just factoring like terms.
My implementation doesn't actually do any factoring. Rather, it efficiently cancels all like terms before doing any multiplications or divisions. The idea is to replace expensive mathematical operations with inexpensive comparisons until I have picked all of the low-hanging fruit. If I pick enough fruit, there isn't enough left on the tree to make factoring worthwhile, and so I can just go ahead a multiply out the rest. (Although it might be interesting to emit the tree after my low-handing-fruit pass and process it with some of the other bits of code in this thread.)

The following shows how my code might compute 7!/(3!4!):

7! ----- 3! 4! { expand fac into series of multiplications } P(2,3,4,5,6,7) ----------------- P(2,3) * P(2,3,4) { merge multiplied products } P(2,3,4,5,6,7) ----------------- P(2,2,3,3,4) { cancel like terms across division boundary } P(/,/,/,5,6,7) ----------------- P(/,2,/,3,/) { multiply and divide remaining terms } 5 * 6 * 7 --------- 2 * 3 { result } 35
In reality, the code does all of these steps in a parallel pipeline, which means I don't have to pay the price for large intermediate terms: bits of big terms are plucked off and used as they are produced lazily. This is an essentially free benefit of using Haskell.

Cheers,
Tom

Update:Added output and updated the list entries based on BrowserUk's final code example

Thanks for the detailed explanation!

Sometimes i get very sloppy when I write. My apologies, i should not have used the word factorize instead i should have used the word 'cancel' out.. Actually I was thinking more in terms removing'like' elements. This is in reference to your code.

In the code that I had, I was removing further cancelable terms

Going back to your example

{ multiply and divide remaining terms } 5 * 6 * 7 --------- 2 * 3 The program I had would have done this before multiplying 5 * 7 ----- 1 {result} 35
Here is a simple implementation of canceling out factorial terms without having to expand the factorials because of the fact

n!/k! = PI(x) where x goes from (k+1) to (n) assuming n > k; PI(x) sta +nds for product of<p> <readmore>
#!/usr/bin/perl use strict; use warnings; # Numerator and Denominator specified as factorials [BrowserUk]'s exam +ple numbers my @snum = (44_289, 11_800, 10_389, 4570); my @sden = (56_089, 989, 9_400, 43_300, 2_400); # old list from previous node # my @snum = (44_289, 11_800, 10_389, 45_700); # my @sden = (56_089, 989, 9_400, 43_300, 11_800, 2_400); # @snum = sort {\$b<=>\$a} @snum; @sden = sort {\$b<=>\$a} @sden; my \$i = 0; # Make the arrays equal size if (@snum < @sden) { foreach \$i (@snum..\$#sden) { \$snum[\$i] = 0;} } else { foreach \$i (@sden..\$#snum) { \$sden[\$i] = 0;} } print +(\$_,\$/) for (@snum); print \$/; print +(\$_,\$/) for (@sden); print \$/; my @nexp = (); my @dexp = (); # n!/k! = (k+1)..(n) [assuming n > k] for \$i (0..\$#snum) { if (\$snum[\$i] > \$sden[\$i]) { print ("Numerator: Going to push from ", \$sden[\$i]+1, " to ", +\$snum[\$i],\$/); push (@nexp, ((\$sden[\$i]+1)..\$snum[\$i])); } elsif (\$sden[\$i] > \$snum[\$i]) { print ("Denominator: Going to push from ", \$snum[\$i]+1, " to " +, \$sden[\$i],\$/); push (@dexp, ((\$snum[\$i]+1)..\$sden[\$i])); } } print ("Length of expanded numerator = ", scalar @nexp, \$/); print ("Length of expanded denominator = ", scalar @dexp, \$/); print ("Total Number of elements = ", scalar @nexp + scalar @dexp, \$/) +;
__END__ Denominator: Going to push from 44290 to 56089 Denominator: Going to push from 11801 to 43300 Numerator: Going to push from 9401 to 10389 Numerator: Going to push from 2401 to 4570 Denominator: Going to push from 1 to 989 Length of expanded numerator = 3159 Length of expanded denominator = 44289 Total Number of elements = 47448

Now with the expanded list I would run the GCD program which cancels out even more terms like for example the number 11 cancels 56089 in the numerator and cancels 2398 in the denominator

I hope it is clear now.

The reason I was trying to factor down even further is to avoid computing at very high precision for numbers that are going to be canceled out anyways

Has been a very interesting exercise so far!!! I will keep thinking about this problem as I am sure there should be a way to factor these numbers down further without going GCD route given they have such a nice pattern..after all they are just sequential products!

Re^5: Algorithm for cancelling common factors between two lists of multiplicands
by QM (Parson) on Aug 10, 2005 at 18:39 UTC
Can you post your Math::BigInt code and test case? I would like to play around with some other ideas.

Update: Ooops, I see it now, thanks.

-QM
--
Quantum Mechanics: The dreams stuff is made of

I already did. It's the second example in 482072.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
Re^5: Algorithm for cancelling common factors between two lists of multiplicands
by tmoertel (Chaplain) on Aug 11, 2005 at 05:04 UTC
The last two digits of the Math::Pari implementation's result seem to be off:
+8.070604647867604097576877675243109941729476950993498765160880e-7030 8.070604647867604097576877675243109941729e-7030 8.070604647867604097576877668E-7030 <-- Math::Pari ^^
Can you increase the implementation's precision? Or is that the limit?

Cheers,
Tom

The precision is setable, but I had left at the default.

Increasing the precision to the point where it match or suppassed the accuracy of your Haskell code (a setting of 58), doubled the time it took to a gnat's under 50 ms:

[ 6:46:43.79] P:\test>MP-FET.pl 8.070604647867604097576877675243109941729476950993498765160880911582E- +7030 1 trial of _default ( 49.865ms total), 49.865ms/trial

BTW: Could you explain something for me? In this:

cancel (x:xs) (y:ys) | x == y = cancel xs ys | x < y = let (xs', ys') = cancel xs (y:ys) in (x:xs', ys') | otherwise = let (xs', ys') = cancel (x:xs) ys in (xs', y:ys') cancel xs ys = (xs, ys)

I understand how it cancels between the lists, but I am confused about when it would pattern match against the second definition?

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
The pattern (_:_) will not match an empty list [], so the first definition will match only when both input lists are non-empty. If one or both of the inputs are empty, the second definition will match.

Create A New User
Node Status?
node history
Node Type: note [id://482557]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (3)
As of 2019-10-18 23:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
Voting Booth?
In 2019 the site I miss most is:

Results (45 votes). Check out past polls.

Notices?