perlquestion swampyankee <p>I've written a little script which will check for duplicate files by walking down my file system. That part's no problem. What is a problem is how long it takes to get md5 signatures of large files, a couple of which are zipped tar files I'm using for backups.</p> <p>Right now, I'm simply skipping files that are too big (2**24 bytes or larger), which is inelegant.</p> <p>So, question 1 is how does md5's execution time scale with file size? (I would expect linearly, but I'm not sure)</p>Question 2: Is there a similarly reliable but quicker algorithm to get a file's signature?</p> <p>I'm using the md5 program that came with my computer, which is a MacBook with a 2.1 Ghz Intel Core 2 processor, 1 BG RAM, and Mac OS X 10.7.4 (don't laugh; it was free ;))</p> <!-- Node text goes above. Div tags should contain sig only --> <div class="pmsig"><div class="pmsig-494652"> <hr><p> Information about American English usage [http://wsu.edu/~brians/errors/index.html|here] and [http://andromeda.rutgers.edu/~jlynch/Writing/|here]. Floating point issues? Please read [http://docs.sun.com/source/806-3568/ncg_goldberg.html| this ] before posting. &mdash; emc</p> </div></div>