Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Re^3: Remove Duplicates from a mbox file

by Anonymous Monk
on Oct 11, 2007 at 03:20 UTC ( #644131=note: print w/replies, xml ) Need Help??


in reply to Re: Re: Remove Duplicates from a mbox file
in thread Remove Duplicates from a mbox file

I couldn't get the perl code above to work right, so I kept searching and I found the one on the web site below, It seems to work great! It removed 2400 duplicates from a 200MB mbox file. It also automatically creates a backup for you. www.wdr1.com/hacks/mbox-dedup.pl
  • Comment on Re^3: Remove Duplicates from a mbox file

Replies are listed 'Best First'.
Re^4: Remove Duplicates from a mbox file
by Anonymous Monk on Oct 21, 2009 at 13:40 UTC
    Yes, but beware it will skip messages which do not have a Message-ID header - and they won't be stored in the resulting file, so you'll have to keep the backup file nevertheless. However, all messages which were skipped will be output.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://644131]
help
Chatterbox?
[Corion]: GrandFather: Yeah, in that aspect, hardware is far more a black box than software
[GrandFather]: Oh, and the manufacturer tells lies too! I'm sure they don't mean to, but I know for sure some of the stuff they say is just wrong. It's possible they don't understand what we are asking, but that's not a good look either
[Corion]: GrandFather: Ah, unit testing hardware is fun, especially when the hardware is uncalibrated ...
[GrandFather]: Us: "What scaling do we need to apply to the numbers from the SDK for the wibble?". Them: "Oh, the numbers from the SDK for the wibble are already correct, they don't need scaling"
[GrandFather]: In our code: wibble range 1 scale by 1, range 2 scale by 2, range 3 scale by 4, range 4 scale by 8

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (8)
As of 2017-08-24 07:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Who is your favorite scientist and why?



























    Results (365 votes). Check out past polls.

    Notices?