What is Mail Deduplicate?
Provides the mdedup CLI, an utility to deduplicate mails from a set of boxes.
- Duplicate detection based on cherry-picked and normalized mail headers.
- Fetch mails from multiple sources.
- Reads and writes to
mbox,maildir,babyl,mhandmmdfformats. - Deduplication strategies based on size, content, timestamp, file path or random choice.
- Copy, move or delete the resulting set of duplicates.
- Dry-run mode.
- Protection against false-positives with safety checks on size and content differences.
- Supports macOS, Linux and Windows.
- Standalone executables for Linux, macOS and Windows.
- Shell auto-completion for Bash, Zsh and Fish.
Warning
Performance and memory usage: mdedup implementation is quite naive and everything resides in memory.
If this is good enough for a volume of a couple of gigabytes, the more emails mdedup try to parse, the closer you'll reach the memory limits of your machine. In which case mdedup will exit abruptly, zapped by the OOM killer of your OS. Of course your mileage may vary depending on your hardware.
You can influence implementation of this feature with pull requests, purchasing business support 🤝 and sponsorship 🫶.
All installation methods are available in the documentation. Below are the most popular ones:
uv is the fastest way to run mdedup from sources on any platform, thanks to its uvx command:
$ uvx --from mail-deduplicate mdedup
Standalone binaries of mdedup's latest version are available as direct downloads for several platforms and architectures:
| Platform | x86_64 |
arm64 |
|---|---|---|
| Linux[^linux] | Download mdedup-linux-x64.bin |
Download mdedup-linux-arm64.bin |
| macOS | Download mdedup-macos-x64.bin |
Download mdedup-macos-arm64.bin |
| Windows | Download mdedup-windows-x64.exe |
Download mdedup-windows-arm64.exe |
