Dmitriy KvasnikovRU
All work

Open source · Development

Deduplicator

An offline Windows app that finds duplicate files by content, including perceptually similar images and audio, and never deletes anything without confirmation.

Period
2026
Role
Author
Stack
C#, .NET 8, Avalonia UI, SQLite (WAL), XxHash128, pHash, Chromaprint, xUnit
Links
Deduplicator — order in your files

Promo for Deduplicator.

pipeline stages
7
screens
12
telemetry
0

What it does

Deduplicator compares files by content, so a renamed or moved copy is still recognised. It finds exact duplicates and perceptually similar images and audio, and can consolidate several scattered drives into one clean archive. It runs offline on Windows 10 and 11.

Scan pipeline

Each stage is cheaper than the next, so the expensive ones only see candidates that survived the earlier ones.

  • Walk the file system and detect the file category by magic bytes.
  • Group by size: only files of equal size are candidates, with no disk reads.
  • Partial hash: XxHash128 of the first and last 64 KB.
  • Full hash only for files whose partial hashes matched.
  • Byte-by-byte comparison before a file is called an exact copy, as protection against collisions.
  • Perceptual analysis: pHash for images and audio fingerprints, indexed into LSH buckets.
  • Grouping into exact and similar sets with a similarity percentage.

Safety rules

  • Nothing is deleted automatically: only on confirmation, and only to the Recycle Bin or quarantine.
  • At least one copy always remains in a group; the interface does not allow removing all of them.
  • Similar files are handled manually only, with no batch actions.
  • File writes are atomic: temporary file, flush to disk, rename. A power loss leaves either the old file or the new one.
  • Progress is stored in the database, so a scan resumes after an interruption.
  • Offline by default: no telemetry, and the network is used only for an explicit update check.