Skip to content

File & Data Tools

Remove duplicate rows

Untick the columns that should not count towards matching. Keeps the first occurrence of each duplicate.

  • Free, no signup
  • No upload — runs on your device
  • No watermark
  • Unlimited use

Remove Duplicate Rows

Processed locally
Drop a CSV file here

or click to browse

Untick generated columns like timestamps and row IDs.

"Duplicate" is a decision about which columns matter

A row is only a duplicate relative to a key. Two customer records with the same email but different last_updated timestamps are duplicates by any sane definition, and unique by exact-match comparison. Every tool that deduplicates on the whole row will keep both, and every tool that offers no column selection is deduplicating on the whole row.

The columns to exclude are usually the ones a system generated: timestamps, sequence IDs, import batch numbers, hashes. The columns to keep are the ones a human would use to say "that's the same person" — email, account number, a name plus a postcode.

Case and whitespace insensitivity matter more than they look. ada@example.com, Ada@Example.com and ada@example.com with a trailing space are the same mailbox and three different strings. Email addresses are case-insensitive in the domain part by specification and in practice everywhere. Leave both options on unless you have a specific reason not to.

How it works

  1. Drop in a CSV
  2. Untick columns that should not count towards matching
  3. Click Remove duplicates and download
Why nothing uploads. Every operation on this page happens inside your browser using JavaScript and WebAssembly. Your file is read into memory, processed, and offered back as a download. It is never transmitted. Disconnect from the internet after this page loads and the tool keeps working.

Frequently asked questions

How do I deduplicate on just the email column?
Untick every other column. Matching then considers only email, and rows with the same address collapse to the first occurrence regardless of what else differs.
Which row is kept when duplicates are found?
The first occurrence in file order. If you need the most recent instead, sort the file by date descending before deduplicating.
Should I ignore case when matching?
For email addresses and most identifiers, yes. For anything where a capital letter carries meaning — a case-sensitive product SKU, a hash — no.
Does this remove rows that are only slightly different?
No. Matching is exact after the case and whitespace options are applied. Fuzzy matching on near-identical names requires a different tool and a human reviewing the results.
How many rows can it handle?
Hundreds of thousands comfortably — matching uses a hash set, so it is linear in the number of rows rather than quadratic.

Related tools