Example result
Input keys: A, B, A. Select the key column and the result keeps A and B, removing the later A row.
How to remove duplicate CSV rows by selected columns
Load your CSV in the dedupe tool with the correct delimiter and verify the displayed column names. Blank rows in the source file are skipped on ingest. Select one or more columns that define uniqueness for your task, such as email alone or customer id plus transaction date together. The tool compares entire rows only on those fields; other columns are carried along but ignored for match decisions. When multiple data rows share the same key values under exact comparison, the earliest row in file order is kept and later matches are dropped from the output. Case and spacing in decoded key columns must match exactly for rows to be considered duplicates. Preview shows up to twelve surviving rows; download returns the full deduped CSV with the same column order as the input.
Worked example: subscription list with repeated emails
Consider a file with columns email, signup_date, and source. Row two and row four both have email "user@example.com", signup_date "2024-01-10", and source "web"; row four appears later in the file. With only email selected as the key, both rows match on one column and the first occurrence (row two) is retained; row four is removed even if signup_date or source differed. If you instead select email and signup_date together, rows that share email but differ on date would both remain. A third row with "User@example.com" would not match "user@example.com" because comparison is case-sensitive. After dedupe, row count drops by the number of discarded duplicates; you can verify by sorting the download on the key columns and confirming no adjacent identical keys remain under your chosen fields.
Duplicate logic on keys you choose
Two rows are duplicates when every selected key column has identical text in each corresponding cell. Unselected columns do not affect the decision: a later row with the same id but an updated phone number still deletes if only id is in the key set and the id matches an earlier row. Exact match means no fuzzy deduplication by soundex, domain normalization, or trimmed whitespace unless the cells already match as stored. All-blank keys on multiple lines will collide if those columns are the sole keys, which may or may not be what you want for sparse data. Rows that differ in one key field by a single character count as distinct. For composite keys, partial overlap is insufficient; all chosen fields must match simultaneously.
CSV structure issues that affect deduplication
Headers come from row one; if keys include misspelled header names from a bad export, pick the actual column labels shown in the tool. Delimiter errors can concatenate what should be two key fields into one, hiding duplicates or creating false uniqueness. Quoted fields keep embedded delimiters inside one value so "123,456" stays one id. Line endings from Windows or Unix are normalized by the parser but row order follows the sequence in the file as read, which defines which duplicate wins. If your policy is "keep latest by date", this tool always keeps first by position; you would sort the file appropriately before upload or adjust outside the browser. Skipped blank lines do not participate, so line numbers in a text editor may not match row indices in the preview.
Privacy, preview limits, and output shape
Dedupe runs locally like the other CSV utilities on this site; files stay on your device. Output preserves all original columns for kept rows and drops entire duplicate lines rather than merging field values from duplicates into one row. No aggregate statistics are computed here. Preview capped at twelve rows is for layout inspection only; rely on download for reconciliation counts. Large files dedupe in memory; extremely wide or long sheets may require patience or preprocessing. Because processing is ephemeral, download the result before navigating away. The result file remains comma-separated text suitable for re-import elsewhere; encoding follows typical UTF-8 expectations for browser-generated downloads.
Confirming the right rows survived
After download, sort by your key columns and scan for repeated keys; none should appear twice if keys were chosen correctly. Compare input and output record counts: their difference equals the number of rows removed. Check edge cases: first-row headers never dedupe against data. Single-row files pass through unchanged. If too many rows vanished, widen or narrow the key set—perhaps you keyed only on zip code when you needed address plus zip. If too few vanished, the selected key may contain extra distinguishing fields or casing splits what you consider one entity. Spot-check a known duplicate pair in the original against the output to confirm the earlier row’s non-key columns are the ones you wanted. Case-sensitive duplicates may need a normalization pass upstream before dedupe if policy requires it.
Typical dedupe scenarios and chaining tools
Event registrations, newsletter signups, and lead dumps often need one row per email or per person id. Transaction logs might need dedupe on order id to remove accidental double posts while keeping legitimately repeated product lines on different orders when keyed differently. After dedupe, run the count tool on a status or region column to profile the cleaned set. Wide metrics sheets might unpivot before dedupe if uniqueness applies per id and metric name in long form rather than repeated wide columns. For migration, dedupe before loading a database unique index reduces constraint failures. Document which columns formed the key in your run notes so teammates reproduce the same rule. When business logic requires keeping the newest duplicate, pre-sort descending on a timestamp column in your editor, then dedupe so “first” in file equals newest.
CSV format reference
CSV can use quoted fields for commas, quotes, and line breaks. See the CSV format specification, RFC 4180, for details about common CSV conventions. Check the file's encoding and delimiter before processing.
Frequently asked questions
Which duplicate row is kept?
The first matching row in file order is kept; later rows with the same selected values are removed.
Can I choose multiple columns?
Yes. All selected column values must match for two rows to be treated as duplicates.
Are spaces and capitalization ignored?
No. Comparison is exact and case sensitive; spaces are preserved.