Remove Duplicate Lines
Remove duplicate lines from a list, keeping the original order, with case and whitespace options.
The deduplicated list appears here, in the original order.
Computed in this tab as you type.
Saving needs a Dex account. The tool never does.
How it works
- Paste a list with one value per line. Duplicates are removed as you type and the first occurrence of each value keeps its original position.
- Case sensitivity, trimming and empty lines are separate switches. Trimming is what you want when the list came out of a spreadsheet and half the values have a trailing space.
- The count above the result tells you how many lines went in, how many were unique and how many were removed, so a list that did not change is obvious rather than suspicious.
- The list is processed in the page. Nothing is uploaded, which matters for customer lists and anything exported from a CRM.
Examples
Cleaning an email export
Paste the column, turn on trimming and case-insensitive matching, and the duplicate addresses disappear while the order of first appearance stays intact.
Deduplicating a keyword list
Paste the keywords, keep case sensitivity on, and the count tells you how much of the list was repetition before you hand it to anyone.
Frequently asked questions
Does it keep the original order?
Yes. The first occurrence of each line stays where it was; later duplicates are removed. Nothing is sorted unless you use the sort tool.
What does trimming do?
It removes leading and trailing whitespace from each line before comparing, so "apple" and "apple " count as the same value, and the output is trimmed too.
Why did nothing change?
If the count says every line was unique, there were no duplicates. Check whether case sensitivity is on when you expected it off, or the reverse.
Is my list uploaded?
No. The comparison runs in the page as you type and the list is never transmitted.