remove duplicates · sort · keep first
Duplicate line remover
Removing exact repeats from a 7-line list in which “apple” appears 3 times leaves 5 lines. Paste your own list to remove its repeats.
- 7 → 5 (−2)
- 7 → 3 (−4)
The result updates as you change the text or an option. Output keeps the line order of the input unless you pick a sort order. The text never leaves this tab.
Scanned by 
NortonTM Safe Web
At a glance
- Price
- No charge
- Account
- None needed
- Upload
- Not applicable — the list never leaves your tab
- Keep modes
- First copy, last copy, only unique lines, only repeated lines
- Sort
- Input order, A to Z, Z to A and natural, with the sorting rules of the page language
- Size limit
- 5,000,000 characters per text
The same list under each keep mode, and each sort order
The sample list is apple, Banana, apple, cherry, banana, one empty line, apple. These tables come from the same code as the tool; ↵ marks a line break.
| Keep | Lines kept | Lines removed | Result |
|---|---|---|---|
| First copy | 5 | 2 | apple↵Banana↵cherry↵banana↵ |
| Last copy | 5 | 2 | Banana↵cherry↵banana↵↵apple |
| Only unique lines | 4 | 3 | Banana↵cherry↵banana↵ |
| Only repeated lines | 1 | 6 | apple |
| Sort | Result |
|---|---|
| Keep input order | item10↵item2↵Item1 |
| A to Z | Item1↵item10↵item2 |
| Z to A | item2↵item10↵Item1 |
| Natural | Item1↵item2↵item10 |
How to remove duplicate lines
Paste the list
Put one item per line in the box. A sample list is already there. Windows and Unix line breaks both work, and the result keeps the kind of line break your text uses.
Choose what counts as a duplicate
Tick Ignore case to treat “Apple” and “apple” as the same line, Trim spaces to strip spaces at the start and end of each line, and Remove empty lines to drop blank lines. Pick which lines to keep: the first copy, the last copy, only unique lines or only repeated lines.
Sort if you want, then copy
Leave Sort on Keep input order, or sort A to Z, Z to A, or in natural order, where item2 comes before item10. Press Copy or download the result as a .txt file. The tiles show how many lines were read, removed and kept.
How duplicates are found
Each line is compared with the others as a whole, character for character. Two lines are duplicates only when they are identical, unless you tick Ignore case or Trim spaces, which compare lines without regard to capitals or to spaces at the ends. Keep first keeps the first copy of each line, and Keep last keeps the final copy. Only unique lines keeps the lines that occur exactly once, and Only repeated lines keeps one copy of each line that occurs more than once.
Sorting happens after the duplicates are removed and uses the sorting rules of the page language, through the browser's Intl.Collator, so letters such as ä sort where that language puts them. Natural order reads digit runs as numbers: item2 comes before item10, while plain A to Z puts item10 first. The sort is stable, so lines that the language treats as equal stay in their input order. The removed tile counts every line that is not in the output, including blank lines dropped by Remove empty lines. The text may have up to 5,000,000 characters.
Duplicate line remover questions
How do I remove duplicate lines from a list?
Paste the list with one item per line. Lines that repeat an earlier line are dropped and the first copy stays, in its original position. The list “apple, banana, apple, cherry” becomes “apple, banana, cherry”. Copy the result when the tiles show the count you expect.
What is the difference between keep first, keep last, only unique and only repeated?
For the list a, b, a: keep first gives a, b; keep last gives b, a; only unique gives b, because a occurs twice; only repeated gives a, one copy of the line that occurs more than once. Pick the mode that matches what you want to do with the repeats.
Are “Apple” and “apple” duplicates?
Not by default: lines are compared exactly, so differences in capitals or spaces make two lines distinct. Tick Ignore case to treat them as the same line, and Trim spaces to ignore spaces at the start and end of a line. With Keep first, the first spelling is the one that stays.
What does natural sort mean?
Natural sort reads runs of digits as numbers. In natural order the lines item1, item2 and item10 stay in that order; in a plain A to Z sort, item10 comes before item2 because the character 1 sorts before 2. Use it for file names and numbered lists.
Does it keep the original order of the lines?
Yes, unless you choose a sort. With Sort set to Keep input order, the lines that remain appear in the order they had in your text. A sort is stable, so lines that compare as equal also keep their input order.
How large can the list be and is it uploaded?
The text may have up to 5,000,000 characters; a list of 300,000 short lines is processed in a fraction of a second. Nothing is uploaded: the work runs in your browser tab and closing the tab discards the text.