Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a tab separated file that looks like this:

123   some text
123   some different text
334   some other text
341   more text

and I want to do two things. One is to order everything numerically (this is easy to do) and the other is to remove a line if it's number is already present. I.e. the output would look like this:

123   some text
334   some other text
341   more text

I tried getting a file of just the unique numbers, i.e.


and joining it with the original file with:

join -j 1 justNumbers.txt original.txt

but this gave me the original file back. Any ideas?

share|improve this question
up vote 6 down vote accepted

If you want to sort/test for uniqueness the first field specifically, and your system has the GNU coreutils version of sort, then I think you could just use

sort -nu file


$ sort -nu file
123   some text
334   some other text
341   more text

From info coreutils 'sort invocation'

The commands sort -u and sort | uniq are equivalent, but this equivalence does not extend to arbitrary sort options. For example, sort -n -u inspects only the value of the initial numeric string when checking for uniqueness, whereas sort -n | uniq inspects the entire line.

share|improve this answer
That should work with any POSIX sort, not only the GNU one. – Stéphane Chazelas Jul 4 '14 at 17:14
Note however that it you have one line starting with 0123 and another one starting with 123, it will keep only one of them (which may be what you want). Use sort -ubk1,1 | sort -n if it's not what you want. – Stéphane Chazelas Jul 4 '14 at 17:17
@StéphaneChazelas thanks (as usual) for your most knowledgeable input. – steeldriver Jul 4 '14 at 21:20

Try this awk:

awk '!($1 in a){a[$1];print}' file
123   some text
334   some other text
341   more text

This assumes that file was sorted.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.