0

I have a.txt,b.txt,c.txt. Each has different numbers as below:

a.txt:

12
14
111
1
15
2

b.txt

12
18
22
23
1
2

c.txt

12
14
15
16
17
1200

The output should contain all the numbers from each file, but without any duplication. Is there a command to export such a thing into a text file? The actual text files include hundreds of rows.

  • You say csv but there are no commas in there ! – user80551 Jun 16 '14 at 6:55
  • My bad. Its actually a text file. each line separated by \n – ALH Jun 16 '14 at 7:01
1

You could do like this if there are more number of files,

grep '' *.csv | cut -d: -f2 | sort -u > output.csv
  • How this line makes output unique? – ALH Jun 16 '14 at 7:24
  • sort -u command makes unique output. – Avinash Raj Jun 16 '14 at 7:27
  • What does grep do? – mikeserv Jun 16 '14 at 7:33
  • Grep prints all the contents of all csv files presentinside the current directory along with the filenames.cut part further cutdown the filenames leaving only the content. – Avinash Raj Jun 16 '14 at 7:42
  • Two completely unneeded processes spawned just to make sort read several files? A bit overkillish. – peterph Jun 16 '14 at 10:20
5

You say that your files are csv i.e. Comma Separated Values but the examples you give are new-line separated. You can do

cat a.csv b.csv c.csv | sort -u > output.csv
  • Its hard to tell with his output, but I presume there aren't really newlines and they're actually comma separated. Try cat ...|tr , \\n|sort -u|tr \\n , >out – mikeserv Jun 16 '14 at 7:07
  • How do you make it unique? – ALH Jun 16 '14 at 7:28
  • 1
    The -u switch in sort -u makes it unique. – user80551 Jun 16 '14 at 7:31
2

For POSIX compatible implementations of sort all you need is:

sort -u *.txt > output.uniq

for sed will concatenate the files before sorting.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.