From your question and your clarifications in the comments, the way I understand your problem is that you are looking for the number of unique relations. So, if given a file as follows:
345 is related to 123
678 is related to 123
187 is related to 732
678 is related to 123
The result should be 3. To do this, sort
, uniq
, cat
, wc
, and the pipe operator will be your friend.
If all the files you want to search through are in one folder, you can simply use *
(a wildcard operator) to search every file. For example, cat *
will print the contents of every file in the current directory. You can also specify multiple files by listing them. cat file1.txt file2.txt
will print both file1
and file2
. This can be combined with the wildcard to specify files in specific subdirectories: cat dir1/* dir3/*
(skipping dir2/
). You can also specify a partial filename: cat file*
will match file1
and file123
but not afile
.
Using the above, you can include any files you want into the input for sort
. The pipe operator (|
) takes the output of one command and pipes it into the input of another. cat file1 | sort
will sort the contents of file1
. You can then pipe this into uniq
to filter out all the duplicate entries, and then count the lines with wc
like so: cat file1 | sort | uniq | wc -l
.
Using the example file I gave above the sort
command will change the output into:
187 is related to 732
345 is related to 123
678 is related to 123
678 is related to 123
uniq
will remove the duplicate entries (in this example at the bottom):
187 is related to 732
345 is related to 123
678 is related to 123
and wc -l
will count the lines and return 3
.
NOTE: This will count 123 is related to 321
and 321 is related to 123
as two separate relations. If you want to get around this you will need to do something more complicated involving parsing the input data and organizing the entries into a table or hashmap.
Also, on extremely large inputs this command chain will likely hang and look as if it has frozen. If you are using large sets of input data I would look into setting up a database to manage the data, as that will allow you to use queries to perform this type of search much faster.
grep -c Unique_ID *
seems like a simplistic solution; how far does that get you? (Surely there's a duplicate question here somewhere?)345 is related to 123
and678 is related to 123
, those two lines would count as one entry, or two?