I'm trying to use tar(1) to create an archive of files newer than a specific file (fileA). However, when I use find(1) to obtain the list of files to pass to tar, some files are listed multiple times:
I always have been using sort -u to get rid of duplicates until now.
But I am having a real doubt about a list generated by a software tool.
The question is: is the output of sort -u |wc the same as ...
I'm doing some processing trying to get how many different lines in a file containing 160,353,104 lines. Here is my pipeline and stderr output.
$ tail -n+2 2022_place_canvas_history.csv | cut -d, -f2 |...
I have the following:
cat /var/log/example.log | cut -d ' ' -f 3,4,5,6 | sort -u | uniq
The result has several thousand lines but I would like to cut this down to show the actual activity and to do ...
I would like to sort all files in all subdirectories in place. I have 65536 files in 256 directories, every file containing one word a line, and each file contains duplicates.
What I would like is to ...
I have a big text file containing a million lines.
I would like to find identical lines that match my specific text and leave the first occurrence intact. Any ideas?
So the algorithm should roughly be ...
Given a file with paragraphs (group of lines separated by a blank line).
It is guaranteed that the line breaks within equivalent paragraphs are conserved.
Is there a way of using uniq to remove the ...
Hi guys in the terminal I use the command cat file.txt | sort | uniq -c . First I would like to use that command on vim and save the output to another file without the output being print when I run ...
I'm trying to get some trivial statistics on some debug output.
Each debug line is of the form(class name)(delimiter 1)(object ID)(delimiter 2)(method name)(delimiter 3)(log message)
I want to get a ...
I am new to Linux and was wondering how I would go about deleting all duplicates with same file name but only keep one original?
For example I have the following:
dir_new_001 = rdt.txt
dir_new_002 = ...
I am trying to count the number of occurrences without specifying a specific string, just using the output of the cut command to print the number of each repeated string using grep. I then want to get ...
This is a similar question to this one
I want to do the word count but this time, using an array.
For example, I have the following IPs inside a bash array called IPS.
IPS=("184.108.40.206" "220.127.116.11" "3.3....
I want to know the difference between sort -u and uniq.
I tried below command to get the count for the unique strings in a file , sort -u and uniq provides the same output from my knowledge. Then why ...
I have a file that has 2 columns like the following:
If a word appears on multiple lines i want to delete all lines that the repeating word ...
I have a CSV file containing 220k names that I want to batch import into a graph database. The batch importer imposes the restriction that each node must occur only once so I want to remove duplicates ...
In the given data is it possible to uniq sort and print only the top hits against each region?
I've got a file of ~1-2 million lines that I'm trying to reduce down by counting duplicate groups of lines, preserving order.
uniq -c works okay :
$ perl -E 'say for (("foo") x 4, ("bar") x 4, "baz"...
I have a tab delimited .txt file and want to delete rows that have duplications in the first column. If this can be done without sort, that would be amazing, but I've kind of come to terms with that I ...