Tag Info

New answers tagged

1

As long as you're interested in the last column, you can do it with sort and uniq: $ sort -k3n test.txt | uniq -f2 -D 1+1 = 2 2x1 = 2 BLABLABLA = 2 Here, the sort option -k3n causes the file to be sorted starting with the third field, in numeric order; the options to uniq are: -f2 Skip the first two fields before checking for uniqueness -D Print ...


0

Use Uniq. uniq -f 3 <file name>


0

One can use power of vim: :g/part of duplicate string/d Very easy. If you have couple more files (such as gzipped rotated logs), vim will open them without any preliminary uncompression on your side and you can repeat the last command by pressing : and ↑. Just like repeating last command in terminal.


5

According to this article, the main difference between this and the normal duplicate (⌘D) function is that file ownership is retained. The normal duplicate function preserves file permissions but not ownership. The best equivalent to this behavior on OSX is the ditto command. You can simply use sudo ditto src dst and it will preserve everything by default. ...


2

Volume group name should be unique on system, by design. Problem occurs when a disk is moved from a system to another. So you have few options (detailed below) rename the VG on the external [not mounted] disk(s). rename the VG of your system (not realistic) merge both volume group into a single one (probably needs to rename first) option 1 - rename the ...


0

Problem is resolved with renaming of VolumeGroup. With this command: vgrename bwQkRq-mgph-9BYf-9WPF-cKz0-FLFq-0Qxs73 storage


1

I'm glad you got the job done with rdfind. Next time you could also consider rmlint. It's extremely fast and offers a few different options to help determine which file is the original in each set of duplicates.


0

I'd be thinking of using Perl: #!/usr/bin/perl use strict; use warnings; use File::Find; use Digest::SHA qw ( sha1_hex ); my %seen; sub find_dupes { next if -d; local $/; open( my $input, "<", $File::Find::name ) or warn $!; my $sha1sum = sha1_hex(<$input>); close($input); if ( $seen{$sha1sum} ) { print ...


4

There is such a program, and it's called rdfind: SYNOPSIS rdfind [ options ] directory1 | file1 [ directory2 | file2 ] ... DESCRIPTION rdfind finds duplicate files across and/or within several directories. It calculates checksum only if necessary. rdfind runs in O(Nlog(N)) time with N being the number of files. If two (or more) ...


0

So, the answer given by arnefm (that's been copied all over the internet) does not deal with spaces in file names. I've written a script that deals with spaces in files. #!/bin/bash fdupes -r -1 CHANGE_THIS_PATH | sed -e 's/\(\w\) /\1|/g' -e 's/|$//' > files while read line; do IFS='|' read -a arr <<< "$line" orig=${arr[0]} ...



Top 50 recent answers are included