The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
2answers
23 views

Get or filter duplicated lines by column

I have a file with three columns and I would like to get duplicated lines by third column, for example: AAA = 342 BLABLABLA = 2 BBBx2 = 23 1+1 = 2 KOKOKO= 5 2x1 = 2 The output sould ...
3
votes
1answer
90 views

Unix command equivalent for OS X Finder's Duplicate Exactly

What command can reproduce this function in bash? In Finder select File then press ⇧⌥ to make the command visible (⇧⌥⌘D). In AppleScript it's available as tell application "Finder" to duplicate ...
2
votes
2answers
32 views

Can't mount second hard drive on CentOS 6.6 - Duplicate VG name

Yesterday I add a SSD to my PC configuration and I make on it a fresh installation. At the moment of installation I replace my old HDD and there was only the SSD. When the installation finish I make a ...
2
votes
3answers
65 views

Search and Delete duplicate files with different names

I have a large music collection stored on my hard drive; and browsing through it, I found that I have a lot of duplicate files in some album directories. Usually the duplicates exist alongside the ...
2
votes
2answers
160 views

How to remove duplicate files using bash

I have a folder with duplicate (by md5sum (md5 on a Mac)) files, and I want to have a cron job scheduled to remove any found. However, I'm stuck on how to do this. What I have so far: md5 -r * | ...
1
vote
1answer
228 views

CentOS 6 interfaces bonding, round-robin instead of active-backup, duplicates frames

Two interfaces, eth0 and eth1 are part of a network bonding bond0 on CentOS 6. All worked well under CentOS 5, but after the upgrade to CentOS 6.6, keeping the same configuration, the network works ...
9
votes
8answers
248 views

Find duplicate PDF files by content

Some journals generate a different PDF for each download. APS for example stores time and the IP address in the PDF. Or there is a paper version with hyper links and one with text references. How ...
1
vote
3answers
87 views

check if all files from a folder are also in another folder

I basically have a directory a with lots of images. Now I want to check if all of these images are in directory b. The point is, that lots of images in b ain't directly in b but in subdirectories. ...
4
votes
2answers
186 views

Print repeating pattern

I have a marathon file from which I want to print the names of the runners, who have made multiple runs. Something like: M, 2:08:58.0, John Lukas, USA, 1997-07-25, Paris, France F, 2:15:04:0, Abebbe ...
2
votes
3answers
114 views

Find files that have a confirmed duplicate in same directory recursively

Say I have the following directory structure: root |-- dirA |-- file.jpg |-- file-001.jpg <-- dup |-- file2.jpg |-- file3.jpg |-- dirB |-- fileA.jpg |-- ...
8
votes
5answers
578 views

Remove duplicate lines from a file that contains a timestamp

This question/answer has some good solutions for deleting identical lines in a file, but won't work in my case since the otherwise duplicate lines have a timestamp. Is it possible to tell awk to ...
2
votes
2answers
88 views

Finding duplicate files by name ignoring the case in subdirectories

How is it possible to list duplicate file names on a Linux system ignoring the case including all subdirectories Files should not be compared by their content but only by their names. The output ...
2
votes
2answers
412 views

Finding duplicate files and replace them with symlinks

I'm trying to find a way to check inside a given directory for duplicate files (even with different names) and replace them with symlinks pointing to the first occurrence. I've tried with fdupes but ...
0
votes
0answers
12 views

How to find duplicate files with identical content and identical or different names [duplicate]

I have several directories containing a few hundred files in each. Each directory contains files that are duplicated (i.e. have the same content) within the same directory, and/or are duplicated in ...
2
votes
0answers
98 views

Removing duplicate files but ignoring certain lines (e.g. embedded timestamp)

I have a bunch of gzipped text files. I am trying to remove duplicate files (using fdupes) but the problem is these files contain a line that has their timestamp in them, and the files are identical ...
5
votes
4answers
248 views

Removing lines that are common to the outputs produced by two commands

Say, I have two commands that return some text. For example: $ ./c1 /usr/bin/foo /usr/bin/bar /usr/bin/baz $ ./c2 /usr/bin/foo /usr/bin/qux /usr/bin/buzz /usr/bin/bar I want to remove the duplicate ...
6
votes
3answers
343 views

Shell script to search files for identical text entries

Here's a nice teaser for a shell scripting guru: Take a directory with several text files. Can be a few up to ~1000. All files contain an identifier on a given line (always the same line). Identify ...
4
votes
2answers
271 views

Faster way to rename duplicate files (identified by fdupes) in another directory?

I have a directory full of pdf files of journal articles, most of which are named by their bibtex key. Some time ago I made a backup on an external hard drive, but I haven't kept it up to date and ...
4
votes
4answers
1k views

Use basename to parse a list of paths held in a file

I'm running Mac OSX and trying to use the command line to find the number of files I have with the same name. I tried to use the following command: find ~ -type f -name "*" -print | basename | sort ...
1
vote
2answers
472 views

Find duplicate file names with specific matching pattern

I have a folder with some files (snippet of the contents of the folder) PAT1.URGRSVP.50.WR786842JOB11632.WRS20140.FILE0005.DAT PAT1.URGRSVP.50.WR786842JOB11643.WRS20140.FILE0003.DAT ...
3
votes
1answer
116 views

Count and merge consecutive patterns

I'm searching for a short snippet to find, count and merge consecutive duplicates using standard tools or a common scripting language. Say our input is: 1 2 2 2 7 22 a b b c c c c c d dd 2 2 c c ...
1
vote
1answer
167 views

How to safely move a file to another directory and remove duplicates

I want to move files from Downloads, Documents, Pictures, Music, et al to shared locations. This is for various reasons, including because I want to keep media in a central, shared place. I want to ...
0
votes
1answer
187 views

Ping a list of IP addresses and outputing duplicates

In Linux how can I ping a list of IP addresses (in a text file) and only output those that have duplicate ping replies?
2
votes
2answers
124 views

Copy website from RedHat to CentOS

We currently have our website and website's CMS hosted on a dedicated host which runs Red Hat. We want to remove Red Hat and install CentOS instead. My question: is there a straight forward way of ...
5
votes
2answers
2k views

Remove duplicate mp3 with different name, size, and hash

I have a massive music library (all mp3), but I some of the music is almost the same but: Maybe one or two second longer About 97% the same as another song Or another bitrate. Is there a way to ...
39
votes
6answers
20k views

Find duplicate files

Is it possible to find duplicate files on my disk which are bit to bit identical but have different file-names?
1
vote
3answers
964 views

Killing all the process of a command except first process

I am sometimes stuck in a situation where a script/command kept in Cron runs more than once because of some reasons (the first instance is not completed fully, the second instance of the same process ...
12
votes
5answers
5k views

How can I copy a file and create the target directories at the same time?

I want to cp aaa/deep/sea/blob.psd to bbb/deep/sea/blob.psd How do I do the copy if the deep and sea directories don't exist under bbb so that the copy both creates the directories that are needed ...
0
votes
2answers
810 views

Do I have duplicate packages?

I'm on linux mint 12, when i run sudo apt-get update i see: W: Duplicate sources.list entry http://www.duinsoft.nl/pkg/ debs/all i386 Packages ...
0
votes
0answers
363 views

Remove duplicated Java packages

I'm on linux mint 12 I have installed Java, following the NetBeans instructions. It wasn't working on chrome, so I installed it like that: sudo echo 'deb http://www.duinsoft.nl/pkg debs all' ...
9
votes
1answer
4k views

Open source duplicate image finder for Linux?

Is there a free and open source duplicate image finder for Linux based systems? Finding exact duplicates (based on content, not file name) is sufficient for me, but the ability to find similar images ...
13
votes
5answers
5k views

case-insensitive search of duplicate file-names

I there a way to find all files in a directory with duplicate filenames, regardless of the casing (upper-case and/or lower-case)?
65
votes
16answers
18k views

Is there an easy way to replace duplicate files with hardlinks?

I'm looking for an easy way (a command or series of commands, probably involving find) to find duplicate files in two directories, and replace the files in one directory with hardlinks of the files in ...