The tag has no wiki summary.

learn more… | top users | synonyms

47
votes
15answers
12k views

Is there an easy way to replace duplicate files with hardlinks?

I'm looking for an easy way (a command or series of commands, probably involving find) to find duplicate files in two directories, and replace the files in one directory with hardlinks of the files in ...
11
votes
5answers
1k views

How can I copy a file and create the target directories at the same time?

I want to cp aaa/deep/sea/blob.psd to bbb/deep/sea/blob.psd How do I do the copy if the deep and sea directories don't exist under bbb so that the copy both creates the directories that are needed ...
11
votes
4answers
3k views

Duplicating a Linux installation (Yum-based)

Given an installation based on Yum (specifically in my case, a Scientific Linux 5.1 x86_64 installation), how would I duplicate the installed programs and utilities to a new machine based on Fedora ...
10
votes
5answers
3k views

case-insensitive search of duplicate file-names

I there a way to find all files in a directory with duplicate filenames, regardless of the casing (upper-case and/or lower-case)?
9
votes
1answer
3k views

Open source duplicate image finder for Linux?

Is there a free and open source duplicate image finder for Linux based systems? Finding exact duplicates (based on content, not file name) is sufficient for me, but the ability to find similar images ...
6
votes
3answers
159 views

Shell script to search files for identical text entries

Here's a nice teaser for a shell scripting guru: Take a directory with several text files. Can be a few up to ~1000. All files contain an identifier on a given line (always the same line). Identify ...
4
votes
4answers
195 views

Removing lines that are common to the outputs produced by two commands

Say, I have two commands that return some text. For example: $ ./c1 /usr/bin/foo /usr/bin/bar /usr/bin/baz $ ./c2 /usr/bin/foo /usr/bin/qux /usr/bin/buzz /usr/bin/bar I want to remove the duplicate ...
4
votes
2answers
131 views

Faster way to rename duplicate files (identified by fdupes) in another directory?

I have a directory full of pdf files of journal articles, most of which are named by their bibtex key. Some time ago I made a backup on an external hard drive, but I haven't kept it up to date and ...
3
votes
4answers
215 views

Use basename to parse a list of paths held in a file

I'm running Mac OSX and trying to use the command line to find the number of files I have with the same name. I tried to use the following command: find ~ -type f -name "*" -print | basename | sort ...
3
votes
2answers
603 views

Remove duplicate mp3 with different name, size, and hash

I have a massive music library (all mp3), but I some of the music is almost the same but: Maybe one or two second longer About 97% the same as another song Or another bitrate. Is there a way to ...
2
votes
2answers
103 views

Copy website from RedHat to CentOS

We currently have our website and website's CMS hosted on a dedicated host which runs Red Hat. We want to remove Red Hat and install CentOS instead. My question: is there a straight forward way of ...
2
votes
1answer
45 views

Count and merge consecutive patterns

I'm searching for a short snippet to find, count and merge consecutive duplicates using standard tools or a common scripting language. Say our input is: 1 2 2 2 7 22 a b b c c c c c d dd 2 2 c c ...
1
vote
3answers
581 views

Killing all the process of a command except first process

I am sometimes stuck in a situation where a script/command kept in Cron runs more than once because of some reasons (the first instance is not completed fully, the second instance of the same process ...
1
vote
2answers
226 views

Find duplicate file names with specific matching pattern

I have a folder with some files (snippet of the contents of the folder) PAT1.URGRSVP.50.WR786842JOB11632.WRS20140.FILE0005.DAT PAT1.URGRSVP.50.WR786842JOB11643.WRS20140.FILE0003.DAT ...
1
vote
0answers
33 views

Removing duplicate files but ignoring certain lines (e.g. embedded timestamp)

I have a bunch of gzipped text files. I am trying to remove duplicate files (using fdupes) but the problem is these files contain a line that has their timestamp in them, and the files are identical ...
1
vote
1answer
119 views

How to safely move a file to another directory and remove duplicates

I want to move files from Downloads, Documents, Pictures, Music, et al to shared locations. This is for various reasons, including because I want to keep media in a central, shared place. I want to ...
0
votes
1answer
137 views

Ping a list of IP addresses and outputing duplicates

In Linux how can I ping a list of IP addresses (in a text file) and only output those that have duplicate ping replies?
0
votes
2answers
659 views

Do I have duplicate packages?

I'm on linux mint 12, when i run sudo apt-get update i see: W: Duplicate sources.list entry http://www.duinsoft.nl/pkg/ debs/all i386 Packages ...
0
votes
0answers
306 views

Remove duplicated Java packages

I'm on linux mint 12 I have installed Java, following the NetBeans instructions. It wasn't working on chrome, so I installed it like that: sudo echo 'deb http://www.duinsoft.nl/pkg debs all' ...