0

In CentOS with Bash 3.2.52(2) I have to copy many files (not all) from one directory to another.

I can create a long one liner such as cp /"$HOME"/dir1/{file1,fil2} ... /"$HOME"/dir2 but because there are many files in dir1 I prefer to copy files in a multi line fashion.
EDIT: I create the list manually.

How can this be done?
I would prefer a no backslashes solution | I didn't find a clue in man cp | maybe only heredocument?

  • 1
    Do you want copy all files in dir1 directory or some? – Arkadiusz Drabczyk Aug 17 at 12:13
  • Just some in a multi line list I will make inside the script. – JohnDoea Aug 17 at 12:19
  • Why would you not want backslashes? Do the name of the files that you'd want to copy follow some pattern, or do the names of the files that you don't want to copy follow some pattern? – Kusalananda Aug 17 at 14:06
  • Repetitive usage of backslashes is something I personally feel bad with. None of the files necessarily follow any specific pattern; some might have extensions, some not, some would be named only by numbers, some only by letters, or a mix, some with or without special characters and so forth. – JohnDoea Aug 17 at 14:23
2
files=(
    file1 file2 "the goose incident.png"
    "another file"
    file3 file-4 "fifth file" file6
    "this is file7 with a
newline in the middle of the name" )

cd ~/dir1 &&
cp "${files[@]}" ~/dir2

This would copy the names mentioned in the files list from ~/dir1 to ~/dir2.

The newlines between the elements in the files list are not important, except for the newline in the last element which is a filename with a newline embedded in it (just to show that you can have these as well, without issues).

The list could also have been written

files=(
    file1
    file2
    "the goose incident.png"
    "another file"
    file3
    file-4
    "fifth file"
    file6
    "this is file7 with a
newline in the middle of the name"
)

or as

files=( file1 file2 "the goose incident.png" "another file" file3 file-4
        "fifth file" file6 "this is file7 with a
newline in the middle of the name" )
  • Thanks; please share why you chose to not separate some of the files to different lines (as with file1 and file2) and why are there strings inside? In appreciation, – JohnDoea Aug 17 at 14:41
  • @JohnDoea See update. Each string in the files list is a filename that you want to copy. Since you never said anything about what your filenames were, I had to invent my own, so I tried to cover all cases including single-word filenames, filenames with spaces, and filenames with newlines embedded in them. – Kusalananda Aug 17 at 14:47
  • With seq 1 1000000 files in the list it fails with /usr/bin/cp: Argument list too long. OP said that he needs to copy many files but didn't specify an exact number so I think he should be aware of that – Arkadiusz Drabczyk Aug 17 at 15:06
  • @ArkadiuszDrabczyk It does, but if the files don't follow a fixed pattern (not even a common filename prefix or suffix), it is less likely that the number of files is that great. As soon as either the files to include or the files to exclude follows any form of pattern (such as a numeric interval, a prefix/suffix or some other pattern), the names would no longer have to be stored in a list at all, as it becomes a simple matter of testing the names against a set of conditions. – Kusalananda Aug 17 at 15:19
0

How about this:

#!/usr/bin/env bash

files="1
2
3
4
4
5
6
7
8
9 10 11"

IFS=$'\n'
for file in $files
do
    touch "$file"
done

Just replace touch "$file" with what you need. The disadvantage of this solution is that it will fork a new process for each file as you can see with strace and thus will be slow for a large number of files:

$ strace -f ./cp-here-doc.sh  |& grep 'execve("/usr/bin/touch"'
[pid 17917] execve("/usr/bin/touch", ["touch", "1"], [/* 63 vars */]) = 0
[pid 17918] execve("/usr/bin/touch", ["touch", "2"], [/* 63 vars */]) = 0
[pid 17919] execve("/usr/bin/touch", ["touch", "3"], [/* 63 vars */]) = 0
[pid 17920] execve("/usr/bin/touch", ["touch", "4"], [/* 63 vars */]) = 0
[pid 17921] execve("/usr/bin/touch", ["touch", "4"], [/* 63 vars */]) = 0
[pid 17922] execve("/usr/bin/touch", ["touch", "5"], [/* 63 vars */]) = 0
[pid 17923] execve("/usr/bin/touch", ["touch", "6"], [/* 63 vars */]) = 0
[pid 17924] execve("/usr/bin/touch", ["touch", "7"], [/* 63 vars */]) = 0
[pid 17925] execve("/usr/bin/touch", ["touch", "8"], [/* 63 vars */]) = 0
[pid 17926] execve("/usr/bin/touch", ["touch", "9 10 11"], [/* 63 vars */]) = 0

You can use xargs to run touch, or cp in your final script only once and make script run faster:

#!/usr/bin/env bash

files="1
2
3
4
4
5
6
7
8
9 10 11"

echo "$files" | tr '\n' '\0'  | xargs -0 touch

Result:

$ strace -f ./cp-here-doc.sh  |& grep 'execve("/usr/bin/touch"'
[pid 18290] execve("/usr/bin/touch", ["touch", "1", "2", "3", "4", "4", "5", "6", "7", "8", "9 10 11"], [/* 63 vars */]) = 0

Also notice that on Linux at least filenames can contain newlines so you have to pick another delimiter if at least one of your filenames contains a newline.

OP asked:

what does IFS=$'\n' mean

It means a literal new line. You can read about it man bash:

   Words of the form $'string' are treated specially.  The word
   expands to string, with backslash-escaped char- acters replaced
   as specified by the ANSI C standard.  Backslash escape
   sequences, if present, are decoded as follows:
          \a     alert (bell)
          \b     backspace
          \e
          \E     an escape character
          \f     form feed
          \n     new line

Your final script could look like this:

#!/usr/bin/env bash

files="1
2
3
4
5
6
7
8
9 10 11"

echo "$files" | tr '\n' '\0'  | xargs -0 cp -t {} dir

I'd really go with xargs option as it is much faster and safer - see Kusalananda's comment below. I didn't run tests with cp command but when I tested touch to create files in the $(seq 1 1000000) list it took only 16 seconds for xargs to do this but it took a whooping 42 minutes for the for loop version. Also, to my surprise xargs was able to split argument list into several commands so that Argument list too long will not be an issue.

  • Hi and thanks; please tell me, what does IFS=$'\n' mean? Also, shouldn't touch create empty files instead of replicating the "copied" files? Thank you, – JohnDoea Aug 17 at 13:24
  • See updated answer. Yes, touch is just an example as I said. We don't know what is your destination directory and what options you want to use with cp. – Arkadiusz Drabczyk Aug 17 at 13:32
  • Unless you also use set -f, the shell will additionally perform filename globbing on each of the split-up words. – Kusalananda Aug 17 at 14:10
  • @Kusalananda: ok, thanks, I forgot about that. I've updated my answer. – Arkadiusz Drabczyk Aug 17 at 15:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.