0

I would like to copy files from the current directory which are divided into different subdirectories. The files I want to copy should only have True and False as their parents. True directory files should be copied in dst/report/1/ and False directory files should be in dst/report/2/.

I haven't been able to figure out how to automate the copying process:

  • I thought I could use the find . -maxdepth digit but I don't know what the exact depth is.

  • The files should only be *.txt files, and even though some names may be duplicated, their contents are different, so I need either of them in the destination folder, but cp would overwrite them.

  • True and False intermediate parent directories could exist, but my intended files are generated in the last parent directory.

  • In addition, directory space is about 200GiB, so it takes a while to search through it.

  • I'm not sure about escape characters in directory and files so I don't really want to break the copying process.

sample working space:

```
__ current dir
  |__ path_1
        |__ True
              |__ 00000.txt
              |__ 020.txt
  |__ p_x
        |__ 100
              |__ True
                    |__ 00000.txt
                    |__ 020.txt
                    |__ 10.txt
              |__ False
                    |__ 1.txt
                    |__ 2.txt
                    |__ 200.txt
        |__ x
              |__ True
                    |__ 00000.txt
                    |__ 020.txt
              |__ False
                    |__ 1.txt
                    |__ 2.txt
        |__ True
              |__ path_2
                    |__ True
                        |__ 1.txt
  |__ x_p
        |__ ...
  |__ ...
  .
  .
  .

Expected results:

   __ dst
      |__ report
          |__ 1
              |__ 00000.txt
              |__ 020.txt
              |__ 00000_1.txt
              |__ 020_1.txt
              |__ 10.txt
              |__ 1.txt
  
          |__ 2
              |__ 1.txt
              |__ 2.txt
              |__ 200.txt
              |__ 1_1.txt
              |__ 2_2.txt

Update

As @muru answered cp could be handful but there is a constraint in number of argument it can handle through bash (getconf ARG_MAX).

shopt -s globstar
cp --backup=numbered **/True/*.txt dst/report/1/
cp --backup=numbered **/False/*.txt dst/report/2/

So back then, after executing these commands, was getting -bash: /bin/cp: Argument list too long in return.

Again as @muru mentioned in comments, it could be possible to solve this issue via xargs (assuming GNU xargs and cp).

printf "%s\0" **/True/*.txt | xargs -0 cp --backup=numbered -t dst/report/1/

In case of duplicate files with number extension (e.g; bar.txt.~1~, etc) rename would be useful.

prename 's/(.txt).~(\d+)~$/-$2$1/' dst/report/*/*, 

1 Answer 1

2

With bash, enable the recursive glob ** with shopt -s globstar and then use **/True/*.txt and **/False/*.txt as patterns, and if you have GNU cp, use the --backups or -b options to avoid overwriting files:

shopt -s globstar
cp --backup=numbered **/True/*.txt dst/report/1/
cp --backup=numbered **/False/*.txt dst/report/2/

Note that cp will add numbering after the extension:

% touch foo.txt
% cp foo.txt bar.txt                   
% cp --backup=numbered foo.txt bar.txt
% ls bar.txt*
bar.txt  bar.txt.~1~
% cp --backup=numbered foo.txt bar.txt
% ls bar.txt*                         
bar.txt  bar.txt.~1~  bar.txt.~2~
5
  • Thanks, Is there any way to change bar.txt.~1~ to bar-1.txt and so on for other files. Thinking of find . -type f -exec sed -i 's/.txt.~number~/-number.txt/g' {} + but not sure how to keep the number for later use in sed.
    – amkyp
    Jun 5, 2022 at 14:21
  • 1
    If you have the Perl rename available in your OS, you can try that: prename 's/(.txt).~(\d+)~$/-$2$1/' dst/report/*/*, but then you still risk some files getting overwritten (e.g., if you had x.txt, x-1.txt, x.txt.~1~).
    – muru
    Jun 5, 2022 at 15:11
  • Hmmm, thanks @muru . Also I made that to work like this find . -type f -not -name "*.txt" | rename -E 's/.txt.~([0-9]+)~/___$1.txt/g'.
    – amkyp
    Jun 5, 2022 at 15:43
  • Hmmm, seems to be not working, getting error bash: /usr/bin/cp: Argument list too long.
    – amkyp
    Jun 5, 2022 at 20:08
  • 2
    @LocalHosT if you get that, then you can use xargs to work around that: printf "%s\0" **/True/*.txt | xargs -0 cp --backup=numbered -t dst/report/1/ (assuming GNU xargs and cp)
    – muru
    Jun 6, 2022 at 1:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.