0

I would like to copy files from the current directory which are divided into different subdirectories. The files I want to copy should only have True and False as their parents. True directory files should be copied in dst/report/1/ and False directory files should be in dst/report/2/.

I haven't been able to figure out how to automate the copying process:

  • I thought I could use the find . -maxdepth digit but I don't know what the exact depth is.

  • The files should only be *.txt files, and even though some names may be duplicated, their contents are different, so I need either of them in the destination folder, but cp would overwrite them.

  • True and False intermediate parent directories could exist, but my intended files are generated in the last parent directory.

  • In addition, directory space is about 200GiB, so it takes a while to search through it.

  • I'm not sure about escape characters in directory and files so I don't really want to break the copying process.

sample working space:

```
__ current dir
  |__ path_1
        |__ True
              |__ 00000.txt
              |__ 020.txt
  |__ p_x
        |__ 100
              |__ True
                    |__ 00000.txt
                    |__ 020.txt
                    |__ 10.txt
              |__ False
                    |__ 1.txt
                    |__ 2.txt
                    |__ 200.txt
        |__ x
              |__ True
                    |__ 00000.txt
                    |__ 020.txt
              |__ False
                    |__ 1.txt
                    |__ 2.txt
        |__ True
              |__ path_2
                    |__ True
                        |__ 1.txt
  |__ x_p
        |__ ...
  |__ ...
  .
  .
  .

Expected results:

   __ dst
      |__ report
          |__ 1
              |__ 00000.txt
              |__ 020.txt
              |__ 00000_1.txt
              |__ 020_1.txt
              |__ 10.txt
              |__ 1.txt
  
          |__ 2
              |__ 1.txt
              |__ 2.txt
              |__ 200.txt
              |__ 1_1.txt
              |__ 2_2.txt

Update

As @muru answered cp could be handful but there is a constraint in number of argument it can handle through bash (getconf ARG_MAX).

shopt -s globstar
cp --backup=numbered **/True/*.txt dst/report/1/
cp --backup=numbered **/False/*.txt dst/report/2/

So back then, after executing these commands, was getting -bash: /bin/cp: Argument list too long in return.

Again as @muru mentioned in comments, it could be possible to solve this issue via xargs (assuming GNU xargs and cp).

printf "%s\0" **/True/*.txt | xargs -0 cp --backup=numbered -t dst/report/1/

In case of duplicate files with number extension (e.g; bar.txt.~1~, etc) rename would be useful.

prename 's/(.txt).~(\d+)~$/-$2$1/' dst/report/*/*, 

1 Answer 1

3

With bash, enable the recursive glob ** with shopt -s globstar and then use **/True/*.txt and **/False/*.txt as patterns, and if you have GNU cp, use the --backups or -b options to avoid overwriting files:

shopt -s globstar
cp --backup=numbered **/True/*.txt dst/report/1/
cp --backup=numbered **/False/*.txt dst/report/2/

Note that cp will add numbering after the extension:

% touch foo.txt
% cp foo.txt bar.txt                   
% cp --backup=numbered foo.txt bar.txt
% ls bar.txt*
bar.txt  bar.txt.~1~
% cp --backup=numbered foo.txt bar.txt
% ls bar.txt*                         
bar.txt  bar.txt.~1~  bar.txt.~2~
5
  • Thanks, Is there any way to change bar.txt.~1~ to bar-1.txt and so on for other files. Thinking of find . -type f -exec sed -i 's/.txt.~number~/-number.txt/g' {} + but not sure how to keep the number for later use in sed.
    – amkyp
    Jun 5, 2022 at 14:21
  • 1
    If you have the Perl rename available in your OS, you can try that: prename 's/(.txt).~(\d+)~$/-$2$1/' dst/report/*/*, but then you still risk some files getting overwritten (e.g., if you had x.txt, x-1.txt, x.txt.~1~).
    – muru
    Jun 5, 2022 at 15:11
  • Hmmm, thanks @muru . Also I made that to work like this find . -type f -not -name "*.txt" | rename -E 's/.txt.~([0-9]+)~/___$1.txt/g'.
    – amkyp
    Jun 5, 2022 at 15:43
  • Hmmm, seems to be not working, getting error bash: /usr/bin/cp: Argument list too long.
    – amkyp
    Jun 5, 2022 at 20:08
  • 2
    @LocalHosT if you get that, then you can use xargs to work around that: printf "%s\0" **/True/*.txt | xargs -0 cp --backup=numbered -t dst/report/1/ (assuming GNU xargs and cp)
    – muru
    Jun 6, 2022 at 1:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .