0

So here's an excerpt of my directory tree:

|-- 20070214_014700.a  
|   |-- info  
|   |-- processed  
|       |-- HH.EL..BHZ  
|       |-- AZ.AS..HHZ  
|       |-- (hundreds more)  
|   |-- raw  
|   |-- resp  
|-- 20100737_055560.a  
|   |-- info  
|   |-- processed  
|   |-- raw  
|   |-- resp  
|-- 20190537_028750.a  
|   |-- info  
|   |-- processed  
|   |-- raw  
|   |-- resp  

I have ~13,000 directories (ending in .a) and each directory has a 'processed' subdirectory which has files I'd like to copy from every processed/ directory into a single directory. Some of these files may have the same filename so I'd also like to rename them based on their parent directory. I'm not too picky but something similar to:

20070214_014700_HH.EL..BHZ

The whole dataset is 3 TB so I've been testing on just a few directories using 'find':

find . -name processed -exec cp -r '{}' 'test/{}' \;

For some reason this dumps some files into test/ but also creates another processed/ directory inside of that. I'm not sure how to include a copy command and renaming function into find at the same time so any advice would be great. Thanks for the help.

0
find . -type f -path "./*.a/processed/*" -exec sh -c '
for path; do
  prefix=${path%%.a/processed*}
  cp "$path" "test/${prefix##*/}_${path##*processed/}"
done
' sh {} +

Option -type f searches for regular files in the given path and the -exec option starts a shell script with find's result as arguments ({} +). In the for-loop each argument is assigned to the path variable.

Example: If variable path is ./20070214_014700.a/processed/AZ.AS..HHZ, then

  • prefix=${path%%.a/processed*} removes the suffix -> ./20070214_014700
  • ${prefix##*/} removes the prefix to the first / -> 20070214_014700
  • ${path##*processed/} also removes the prefix and leaves the filename -> AZ.AS..HHZ

The resulting target filename of the cp command is test/20070214_014700_AZ.AS..HHZ.

| improve this answer | |
  • Perfect, tested it out and it fits the bill exactly. Thanks a lot for the help, just starting to learn the command line so your example is very educational. – LividNoodle Aug 24 '19 at 2:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.