0

I have a deeply nested folder structure in which there are hundreds of files called data.log. I need a script to rename each of these data.log files according to the name of the parent folder they are in and then move the renamed filed to a defined target folder. The original data.log files should remain in place.

Example:

The file /opt/slm/data/system/amd-823723/data.log needs to be renamed to amd-823723 and then moved to /opt/slm/output/, whereby the original data.log file remains in place.

  • What is the parent directory we are searching? /opt/slm/data/system? /opt/slm/data? Something else? Will all the new files be in /opt/slm/output/? – Jesse_b May 24 at 17:33
  • Why do you say the file will be moved and at the same time the file remains in place? – RalfFriedl May 24 at 17:35
  • 1
    You want to rename each file but then move it and also message the original in place? That sounds like you're actually describing a copy/rename rather then a rename/move/keep. – roaima May 24 at 18:00
  • "The file /opt/slm/data/system/amd-823723/data.log needs to be renamed to amd-823723." The parent directory is the directory in which the "data.log" file is. The file "data.log" should be copied to the output and renamed to its original parent folder. Sorry for not being clear enough on this one. – Marco Stahl May 27 at 9:40
1
#!/bin/bash
OUTDIR=/opt/slm/output/

find /opt/slm/data -name data.log |
while read FILE; do

  OUTFILE="$(basename "$(dirname "$FILE")")"
  cp -p "$FILE" "$OUTDIR$OUTFILE"

done
  • 1
    ok, I made it work but more quoting was needed: OUTFILE="$(basename "$(dirname "$FILE")")" – Marco Stahl May 27 at 9:49
  • This will fail when you have files with newline. Use find -print0 and maybe something like this stackoverflow.com/questions/8677546/…, or xargs -0. – pLumo May 27 at 11:45
  • It solved it for me. :) – Marco Stahl May 28 at 9:26
1

With zsh:

autoload zmv # best in ~/.zshrc
zmv -n -C '**/(*)/data.log' '/opt/slm/output/$1'

remove the -n (for dry run) if happy.

The -C is for copy (as oppose to rename/mv) as even though your description says rename, it looks like you want to copy the files instead into the output directory.

1

Use find -exec:

find /opt/slm/data -name data.log -exec sh -c '
      TARGET="/opt/slm/output/$(basename "$(dirname "$1")")";
      cp -i "$1" "$TARGET"
    ' find-sh {} \;
-1

Bash:

while read -r line; do
    copy="/opt/slm/output/$(sed 's/^.*\/\(.*\)\/data.log/\1/' <<<$line)"
    cp -v "$line" "$copy"
done <<<"/opt/slm/data/system/amd-823723/data.log"

Seems like a simple enough solution. Replace <<<"/opt/slm/data/system/amd-823723/data.log" with appropriate find command, or something such.

  • copy="/opt/slm/output/$(awk -F'/' '{print $NF}' <(dirname $line))" is probably neater and easier to follow, though. – Kaffe Myers May 27 at 11:47
  • Your while should be while IFS=, otherwise it will eat leading and trailing spaces. – Sparhawk May 27 at 11:50
  • sed and awk are both a bad idea as they both work on a line-basis and file names are allowed to have newlines. Your loop will already fail on this. – pLumo May 27 at 11:51
  • It's very corner-case with newline filenames, but you are correct. – Kaffe Myers May 27 at 11:59
-1

the solution only works for directory names without spaces:

#!/bin/bash
targetDir=/opt/slm/output/
for myPath in `find /opt/slm/data/system |grep "data.log$"`
do
    newName = `echo "$myPath" |awk 'BEGIN {FS="/"} /data.log/ {print $(NF-1)}'`
    cp "$myPath" ${targetDir}"${newName}"
done

in fact all the given solutions, including mine, are wrong because they lead to the loss of information for the catalogs:

a/a/data.log
a/data.log
b/a/data.log

all will have the name "a" and will be overwritten with the last file b / a / data.log

this problem is solved by the example program in python3:

import os
for x in (os.walk('/opt/slm/data/system')):
     if x[2] == ['data.log']:
        newName=x[0].replace('/','#').replace(' ','_')
        os.system('cp "'+str(x[0])+'/data.log" /opt/slm/output/'+str(newName))

Creates names that contain the full path and the _ character instead of the space:

#opt#slm#data#system#dirname

Of course, naming and paths have to be adapted to your needs.

  • Thank you so much, Slawomir! I will try that out asap and let you know. – Marco Stahl May 25 at 10:21
  • "cp: cannot stat ‘Library)/Sub/C0013S01/data.log’: No such file or directory cprn.sh: line 5: newName: command not found". That what I get as bash output. – Marco Stahl May 27 at 9:14
  • in the example you provided, there were no parentheses') or space in the names of directories and then it works perfectly. You must therefore adjust the $myPath call to add escape sequences before each special character in the name. I can check it out only for a dozen or so hours. – Slawomir Dziuba May 27 at 10:45
  • check now, I added a protection against a space in the name of the path. – Slawomir Dziuba May 27 at 11:00
  • -1 , because: 1. Do not use loops like this. Never. 2. Do not use grep to filter find results. find is just perfect for doing exactly this. 3. awk seems to be clever here, but keep in mind that filenames are allowed to have newline. This will make your awk fail. – pLumo May 27 at 11:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.