2

I have a thousand .xyz files, all numerical data and in the same directory. Some of them have the text string "END" at the very end of them. I'm not sure how to loop a grep command so that it removes the "END" and creates a new file, while appending something to end of the original filename.

Example of the last lines of some of my files

-1.10949170 55.68890280 -67.60000000
-0.92807500 55.64095280 -89.80000000
-0.95770560 55.66495830 -82.40000000
-0.90995000 55.63676110 -94.00000000
-1.03738890 55.65107220 -70.20000000
END

I want this (removing the END) from many files

-1.10949170 55.68890280 -67.60000000
-0.92807500 55.64095280 -89.80000000
-0.95770560 55.66495830 -82.40000000
-0.90995000 55.63676110 -94.00000000
-1.03738890 55.65107220 -70.20000000

Original file name: survey_2015_xxx.xyz
New file name: survey_2015_xxx_s.xyz

3

If your head supports negative offsets:

for file in *.xyz; do
  if [ "$(tail -n 1 < "$file")" = END ]; then
    head -n -1 < "$file" > "${file%.xyz}_s.xyz"
  fi
done

(if it doesn't, replace head -n -1 with sed '$d').

You can make it more efficient with ksh93:

for file in *.xyz; do
  if IFS= read -r last4 < "$file" <#((EOF-4)) <#((here=CUR)) &&
    [ "$last4" = END ]; then
    command /opt/ast/bin/head -c "$here" < "$file" > "${file/%.xyz/_s\0}"
  fi
done

As it's only using builtin commands.

If your file system supports reflink copies (that is copying files where the data is not duplicated until modified which here would save time and disk space), you could do, still with ksh93 and with GNU cp:

for file in *.xyz; do
  if IFS= read -r last4 < "$file" <#((EOF-4)) &&
    [ "$last4" = END ]; then
    newfile=${file/%.xyz/_s\0}
    cp --reflink=auto -- "$file" "$newfile" &&
      : 1<>; "$newfile" >#((EOF-4))
  fi
done

That is, we're making a reflink copy and truncating it 4 bytes from the end.

1

An ed alternative, because you only live once:

for f in *.xyz; do
    printf '%s\n' ';g/END/d\' "w ${f%.xyz}_s.xyz" q | ed -s "$f"
done

Filename caveats:

  • Filenames cannot begin with !, otherwise ed will attempt to pipe the buffer into a command.
  • Since ed uses blank characters to delimit the w command from its pathname argument, leading blanks in filenames will not be preserved.
  • Filenames with newlines will cause problems since ed uses newlines to delimit commands.

In a sane world, none of these limitations are relevant. Alas ...

  • Yeah, ed can definitely do this job ... With some escaping and a slight change of code you can work around those limitations and deal with pretty much any kind of filename. Not worth it though. – don_crissti Feb 26 '16 at 11:33
0

Just other solution using sed

for file in *.xyz
do
    if [ "$(sed -e '$!d' $file)" = END ]
    then
        sed -e '$d' $file > $file_s.xyz
    fi
done

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.