I have a problem in cleaning my file where there are unnecessary free spaces below my last data row. This is created when I am copying and pasting my data from excel to text. I want my text file to fit my data only. This will reduce the computational time when I am running my model. I just came to know the size of my file is big due to the unwanted free space below my last data row.

I want you help me how clean up the unwanted rows.

awk '{l=l$0 RT};NF{printf "%s",l;l=""}' < your-file

Would remove the trailing blank lines.

If the file is big, you may want to read it from the end until you find the first non-blank line.

With GNU tools:

tac your-file | sed -ne '/[^[:blank:]]/!{p;d;}' -e q | wc -c

Would return the size of the empty lines at the end.

And you could use that to truncate the file.

For instance with bash:

find . -type f -name '*.txt' -print0 |
  while IFS= read -rd '' file; do
    n=$(tac "$file" | sed -ne '/[^[:blank:]]/!{p;d;}' -e q | wc -c)
    [ "$n" -eq 0 ] || truncate -r "$file" "-s-$n" "$file"
| improve this answer | |
  • Thank you. It is working, but I have lots of files to clean. Can you modify the code a little bit to direct the input files and store the output in to the same file while running the code? – AiB Sep 3 '13 at 18:58
  • My file names looks data_lat_long. All the files start with data_ and vary with the lat_long. – AiB Sep 3 '13 at 19:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.