How can I trim the contents of a text file?

Question

I want to remove all "blank" characters from the very beginning and the very end of a text file, including \n if exists. (basically mimicking the behaviour of trim() function of most programming languages, if the "file" was a big string).

Answer 1

Use sed:

sed -z 's/^\s*//; s/\s*$//' infile

• s/^\s*//, deletes whitespaces/empty lines at very begging of the infile as a input file.

• s/\s*$//, deleted whitespaces/empty lines at very end of the infile as a input file including \n at very end of infile.

Example cat -e infile:

$
$
$
Three blank lines above$
$
$
Two blank lines in the middle$
a blank lines below$
$
a line with trailing whitespaces                $
          a line with leading whitespaces$
below are two empty lines + one whitespaces then an empty line again$
$
$
                                    $
$

The output:

Three blank lines above


Two blank lines in the middle
a blank lines below

a line with trailing whitespaces
          a line with leading whitespaces
below are two empty lines + one whitespaces then an empty line again

---

Or you can use printf to print the result of sed that removed very first whitespaces/empty lines and used it within command substitution that deletes empty lines only at very end and \n.

printf '%s' "$(sed -z 's/^\s*//' infile)"

Answer 2

Assuming you want to remove all spaces and newlines, not just the last newline, you can use tr:

tr -d '[[:space:]]' < file > file.trimmed

Or, more precisely:

tr -d '\t\n ' < file > file.trimmed

Answer 3

Assuming you want to remove leading/trailing whitespace for each line and not the file as a whole.

A test file, where some lines have leading or trailing whitespace, and blank lines:

$ cat -e file
line 1$
  line 2$
line 3 has trailing spaces   $
$
blank line above$

And a GNU sed command to do the trimming

$ sed -r 's/^\s+//; s/\s+$//; /^$/d' file | cat -e
line 1$
line 2$
line 3 has trailing spaces$
blank line above$

Answer 4

Well, not shell script, but I wrote a PHP solution:

script name: trimfile.php

<?php
if ( (!empty($argv[1])) && (file_exists($argv[1])) )  {
    $file_contents = file_get_contents($argv[1]);
    $new_file_contents = trim($file_contents) . "\n";
    if ($file_contents <> $new_file_contents) {
        file_put_contents($argv[1], $new_file_contents);
    }
}
?>

Usage:

$ php trimfile.php yourfile.txt

Answer 5

You can try with ed which unfortunately don't came in standard install of debian.

printf '%s\n' \
 'g/[^[:blank:]][^[:blank:]]*/kx' \
 "'"'x,$j' \
 '$s/[[:blank:]]*$//' \
 '/[^[:blank:]][^[:blank:]]*/kx' \
 '1,'"'"'xj' \
 '1s/^[[:blank:]]*//' \
 'wq' | 
ed -s infile

Or with gnu sed

sed -Ezi 's/^\s+|\s+$//g;s/$/\n/' infile

Answer 6

In awk:

                 { isblank = 0 }
/^[[:blank:]]*$/ { isblank = 1 }

state == 0 &&  isblank { next }
state == 0 && !isblank { state = 1 }

state == 1 &&  isblank { buffer[++n] = $0 }
state == 1 && !isblank { for (i = 1; i <= n; ++i) print buffer[i]; n = 0; print }

state denotes what state the program is in. When state == 0, the program will remove any blank line. A blank line is a line that is empty or that only contains spaces or tabs (which is what [[:blank:]] matches). The program stays in state == 0 until a non-blank line is found. This trims the beginning of the file.

When state == 1, the program will save blank lines into the buffer array. These blank lines needs to be outputted as soon as a non-blank line is found. If no non-blank line is found, they will be discarded (this trims the end of the file).

The overall effect is that blank lines at the start and end of the file are removed, while blank lines elsewhere are preserved (including their spaces or tabs, if they contained these in the original file).

Testing:

$ cat file



Three blank lines above


Two blank lines in the middle
Three blank lines below



$ awk -f trim.awk file
Three blank lines above


Two blank lines in the middle
Three blank lines below

The program does not modify blanks at the start of the first non-blank line nor at the end of the last non-blank line. If that is needed, one could use

$ awk -f trim.awk file | sed -e '1s/^[[:blank:]]*//' -e '$s/[[:blank:]]*$/'