How do you truncate all columns in a file to a specific length?

Question

Given columned text in a file that is delimited by a certain character, how do you truncate the width of either a specific column(s) or all columns?

The use case is that I am reading some log files that have space delimited columns. Some of the columns in the log are very long urls that make the file difficult to read. I don't really care about the entire content of the urls, so I would like to truncate the length of these columns to something more readable.

I would be curious to know how to specify specific columns to truncate, but really I only need to figure out how to just truncate all columns of length greater than N.

So far I have found utils column, colrm, and cut, which all seem to be similar to this, but nothing that does what I am looking for.

(Optimally, if there is a compiled utility that already does that, this would be best, but I would also be happy to find a way to do this with sed, awk, perl, etc.)

Answer 1

sed 's/\([^,]\{0,3\}\)[^,]*/\1/g' will truncate each column to first 3 characters. Columns are delimited by ',' character.

How it works:

1. It finds (greedy) a string consisting of 0 to 3 characters other than ',' and remembers it as first group (referenced by \1)

2. Then it tries to find as much more characters other than ',' as it could.

3. All of this is substituted with group 1 (which is first 3 characters).

4. Because of g option, it will do this multiple times each line, truncating all the columns, not only first one.

EDIT:

As pointed out in comment, 1st step can be optimised if you use ([^,]\{3\}\). This way we are only matching exactly 3 characters. If there are less, we don't care as we don't have to truncate this field. So the final command would be:

`sed 's/\([^,]\{3\}\)[^,]*/\1/g'`

Answer 2

I could image that it's not essential that the file is structured in columns since you wrote that you are annoyed by the long urls. At least in the case of parsing the file instead of viewing it.

In this case my solution is

perl -pe 's%(http://\S+)%substr($1, 0, 15)%ge' <file_with_long_urls> | column -t

This removes all characters of the url behind the fifteenth character after a http://.

If the columns are not delimited by whitespaces but with the character & (which stands for an arbitrary delimiter) my solution looks like

perl -pe 's%(http://\S+?)(?=&|$)%substr($1, 0, 15)%ge' <file_with_long_urls> | column -s '&' -t

Maybe you can work with this!?