8

Given input of the form

XY981743    foobarlkasdf saflkas asfZR!sgfad asdSAD asdsadf SAdfasdf46lk lksad bar foolkasjfdrte

how can I truncate only the second column? The delimiter is TAB and the second column must be at most 75 characters long.

  • A bit more generic: awk 'BEGIN{OFS=FS="\t"} {$2=substr($2,1,75)}1' file – fedorqui May 11 '15 at 9:27
  • Do you want to truncate (delete characters after the 75th) or fold (print them on another line)? Also, should the spaces be counted towards the 75 characters or not? – terdon May 11 '15 at 10:41
6

If you want to print only the first 75 characters of the second column (including spaces, and assuming only two columns in the file), you can do:

$ perl -pe 's/(\t.{75}).*/$1/' file
XY981743    foobarlkasdf saflkas asfZR!sgfad asdSAD asdsadf SAdfasdf46lk lksad bar fool

Or, with GNU sed:

$ sed 's/\(.*\t.\{75\}\).*/\1/' file
XY981743    foobarlkasdf saflkas asfZR!sgfad asdSAD asdsadf SAdfasdf46lk lksad bar fool

Or:

$ sed -r 's/(.*\t.{75}).*/\1/' file
XY981743    foobarlkasdf saflkas asfZR!sgfad asdSAD asdsadf SAdfasdf46lk lksad bar fool

Alternatively, you could use fold, telling it to cut at the first 91 characters (that's 8 for the identifier and another 8 for the tab), and printing only the first line:

$ fold -w 91 file | head -n1
XY981743    foobarlkasdf saflkas asfZR!sgfad asdSAD asdsadf SAdfasdf46lk lksad bar fool

If your file can have more than 2 columns and you only want to truncate the second, you can do (which, as I just noticed, is just a rewording of Stephen's answer):

$ awk -F"\t" -vOFS="\t" '{$2=substr($2,1,75)}1;' file
XY981743    foobarlkasdf saflkas asfZR!sgfad asdSAD asdsadf SAdfasdf46lk lksad bar fool

Or (note that this will break if the first 75 characters of the 2nd column can be interpreted as a regular expression):

$ perl -F"\t" -pale 's/$F[1]/substr($F[1],0,75)/e' file
XY981743    foobarlkasdf saflkas asfZR!sgfad asdSAD asdsadf SAdfasdf46lk lksad bar fool
  • Those may truncate an other columns than the second one. Your first sed command is also using a GNUism (\t). – Stéphane Chazelas May 11 '15 at 11:01
  • @StéphaneChazelas what? \t is a GNUism? Seriously? What is the portable way of describing a tab then? – terdon May 11 '15 at 11:04
  • 1
    Insert it verbatim of see my answer. The only escape sequence recognised portably on the LHS is \n (and again not inside [...] with many implementations), none on the RHS. – Stéphane Chazelas May 11 '15 at 11:06
  • @StéphaneChazelas damn, thanks. I also added a solution which can handle multiple columns. – terdon May 11 '15 at 11:10
  • Your last perl one makes little sense. Think for instance of an input like aba\t.* – Stéphane Chazelas May 11 '15 at 13:37
10

Using awk, split the file using tabs and output the first field in full and the first 75 characters (at most) of the second:

awk -F "\t" 'BEGIN { OFS=FS }; { print $1, substr($2, 1, 75); }'

As pointed out by fedorqui, you can handle files with more than two fields by replacing the fields you need to truncate:

awk -F "\t" 'BEGIN { OFS=FS }; { $2=substr($2, 1, 75); print }'

You could apply the substr to multiple fields by looping over them if necessary.

  • @Stéphane, in which cases are the extra ; necessary? – Stephen Kitt May 11 '15 at 11:06
  • They are required by POSIX. Now I don't know of any implementation where they are required, but when I asked that that POSIX requirement be relaxed, it was rejected by the gawk maintainer (where all examples in the doc have the ;). – Stéphane Chazelas May 11 '15 at 11:11
  • Ah, good to know, thanks! So the spec and the documentation are stricter than all the implementations... – Stephen Kitt May 11 '15 at 11:17
  • all the implementations that I know of at least (not so many). The point is omitting the delimiter makes for non-standard syntax. So current and future implementations may and are entitled to choke on it or introduce extensions that make use of it (like an exception handling one /pattern/ {action} {exception-handling} for instance). Now that would be extremely unlikely given that omitting those ; is fairly common. – Stéphane Chazelas May 11 '15 at 12:08
4

Portably/POSIXly with sed:

tab=$(printf '\t')
sed "s/\($tab[^$tab]\{0,75\}\)[^$tab]*/\1/"

Or to truncate every column:

sed "s/\([^$tab]\{75\}\)[^$tab]*/\1/g"
2

If there are only 2 columns:

sed -r 's/^([^\t]*\t)(.{0,75}).*/\1\2/'

{0,75} means select from 0 up to 75 chars.
.* is the removed section beyond char 75.


If there are 2 or more columns:

sed -r 's/^([^\t]*\t)([^\t]{0,75})[^\t]*(.*)/\1\2\3/' file

[^\t]* is the removed section beyond char 75.

  • Note that that assumes GNU sed and that POSIXLY_CORRECT is not in the environment.. – Stéphane Chazelas May 11 '15 at 10:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.