How to remove column from file without altering format?

Question

I need to remove the first column from files like this:

165 1   chr22   42090593    0   1   chr22   42090609    1   42  42
166 1   chr22   42090593    0   1   chr22   42090654    1   42  42
167 1   chr22   42090595    0   1   chr22   42090633    1   42  42
168 0   chr22   42090612    0   1   chr22   42090656    1   42  42
169 0   chr22   42090614    0   0   chr22   42090617    1   40  42
170 0   chr22   42090647    0   1   chr22   42090749    1   42  42
171 1   chr22   42090684    0   1   chr22   42090692    1   42  42
172 1   chr22   42090733    0   1   chr22   42090743    1   42  42
173 1   chr22   42090733    0   1   chr22   42090775    1   42  42
174 1   chr22   42090733    0   1   chr22   42090789    1   42  42
175 1   chr22   42090757    0   1   chr22   42090787    1   42  24
176 0   chr22   42090778    0   0   chr22   42090790    1   42  42
177 0   chr22   42090800    0   0   chr22   42090802    1   42  42
178 0   chr22   42090803    0   0   chr22   42090806    1   42  42

The command

awk '{$1=""; print $0}'

correctly remove the first column but alters the format this way

1 chr22 51178322 0 0 chr22 51178659 1 42 42
0 chr22 51178661 0 0 chr22 51178663 1 42 42
0 chr22 51178667 0 1 chr22 51178790 1 42 23
1 chr22 51178755 0 0 chr22 51178764 1 42 42
0 chr22 51178808 0 1 chr22 51178871 1 42 42
1 chr22 51178869 0 1 chr22 51178895 1 42 42
1 chr22 51178881 0 1 chr22 51178893 1 42 42
1 chr22 51178881 0 1 chr22 51178895 1 42 42
1 chr22 51179213 0 1 chr22 51179213 1 42 42
1 chr22 51180087 0 1 chr22 51180093 1 42 42
1 chr22 51180134 0 0 chr22 51181889 1 42 42
0 chr22 51186192 0 0 chr22 51186192 1 42 42
0 chr22 51186192 0 0 chr22 51186192 1 42 42

Any idea?

Answer 1

There are two issues with your approach. First, that looks like a tab-delimited file and you are not telling awk to use tabs. Second, when you set a field to "" in awk, you're not deleting the field, you are just emptying it. So it is still printed and that's why you have an extra space at the beginning of each line in your output.

So, if you want to do this in awk, you would need something like this (this assumes the leading spaces in your example aren't actually part of your file):

$ awk -F"\t" 'BEGIN{OFS="\t"}{for(i=2;i<NF;i++){printf "%s%s",$i,OFS} print $NF}' file 
1   chr22   42090593    0   1   chr22   42090609    1   42  42
1   chr22   42090593    0   1   chr22   42090654    1   42  42
1   chr22   42090595    0   1   chr22   42090633    1   42  42
0   chr22   42090612    0   1   chr22   42090656    1   42  42
0   chr22   42090614    0   0   chr22   42090617    1   40  42
0   chr22   42090647    0   1   chr22   42090749    1   42  42
1   chr22   42090684    0   1   chr22   42090692    1   42  42
1   chr22   42090733    0   1   chr22   42090743    1   42  42
1   chr22   42090733    0   1   chr22   42090775    1   42  42
1   chr22   42090733    0   1   chr22   42090789    1   42  42
1   chr22   42090757    0   1   chr22   42090787    1   42  24
0   chr22   42090778    0   0   chr22   42090790    1   42  42
0   chr22   42090800    0   0   chr22   42090802    1   42  42
0   chr22   42090803    0   0   chr22   42090806    1   42  42

But other tools, like cut which has already been mentioned are simpler here. If your file is tab-separated, you can just do:

$ cut -f2- file 
1   chr22   42090593    0   1   chr22   42090609    1   42  42
1   chr22   42090593    0   1   chr22   42090654    1   42  42
1   chr22   42090595    0   1   chr22   42090633    1   42  42
0   chr22   42090612    0   1   chr22   42090656    1   42  42
0   chr22   42090614    0   0   chr22   42090617    1   40  42
0   chr22   42090647    0   1   chr22   42090749    1   42  42
1   chr22   42090684    0   1   chr22   42090692    1   42  42
1   chr22   42090733    0   1   chr22   42090743    1   42  42
1   chr22   42090733    0   1   chr22   42090775    1   42  42
1   chr22   42090733    0   1   chr22   42090789    1   42  42
1   chr22   42090757    0   1   chr22   42090787    1   42  24
0   chr22   42090778    0   0   chr22   42090790    1   42  42
0   chr22   42090800    0   0   chr22   42090802    1   42  42
0   chr22   42090803    0   0   chr22   42090806    1   42  42

Some other alternatives:

$ grep -oP '^\s*\S+\s*\K.*' file 
1   chr22   42090593    0   1   chr22   42090609    1   42  42
1   chr22   42090593    0   1   chr22   42090654    1   42  42
1   chr22   42090595    0   1   chr22   42090633    1   42  42
0   chr22   42090612    0   1   chr22   42090656    1   42  42
0   chr22   42090614    0   0   chr22   42090617    1   40  42
0   chr22   42090647    0   1   chr22   42090749    1   42  42
1   chr22   42090684    0   1   chr22   42090692    1   42  42
1   chr22   42090733    0   1   chr22   42090743    1   42  42
1   chr22   42090733    0   1   chr22   42090775    1   42  42
1   chr22   42090733    0   1   chr22   42090789    1   42  42
1   chr22   42090757    0   1   chr22   42090787    1   42  24
0   chr22   42090778    0   0   chr22   42090790    1   42  42
0   chr22   42090800    0   0   chr22   42090802    1   42  42
0   chr22   42090803    0   0   chr22   42090806    1   42  42

or

$ perl -pe 's/^\s*\S+\s*//' file 
1   chr22   42090593    0   1   chr22   42090609    1   42  42
1   chr22   42090593    0   1   chr22   42090654    1   42  42
1   chr22   42090595    0   1   chr22   42090633    1   42  42
0   chr22   42090612    0   1   chr22   42090656    1   42  42
0   chr22   42090614    0   0   chr22   42090617    1   40  42
0   chr22   42090647    0   1   chr22   42090749    1   42  42
1   chr22   42090684    0   1   chr22   42090692    1   42  42
1   chr22   42090733    0   1   chr22   42090743    1   42  42
1   chr22   42090733    0   1   chr22   42090775    1   42  42
1   chr22   42090733    0   1   chr22   42090789    1   42  42
1   chr22   42090757    0   1   chr22   42090787    1   42  24
0   chr22   42090778    0   0   chr22   42090790    1   42  42
0   chr22   42090800    0   0   chr22   42090802    1   42  42
0   chr22   42090803    0   0   chr22   42090806    1   42  42

or

$ perl -F'\t' -lane 'print join "\t",@F[1..$#F]' file 
1   chr22   42090593    0   1   chr22   42090609    1   42  42
1   chr22   42090593    0   1   chr22   42090654    1   42  42
1   chr22   42090595    0   1   chr22   42090633    1   42  42
0   chr22   42090612    0   1   chr22   42090656    1   42  42
0   chr22   42090614    0   0   chr22   42090617    1   40  42
0   chr22   42090647    0   1   chr22   42090749    1   42  42
1   chr22   42090684    0   1   chr22   42090692    1   42  42
1   chr22   42090733    0   1   chr22   42090743    1   42  42
1   chr22   42090733    0   1   chr22   42090775    1   42  42
1   chr22   42090733    0   1   chr22   42090789    1   42  42
1   chr22   42090757    0   1   chr22   42090787    1   42  24
0   chr22   42090778    0   0   chr22   42090790    1   42  42
0   chr22   42090800    0   0   chr22   42090802    1   42  42
0   chr22   42090803    0   0   chr22   42090806    1   42  42

Answer 2

This could work if you have spaces as delimiters:

sed 's/^ *//' text.file | cut -f1 -d" " --complement

Answer 3

Assuming there are in fact no leading spaces in your file, the following sed-based approach should work:

sed -r 's/^[[:digit:]]+[[:blank:]]+//' input_file

or, more general (as suggested by @terdon)

sed -r 's/^[^[:blank:]]+[[:blank:]]+//' input_file

The first example will replace one or more digits, followed by one or more blank characters, from the beginning of the line, by "nothing", thereby removing this part of the line without change to the rest.

The second example removes any "non-blank" characters, followed by one or more blank characters, and is therefore more generally applicable if the first column may also contain text (e.g. in a header line).

If there are leading spaces, the following modification:

sed -r 's/^[[:blank:]]*[^[:blank:]]+[[:blank:]]+//' input_file

should do the trick since it allows the pattern to be removed to contain zero ore more spaces at the beginning.

Note that [[:blank:]] will match spaces and tabs, so it should also work with tab-separated file formats.

As a general note, changes to the formatting when using awk often come from failure to set the output field separator OFS to the appropriate value. If you modify any individual field in an awk rule, this will cause $0 to be re-generated using the current value of OFS which is SPACE by default, thereby overriding the formatting which the line originally had.

So, if your input file is TAB-separated, modifying any field will lead to the output being space-separated instead, which could explain the change in format you describe. You could overcome this by setting the appropriate internal variable via

awk -v OFS='\t' '{your code here}' input_file

to prevent that, but as @terdon has mentioned, simply setting a field to the empty string does not remove it, and you would end up with an unnecessary additional field separator.

Answer 4

You can do this with gnu sed

$ sed -Ee 's/\S+/\n&/2;s/.*\n//' file 

We mark the beginning of the second field and then delete everything upto it.

Same approach as in above but using awk

awk '
  BEGIN {
    s = "[[:space:]]"; S = "[^[:space:]]" 
    F = S"+"s"*" 
  }
  sub(s"*" F, "")+1
' file

Same approach as above but with match function

awk '
   BEGIN {
     s = "[[:space:]]"; S = "[^[:space:]]"
     F = S"+"s"*" 
   }
   match($0, F) {
     $0 = substr($0, RSTART+RLENGTH)
   }1
' file