2

I have the following command with which I can change the field separator for an entire record

$ echo "DOWN_COMP_002_wget_001_6_2020-04-10-13_40_27_395533885" | awk 'BEGIN{FS="_|-";OFS="::"} {$i=$i } 1'
DOWN::COMP::002::wget::001::6::2020::04::10::13::40::27::395533885

But i only want the last seven columns i.e 2020::04::10::13::40::27::395533885

I can try to do that with

$ echo "DOWN_COMP_002_wget_001_6_2020-04-10-13_40_27_395533885" | awk 'BEGIN{FS="_"} {for(i=7;i<=NF;i++) printf $i"::"}'
6::2020-04-10-13::40::27::395533885::

But this will add an :: at the end which I don't want.

So how can I do this using OFS for the last N columns?

CONCLUSION FROM ANSWERS:

I was looking if there is any possible way with OFS

The solutions i saw below are not using the OFS. So its not possible using OFS

So the alternate solution is either to use sed or if inside awk which i am already aware of

So OFS is useful only when we manually do it like below

 print $1,$3,$5

Then the question is: So is there a away to generate print print $1,$3,$5.... programitcally so that OFS is automatically used.

3
  • 4
    You two outputs shows that you either want the - in the date replaced by ::, or you won't. Which is it? Also, 2020::04::10::13::40::27::395533885 looks like seven columns, not six, and 6::2020-04-10-13::40::27::395533885 is five.
    – Kusalananda
    Apr 10, 2020 at 14:41
  • 2
    Please make a little effort to tidy up your question. Do you really want the "last six columns" as in the text, or the last 7, as in the first example? Or the last NF-5 = 8 (13-5=8), as in the second example? Or was it that the last 5 (10-5), since you have changed the field definition)?
    – user313992
    Apr 10, 2020 at 15:03
  • changed to seven. its not six
    – Santhosh
    Apr 10, 2020 at 16:02

6 Answers 6

6

In this particular case, it's probably simpler to use cut instead of awk and then sed to change the delimiter:

$ echo "DOWN_COMP_002_wget_001_6_2020-04-10-13_40_27_395533885" | 
    cut -d_ -f6- | sed 's/_/::/g'
6::2020-04-10-13::40::27::395533885

I prefer this since getting the last N fields in awk requires a cumbersome for loop, while cut supports the N- format for "last N".

If you absolutely must use awk, you could try:

$ echo "DOWN_COMP_002_wget_001_6_2020-04-10-13_40_27_395533885" | 
    awk -F"_" '{for(i=6;i<=NF-1;i++){printf "%s::",$i} print $NF}'
6::2020-04-10-13::40::27::395533885

Or, if you want to use - as a delimiter as well, try:

$ echo "DOWN_COMP_002_wget_001_6_2020-04-10-13_40_27_395533885" | 
    awk -F"[-_]" '{for(i=6;i<=NF-1;i++){printf "%s::",$i} print $NF}'
6::2020::04::10::13::40::27::395533885
3
  • Note that in the first script OP uses both _ and - as a delimiter, whereas in the second one only _ is used, not sure what is the desired output.
    – user000001
    Apr 10, 2020 at 14:37
  • 1
    @user000001 hmm, true. I can only assume the desired output is what they want though.
    – terdon
    Apr 10, 2020 at 14:38
  • @unxnut I rolled back your edit since it isn't clear what the OP wants (their desired output doesn't use - as a delimiter and anyway, your edit changed the input but didn't also edit the output to reflect your change. Please make sure to always update the output of a command if you edit it in an answer.
    – terdon
    Apr 10, 2020 at 14:47
3

With (GNU) sed:

echo "DOWN_COMP_002_wget_001_6_2020-04-10-13_40_27_395533885" | sed 's/_/::/6g'
DOWN_COMP_002_wget_001_6::2020-04-10-13::40::27::395533885

Please note info sed on this construct:

The 's' command can be followed by zero or more of the following FLAGS:

'g'

Apply the replacement to _all_ matches to the REGEXP, not just the first.

'NUMBER'

Only replace the NUMBERth match of the REGEXP. 

interaction in 's' command Note: the POSIX standard does not specify what should happen
when you mix the 'g' and NUMBER modifiers, and currently there is no widely agreed upon
meaning across 'sed' implementations.  For GNU 'sed', the interaction is defined to be:
ignore matches before the NUMBERth, and then match and replace all matches from the NUMBERth on.
0
3

I'm interpreting your question "How to do the output with OFS" as you want to perform a single print, with no arguments, and let awk insert the new field separator automatically between the fields.

To do this, you will have to rebuild $0, i.e. the current record. You can do this easily by first setting $0 to an empty string, and then assigning to the fields.

Since you empty the current record, you will have to store the data that will be put back into it in an array.

The following is an example of how this could be done:

echo 'DOWN_COMP_002_wget_001_6_2020-04-10-13_40_27_395533885' |
awk -F '[-_]' '
        BEGIN { OFS = "::" }
        {
                nf = 0

                # save the last seven fields in "field"
                for (i = NF - 6; i <= NF; ++i)
                        field[++nf] = $i

                # clear record
                $0 = ""

                # reassign data into record
                for (i = 1; i <= nf; ++i)
                        $i = field[i]

                # output record
                print
        }'

This code outputs

2020::04::10::13::40::27::395533885

And it does this by storing the data that should be part of the output record in the array field, clearing the current record using $0 = "", and then assigning the saved data to the record before printing.

2
  • For multiple lines of input, if the nf counter isn't reset, each line will be appended to the previous
    – guest
    Apr 11, 2020 at 6:44
  • @guest Yes, better safe than sorry. Thanks.
    – Kusalananda
    Apr 11, 2020 at 6:46
2

Use a ternary operator to select the correct delimiter:

echo "DOWN_COMP_002_wget_001_6_2020-04-10-13_40_27_395533885" | 
    awk 'BEGIN{FS="_|-"} {for(i=6;i<=NF;i++) printf $i (i < NF ? "::" : "\n")}'

Output:

6::2020::04::10::13::40::27::395533885
0
2
$ echo "DOWN_COMP_002_wget_001_6_2020-04-10-13_40_27_395533885" |
    awk -v n=7 -F'[_-]' -v OFS='::' 'NF>n{sub("([^_-]*[_-]){"NF-n"}","")} {$1=$1} 1'
2020::04::10::13::40::27::395533885
10
  • 2
    Readers take heed: this answer is using interval expressions which are not yet supported in most awk implementations, except for GNU awk and busybox awk. As of April 2020, this does not work with the default awk from Debian, OpenBSD or FreeBSD.
    – user313992
    Apr 11, 2020 at 2:37
  • Interval expressions are part of POSIX EREs and so are supported by most awks, including of course all POSIX awks and including /usr/xpg4/bin/awk on Solaris and the default OSX/BSD awk on MacOS. They've been part of gawk since version 3.0 was released about 20 years ago. If your awk doesn't support RE intervals then it's time to get a new awk.
    – Ed Morton
    Apr 11, 2020 at 11:14
  • You're missing my point. Using that feature with awk implementations which do not support it will result in SILENT data corruption, NOT in a syntax error. That's not something I would foist upon unsuspecting users, just to make a point that they should update to my preferred version of awk.
    – user313992
    Apr 12, 2020 at 0:16
  • And FYI, /usr/xpg4/bin/awk on Solaris is NOT standard-compliant (or, more precisely, it's too compliant with an obsolete, faulty version of the standard): with /usr/xpg4/bin/awk, 0=="000" will evaluate to 1 -> TRUE, something that the current standard explicitly forbids. So IMHO, using /usr/xpg4/bin/awk instead of /usr/bin/nawk on Solaris is not a good idea.
    – user313992
    Apr 12, 2020 at 0:16
  • 1
    People reading this (not necessarily the author of the answer, who apparently doesn't care, anyways), may have a look at an answer where I've tried to explain why bounds/interval expression in awk are a HORRIBLE idea in the first place.
    – user313992
    Apr 12, 2020 at 21:56
2

another awk alternative:

awk -F'[_-]' -v col=7 '{ 
    while (col-->0) { printf "%s%s", $(NF-col), (col? "::" : ORS) };
}' infile

adjust col=N with "number of N last columns" you want to print.


Answer to the revised question to:

So is there a away to generate print print $1,$3,$5.... programitcally so that OFS is automatically used

personally and also technically there is no reason to limit yourself to using OFS, but well if you want, here is how to do it:

awk -F'[_-]' -v c=col=7 '{
    while (col-->0) { printf "%s%s", $(NF-col), (col? XXX : ORS) };
}' XXX='::' infile

but you know what is the behind OFS? replace all XXXs above with OFS; now remove XXX='::' and change XXX to just "::"; well so why stuck at using OFS then?

2
  • Using each field as the format argument to printf may fail if field contains %. E.g. echo %c | awk '{printf $1}'
    – guest
    Apr 11, 2020 at 3:54
  • @guest thank for pointing that. fixed. Apr 11, 2020 at 5:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .