I am using for testing purposes 2 lines with and without the optional suffix (last 2 enumerated elements ,D2,E2).

echo -e "A1,B2,C2\nA2,B2,C2,D2,E2" | sed -E 's/^(.*),(.*),(.*)((,.*)(,.*)){0,1}$/[\1],\2,\3\5\6/'


where I would like to obtain


I simply want to make the last 2 components optional and append them at the end in case they are found.

I tried with repetitions {x,y}, with question mark (?), but I don't manage to make really optional the 2 last elements on the sed find pattern. Looks like the sed greediness is extremely greedy because I am wrapping the pattern with '^' and '$' and instead of attempting first the unoptional pattern, it just takes the optional one with very greedy first component..

NOTE: The regex is a simplification of my real need so overcoming by different approaches might be futile. I really want to do it with sed and this particular approach, since this is just to improve my knowledgement on the tricks of the tool

EDIT: Thanks for the answers, this was my goal

sed -E 's/^([^,]*),([^,]*),([^,]*)(,[^,]*)?(,[^,]*)?$/[\1],\2,\3\4\5/'
  • sed 's/^\([^,]*\)/[\1]/' gives the output you show, but I'm not sure I completely understand you. – schrodigerscatcuriosity May 4 '20 at 18:29
  • I need to respect the capability of referring to each match in the output like in \1 - \2 - \3, so I can embellish them invidivdually. I just wrote a simplification of my need – Whimusical May 4 '20 at 19:26
  • You mean you want to capture each repetition and treat each of them individually in the replacement depending on how many repetitions you get? You will have to define some sort of rule. – bu5hman May 4 '20 at 19:37
  • Please don't expand in comments... others cant follow easily, so clarify your answer in the post. Do you mean you are trying to specify something like {0,1} means wrap the first argument, {0,2} means wrap the first 2, maybe {1,2} means wrap the second only? – bu5hman May 4 '20 at 19:47

I'm not sure about the purpose of your sed construct, but I can tell you about your mistake in it.
In your regexp you try to match the first three letter-number combinations with (.*),(.*),(.*). Since regexp in sed are 'greedy' the first (.*) will already match the three combinations as letters, numbers and commas are matched with .. To match single combinations you better match for none commas (e.g ([^,]*), witch matches any amount of none comma characters. Your command would then look like this:

echo -e "A1,B2,C2\nA2,B2,C2,D2,E2" | sed -E 's/^([^,]*),([^,]*),([^,]*)((,[^,]*)(,[^,]*)){0,1}$/[\1],\2,\3\5\6/'

If it is always a three or five combination input, you could also 'shorten' the regex to

echo -e "A1,B2,C2\nA2,B2,C2,D2,E2" | sed -E 's/^([^,]*)(((,[^,]*){2}){1,2})$/[\1]\2/'

but that actually depends on your use case.
The ^([^,]*) matches the first letter-number combination,
the ((,[^,]*){2})matches the next two or four combinations with a comma in front of it (,B2,C2 or ,B2,C2,D2,E2 in your input examples).


Still not sure what OP is trying to achieve, but to make a non-greedy match then you can use a negated character class, in this case [^,].

echo -e "A1,B2,C2\nA2,B2,C2,D2,E2" | 
    sed -E 's/([^,]+)((,[^,]+){2})((,[^,]+){0,2})/[\1]\2\4/'


echo -e "A1,B2,C2\nA2,B2,C2,D2,E2" | 
    sed -E 's/([^,]+)((,[^,]+){2})((,[^,]+){0,2})/[\1]\2\5/'


Though catching each match separately to embellish is a different matter...

If OP would care to give a fuller example....perhaps we can all go round again! The exercise is good for us ;)

  • Checkout that OP changed ± to commas – schrodigerscatcuriosity May 4 '20 at 18:54
  • Yes, sorry I tried to simplify. Thanks for the answer @bu5hman, the problem is that’s I need to be able to reorder/embellish the elements in the output and thus refer to \1\2\3\4... I just simplified my example too much – Whimusical May 4 '20 at 19:22
  • Maybe the best way to summarize it is that even though the optional part fits the long result, it is still using the first capture in a greedy way, so both lines use the repetition as if it were {0,0} – Whimusical May 4 '20 at 19:37
  • 1
    Shorter: s/[^,]*/[&]/ There's never any need to use /g on anchored expressions, a capture group around the whole expression is not needed as & can be used in the replacement, and expressions matches left to right so anchoring isn't necessary here. – Kusalananda May 4 '20 at 19:56
$ echo -e "A1,B2,C2\nA2,B2,C2,D2,E2" | sed -e 's/[^,]*/[&]/' -e 's/[^,]*/(&)/2' -e 's/[^,]*/{&}/3'

Rather than trying to do it all in one single substitution, you can use the fact that you can pick which one of the matches to work with by using a numerical flag at the end (here /2 and /3 for the 2nd and 3rd match).

The sed command used above, for clarity:

sed -e 's/[^,]*/[&]/' \
    -e 's/[^,]*/(&)/2' \
    -e 's/[^,]*/{&}/3'
  • This approach is not useful to me in my particular case because I need to rearrange the output based on the matches, but it is extremely clever if I hadn't had this particular need – Whimusical May 4 '20 at 20:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.