-1

I came across a code in script which add ',' after every 3 digits from backwards. the code only considers numeric data.

Following is the code.

sed 's/\(^\|[^0-9.]\)\([0-9]\+\)\([0-9]\{3\}\)/\1\2,\3/g' number.txt

number.txt

1234
12345
123456

output

1,234
12,345
123,456
1234,567

Can anyone explain the code flow..

  • 1
    You appear to know what the code does; there's no branching, and only one command. Where does your misunderstanding come in? – Jeff Schaller Mar 26 at 10:36
3

sed (the stream editor) can operate in search and replace mode using regular expressions. There is a bit of sed-specific escaping going on, but for the regex itself, you can confer the explanation tool from regexr:

( Capturing group #1. Groups multiple tokens together and creates a capture group for extracting a substring or using a backreference.

^ Beginning. Matches the beginning of the string.

| Alternation. Acts like a boolean OR. Matches the expression before or after the |.

[^ Negated set. Match any character that is not in the set.

0-9 Range. Matches a character in the range "0" to "9". Case sensitive.

. Character. Matches a "." character.

]

)

( Capturing group #2. Groups multiple tokens together and creates a capture group for extracting a substring or using a backreference.

[ Character set. Match any character in the set.

0-9 Range. Matches a character in the range "0" to "9". Case sensitive.

]

+ Quantifier. Match 1 or more of the preceding token.

)

( Capturing group #3. Groups multiple tokens together and creates a capture group for extracting a substring or using a backreference.

[ Character set. Match any character in the set.

0-9 Range. Matches a character in the range "0" to "9". Case sensitive.

]

{3} Quantifier. Match 3 of the preceding token.

)

  • 1
    ... "0-9 Range. Matches a character in the range "0" to "9". Case sensitive"; By Case sensitive, you mean English digits? or? – αғsнιη Mar 26 at 15:39
  • No, case sensitive means big and small letters make a difference. Regular expressions are usually case sensitive (unless configured otherwise e.g. an i switch). In this particular case, the character set only contains digits. They have big and small variants so the case sensitivity does not actually matter. This is the output as given by regexr – feel free to try it. The explanation tool is quite interactive. – Hermann Mar 27 at 0:51
1

The pattern captures (1) the start of line or something that is not a digit nor a dot, followed by (2) any number of digits, at least one, followed by (3) exactly three digits. It then puts them back with a comma between (2) and (3), in effect adding one thousands separator. The first group is only required to avoid touching fractional parts after a decimal point, since we don't want 1.2345 to turn into 1.2,345.

Note that the pattern is written in basic regular expressions (BRE), requiring backslashes in front of each () and {} to make them special. Moreover, it requires GNU sed, where \+ and \| also have special meanings in BRE as an extension. The command would be better written as an extended regular expression (sed -E is supported by many sed implementations):

sed -E 's/(^|[^0-9.])([0-9]+)([0-9]{3})/\1\2,\3/g'

Also, the pattern only does one replacement, it does not add multiple thousand separators to the same number. The /g at the end will match multiple times on the same line, but it doesn't process already replaced data. 1234567 will become 1234,567, not 1,234,567. To fix that, we'll need to add a loop:

sed -E -e :a  -e 's/(^|[^0-9.])([0-9]+)([0-9]{3})/\1\2,\3/g' -e ta

Here, :a is just a label, and the final ta tests for a successful replacement, and jumps back to a if a replacement was done, in effect repeating the process as many times as it does something. Hence 1234567 will turn into 1,234,567.

1
  • sed 's/foo/bar/g' number.txt: This reads the file number.txt, and replaces the regex pattern foo with bar. This will occur for all matches on each line (/g).
  • \(^\|[^0-9.]\)\([0-9]\+\)\([0-9]\{3\}\): This is the pattern to replace. Each part in the escaped parentheses \(…\) is a "capturing group". The pattern inside is "captured" for later use.
    • \(^\|[^0-9.]\): Find the beginning of the line ^ or \| a character that is not a numeral or period [^0-9.]. This essentially finds the character preceding a number.
    • \([0-9]\+\): Find one or more numerals [0-9]\+.
    • \([0-9]\{3\}\): Find 3 numerals [0-9]\{3\}.
  • \1\2,\3: replace the above matches with the first two capturing groups followed by , then the last capturing group. In other words, insert , between the second and third patterns.

Because sed is "greedy", it will try to maximise the length of the match. Hence the final capturing group will be the last three numerals in the number.

N.B. many of the "special" characters are escaped with \, e.g. \(…\) and \{3\}. If your sed supported "extended regular expressions" with -E or -r, then you would not have to escape these. This would improve readability.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.