0

I was trying to rename texts in fileA using sed. The second last column in fileA is where the full description of product names lies. I would like to substitute the product names with its ID. However, some of the texts in description was having similar content (Example as in fileA). There were two occurrences of "Orange juice" in line2 and line4.

I was generating renamefile to substitute the product names with sed. However, sed replaced every "Orange juice" it found with "3071", regardless of the occurrence of "with pulp" after the match on "Orange juice".

fileA:

AB12345    100    0    Apple juice 20/05   AB
CD67890    150    0    Orange juice with pulp 22/05   CS
EF25879    100    0    Watermelon juice 19/05   CG
GH96314    98    0    Orange juice 20/05   PU
IJ74123    95    0    Strawberry juice with lemon 17/05   ST

renamefile:

s/\<Apple juice\>/3071/g
s/\<Orange juice with pulp\>/3072/g
s/\<Orange juice\>/3073/g
s/\<Watermelon juice\>/3074/g
s/\<Apple juice with lemon\>/3075/g
s/\<Strawberry juice with lemon\>/3076/g

current output:

AB12345    100    0    3071 20/05   AB
CD67890    150    0    **3073** 22/05   CS
EF25879    100    0    3074 19/05   CG
GH96314    98    0    3073 20/05   PU
IJ74123    95    0    3076 17/05   ST

desired output:

AB12345    100    0    3071 20/05   AB
CD67890    150    0    3072 22/05   CS
EF25879    100    0    3074 19/05   CG
GH96314    98    0    3073 20/05   PU
IJ74123    95    0    3076 17/05   ST

I was using "<>", as found somewhere that it helped in substituting only if there is an exact match. However, it doesn't seemed to work in this case. (The error was bolded in the current output.)

Is there any better or more efficient ways to ensure the string substitution consider a few more words after the first two strings matched and replace those names with their ID?

Please let me know if I wasn't clear enough. Thanks!

4
  • the problem is you are using global substitution. just omit g from your sed commands. for example 's/\<Apple juice\>/3071/'
    – binarysta
    Commented May 22, 2020 at 18:19
  • 2
    It should give the expected output, since you first specified the longer "Orange juice with pulp" and only after it "Orange juice". I gave it a try (sed -f renamefile fileA) and indeed it gives the expected output. However, "Apple juice with lemon" should likewise come before "Apple juice".
    – Quasímodo
    Commented May 22, 2020 at 18:20
  • Hi @binarysta I re-ran the command with no g, but still getting the same output as before.
    – web
    Commented May 23, 2020 at 10:41
  • Thanks @Quasímodo, it worked this way, but my renamefile here were having more than thousand lines. It gonna take some time to rearrange their position.
    – web
    Commented May 23, 2020 at 10:43

4 Answers 4

1

Need to re-order your renamefile based on length, to first replace longer names

awk '{ print length, $0 }' renamefile| sort -nr | cut -d" " -f2- > renamefile2

output

s/\<Strawberry juice with lemon\>/3076/g
s/\<Orange juice with pulp\>/3072/g
s/\<Apple juice with lemon\>/3075/g
s/\<Watermelon juice\>/3074/g
s/\<Orange juice\>/3073/g
s/\<Apple juice\>/3071/g

Then you can apply without any issue

sed -f renamefile2 fileA

Description:

awk loops over lines

  • length is a built-in function of awk. when calling without argument, it will print the size of the current line (more info at awk length)
  • $0 current line

Following command will print the lenght of each line next to the line itself

awk '{ print length, $0 }' renamefile

24 s/\<Apple juice\>/3071/g
35 s/\<Orange juice with pulp\>/3072/g
25 s/\<Orange juice\>/3073/g

sort will sort the input text

  • -n will sort numerically
  • -r reverse the result, to make it descending.

cut will select a part of text (since we do not want the lengths in the final script and need to only select the sed part of lines)

  • -d" " specifies the delimeter which is space here.
  • -f2- from field 2 till end of line
4
  • @web yes you are right, I posted the correct answer.
    – binarysta
    Commented May 23, 2020 at 11:37
  • Thanks, the code worked just perfect on my file! BTW, do you mind to explain a bit on the "cut" command? Not quite familiar with it. Does the second hyphen meant something?
    – web
    Commented May 23, 2020 at 14:11
  • @web yes, details added. Please check.
    – binarysta
    Commented May 23, 2020 at 14:27
  • It is much clearer now. Thanks! :)
    – web
    Commented May 23, 2020 at 14:43
1

If there are always two digits followed by a / followed by another two digits after the product name, you could include those in your regex and replace them with themselves using a backreference.

You could furthermore match the four preceeding space characters and also replace them with themselves.

renamefile:

s/( {4})Apple juice( [[:digit:]]{2}\/[[:digit:]]{2})/\13071\2/
s/( {4})Orange juice with pulp( [[:digit:]]{2}\/[[:digit:]]{2})/\13072\2/
s/( {4})Orange juice( [[:digit:]]{2}\/[[:digit:]]{2})/\13073\2/
s/( {4})Watermelon juice( [[:digit:]]{2}\/[[:digit:]]{2})/\13074\2/
s/( {4})Apple juice with lemon( [[:digit:]]{2}\/[[:digit:]]{2})/\13075\2/
s/( {4})Strawberry juice with lemon( [[:digit:]]{2}\/[[:digit:]]{2})/\13076\2/

Output:

$ sed -Ef renamefile fileA
AB12345    100    0    3071 20/05   AB
CD67890    150    0    3072 22/05   CS
EF25879    100    0    3074 19/05   CG
GH96314    98    0    3073 20/05   PU
IJ74123    95    0    3076 17/05   ST
0

It's simpler with awk:

$ cat tst.awk
BEGIN {
    id = 3071
    map["Apple juice"]                  = id++
    map["Orange juice with pulp"]       = id++
    map["Orange juice"]                 = id++
    map["Watermelon juice"]             = id++
    map["Apple juice with lemon"]       = id++
    map["Strawberry juice with lemon"]  = id++
}
match($0,/^((\S+\s+){3})(.*\S)((\s+\S+){2})/,a) {
    $0 = a[1] map[a[3]] a[4]
    print
}

$ awk -f tst.awk file
AB12345    100    0    3071 20/05   AB
CD67890    150    0    3072 22/05   CS
EF25879    100    0    3074 19/05   CG
GH96314    98    0    3073 20/05   PU
IJ74123    95    0    3076 17/05   ST

The above uses GNU awk since you were using GNU sed for \< and \> word boundaries.

3
  • Hi, thanks for the code! Was trying out the code, but found an error: line 10: syntax error at or near , .
    – web
    Commented May 23, 2020 at 11:04
  • Then you aren't using GNU awk. Wherever you got GNU sed from, get GNU awk from there too.
    – Ed Morton
    Commented May 23, 2020 at 13:53
  • I see. Thanks anyway!
    – web
    Commented May 23, 2020 at 14:16
0

Employing GNU sed, we first modify the rename file dynami ally (meaning without you having to do any hand edits on it) and then use that as a sed code to perform the edits in the fileA

The change we do in the renamefile is to look for a newline as the RHS boundary rather than \>. But before this we insert a newline in the fileA's pattern space.

$ sed -re '
     1i\
s/(\\s+\\S+){2}\\s*$/\\n&/
     s/\\>/\\n/
' renamefile | sed -rf - fileA

Output:

AB12345    100    0    3071 20/05   AB
CD67890    150    0    3072 22/05   CS
EF25879    100    0    3074 19/05   CG
GH96314    98    0    3073 20/05   PU
IJ74123    95    0    3076 17/05   ST

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .