1

I have a sample input file as below:

Apple   Orange      Gold    Silver Spoon Apple  Orange              Gold    
Apple          
Apple   Orange 
Apple   Orange      
Mango               Gold    Silver Spoon Apple  Orange  

I need to check if the 15th to 20th position is Space and then replace it with a string Silver. Also I need to completely replace the contents from 50th to 70th position with xxxxxxxxxxxxxxxxxxxxx

  • 2
    How to handle lines that are of size less than 15/20 or resp. 50/70 ? – Janis Jun 11 '15 at 4:59
  • do nothing to those lines – Georgi Jun 11 '15 at 12:07
0
sed 's/^\(.\{14\}\)      /\1Silver/
     s/^\(.\{49\}\).\{20\}/\1xxxxxxxxxxxxxxxxxxxxx/
'    <infile >outfile

Does it, I think.

  • Yes, however it should be 49 and 21 on the second line, since the xxxxxxxxxxxxxxxxxxxxx should start at the 50th position and since it looks like you want a number of characters equal to the lenght of the string to be present after in order to perform the replacement – kos Jun 11 '15 at 6:34
  • When I try to do it for replacement above 255, it is not working. Is there a work around to increase the sed maximum limit? – Georgi Jun 11 '15 at 11:59
  • @Georgi - I take you mean s/.\{255\}... is not working for some reason? What getconf LINE_MAX tell you? - it should be >=512. – mikeserv Jun 11 '15 at 16:47
  • $ getconf LINE_MAX 2048 – Georgi Jun 12 '15 at 4:02
  • @Georgi - but when you do: printf %0256s | sed 's/.\{256\}//' | wc -c the report is 256? Because that doesn't make much sense. On the other hand, if you're going to be scaling it to that degree, it's probably worth considering a different tool - or at least some partnership of tools. Can I ask what system you're using? The above is POSIX - and any sed should handle up to LINE_MAX at least (though the spec does indicate that it should handle more in pattern space), but some older seds are not fully conformant. Also, what is the script you use exactly? – mikeserv Jun 12 '15 at 4:35
1

Depending on how smaller lines shall be handled (see my comment above) here is a sed solution:

sed -e '/^.\{14\} \{6\}/s/^\(.\{14\}\) \{6\}/\1Silver/' \
    -e '/^.\{49\}./s/^\(.\{49\}\)\(.\{1,21\}\)\?\(.*\)\?/\1xxxxxxxxxxxxxxxxxxxxx\3/'

and a much better legible (and less error-prone) GNU awk version:

awk '
  BEGIN { FIELDWIDTHS = "14 6 29 21 999" ; OFS = "" }
  $2 == "      " { $2 = "Silver" }
  $4 != ""       { $4 = "xxxxxxxxxxxxxxxxxxxxx" }
  { print }
'

Explanation:

1.) sed: The sed command is built by two independent substitutions, the first handles the "Silver" case the second the "xxx..." one. The sed substitutions are of the form:

/pattern/s/pattern2/replacement/

The respective substitution is performed if the first pattern matches, and for those lines the second pattern is replaced by the replacement expression. The details are the sed typical cryptic expressions:

. - an arbitrary character

\{14\} - a repetition of the preceding subexpression (here 14 times)

\(expr\) - a subexpression that may be referenced in the replacement string by \1, \2, etc., where the actual number is defined by the n'th parenthesis expression

\? - specifies that the preceding subexpression is an optional part

2.) awk: The awk program runs the BEGIN section once, and the subsequent sections for each line of the data file, and the respective right hand action {...} only if the respective left hand condition is true.

FIELDWIDTHS - specifies the width of the data fields in the input lines, so that each field can be addressed by $i (for some field number i )

OFS="" - an empty string; the output fields shall not get any additional separation

$2 == " " - if the second field (according to the FIELDWIDTHS specification) contains six blanks then replace it by the given string

$4 != "" - if the fourth field contains data replace it by the "xxx..." string

{ print } - an unconditional print of the current line, which was either modified by one or both of the preceding actions or shows the unmodified line otherwise

  • since I am new to this, can you explain what we are doing above? thanks for that – Georgi Jun 11 '15 at 12:01
  • @Georgi; I added an explanation. – Janis Jun 11 '15 at 13:02
  • What errors do you propose can be had? And how is .\{14\} illegible exactly? And by the way \? does not specify optional in any but a few seds. \{0,1\} would, though, POSIXly. In fact - the awk is far less legible - it doesn't even say what it does. I've downvoted this because it doesn't make sense. – mikeserv Jun 11 '15 at 16:01
  • @mikeserv; Are you serious? - sed is per se already extremely cryptic, and when you need to add specific meta-characters you need to escape all those characters; this is very unreadable by any sensible comparison. You just need to have an unbiased look at the sed code, it looks more like morse code than like a program. - And now compare that to the awk program. - And, WRT errors, you just need to observe in practice how folks write sed code, and correct it multiple times, rinse repeat, until it works. - sed is a low level language with all inherent drawbacks of such beasts. – Janis Jun 11 '15 at 16:17
  • @mikeserv; The downvote is of course your choice. All I can do is to shake my head, given your statement why you did. - Why wouldn't it "make sense" in your opinion? - Specifying the fieldwidths of the data for ease of processing in semantical units is extremely convenient in the given case. – Janis Jun 11 '15 at 16:20
0

Assuming that lines containing less than 21 characters (for the first replacement) and that lines containing less than 70 characters (for the second replacement) should be discarded, using Perl:

< inputfile perl -pe 's/^(.{14}) {6}/$1Silver/; s/^(.{49}).{21}/$1xxxxxxxxxxxxxxxxxxxxx/' > outputfile

Assuming that lines containing less than 21 characters (for the first replacement) and that lines containing less than 70 characters (for the second replacement) should be padded with spaces first, using awk+Perl:

< inputfile awk '{printf "%-70s\n", $0}' | perl -pe 's/^(.{14}) {6}/$1Silver/; s/^(.{49}).{21}/$1xxxxxxxxxxxxxxxxxxxxx/' > outputfile

Command #1 breakdown:

  • < inputfile: redirects the content of inputfile to perl's stdin
  • -p: forces Perl to print the lines
  • -e: forces Perl to read a line of program from the arguments
  • > outputfile: redirects the content of perl's stdout to outputfile

Command #2 breakdown:

  • < inputfile in awk: redirects the content of inputfile to awk's stdin
  • {printf "%-70s", $0}: pads each line with spaces until the number of characters on the line is 70
  • |: pipes awk's stdout to perl's stdin
  • -p: forces Perl to print the lines
  • -e: forces Perl to read a line of program from the arguments
  • > outputfile: redirects the content of perl's stdout to outputfile

Substitution #1 breakdown:

  • s: asserts to perform a substitution
  • /: starts the search pattern
  • ^: matches the start of the line
  • (: starts the capturing group
  • .{14}: matches 14 occurences of any character
  • ): stops the capturing group
  • {6}: matches 6 occurences of a character
  • /: stops the search pattern / starts the replace pattern
  • $1: replaces with the captured group
  • Silver: adds a Silver string
  • /: stops the replace pattern

Substitution #2 breakdown:

  • s: asserts to perform a substitution
  • /: starts the search pattern
  • ^: matches the start of the line
  • (: starts the capturing group
  • .{49}: matches 49 occurences of any character
  • ): stops the capturing group
  • .{21}: matches 21 occurences of any character
  • /: stops the search pattern / starts the replace pattern
  • $1: replaces with the captured group
  • xxxxxxxxxxxxxxxxxxxxx: adds a xxxxxxxxxxxxxxxxxxxxx string
  • /: stops the replace pattern

Sample output:

:~/tmp$ cat inputfile
Apple   Orange      Gold    Silver Spoon Apple  Orange              Gold    
Apple          
Apple   Orange 
Apple   Orange      
Mango               Gold    Silver Spoon Apple  Orange  
~/tmp$ < inputfile perl -pe 's/^(.{14}) {6}/$1Silver/; s/^(.{49}).{21}/$1xxxxxxxxxxxxxxxxxxxxx/'
Apple   OrangeSilverGold    Silver Spoon Apple  Oxxxxxxxxxxxxxxxxxxxxxld    
Apple          
Apple   Orange 
Apple   OrangeSilver
Mango         SilverGold    Silver Spoon Apple  Orange  
~/tmp$ < inputfile awk '{printf "%-70s\n", $0}' | perl -pe 's/^(.{14}) {6}/$1Silver/; s/^(.{49}).{21}/$1xxxxxxxxxxxxxxxxxxxxx/'
Apple   OrangeSilverGold    Silver Spoon Apple  Oxxxxxxxxxxxxxxxxxxxxxld    
Apple         Silver                             xxxxxxxxxxxxxxxxxxxxx
Apple   OrangeSilver                             xxxxxxxxxxxxxxxxxxxxx
Apple   OrangeSilver                             xxxxxxxxxxxxxxxxxxxxx
Mango         SilverGold    Silver Spoon Apple  Oxxxxxxxxxxxxxxxxxxxxx
~/tmp$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.