0

I have document A here:

@rand1
ABCDEFBHIJKLM
+
<</////
@rand2
NOPQRSTUVW
+
<<//<<<
@anotherrand
XYZABCDE
+
<<//<<<

I would like to have an output that contains all the lines of document A but where the lines 2,6,10... (pattern is 2+n*4)have been trimmed to only have the first 3 characters for example. This output would look like this:

@rand1
ABC
+
<</////
@rand2
NOP
+
<<//<<<
@anotherrand
XYZ
+
<<//<<<

I am doing this on a very big file (>10 million lines) and can't seem to find a way to make this go very quickly. the below code accomplishes what I want, but takes way too long:

r=0 #line number of documentA being read
l=2 #line that needs to be trimmed

while read line; do
  r=$(echo $r +1 | bc)
  echo $r
  if [ $r == $l ]
  then
    echo $line | cut -c -3 >> outputfile
    l=$(echo $l + 4 | bc)
  else
    echo $line >> outputfile
  fi
done < document A
2

With GNU sed (available as gsed on OSX) using the "n skip m" address operator:

sed -E '2~4s/(.{3}).*/\1/' inputfile > outputfile
5
  • this seems to work if I use gsed but not sed on my Mac. Also I would ideally have the 3 be a bash variable here which would usually be accomplished by simply putting the command in double quotations, but doing so results in this error: gsed: -e expression #1, char 16: Invalid content of \{\}. Any idea what is going wrong?
    – DNA_bash
    Aug 28 '19 at 18:20
  • @DNA_bash are you sure that your variable is wholly numeric (no leading/trailing whitespace for example)? which shell are you using? Aug 28 '19 at 18:29
  • ah stupid mistake from me, you are correct there was as space at the end I didn't see! Thank you!
    – DNA_bash
    Aug 28 '19 at 18:41
  • @DNA_bash you're welcome - I hope it is significantly faster than your original method? Aug 28 '19 at 18:51
  • Yes! Takes less than a minute and my way I estimated would take more than 24h!
    – DNA_bash
    Aug 28 '19 at 18:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.