-2

Getting fixed length file from main frame in for one particular attribute, I am getting junk character, so time this junk char read as new line character, as result the whole data got spoiled.

Example of data in a File

0401000000030020170628000000710000366400201706280002750035*T000100N▒101892928550383900000009201 00000000000008402017062800000020  0000006435000000000000000840A1E098D09D9279BE4000561510A00003220000000000000000000000FF

0401000000030020170628000000710000365400201706280001041125
T000100N▒101909856755446700000018201 00000000000008402017062800000000  00000067540000000000000008402ED730917E9D1DC040000B0810A04003240000000000000000000000FF

First record is proper even though I am getting junk character *T000100N▒ but for second record T000100N▒ junk char is read as new line character.

  • Do I understand you correctly: In the second record, at the position where the first record has a *, you have a newline? Now you want a script to concatenate lines that are too short and replace the newline by what? – Philippos Jul 19 '17 at 6:25
  • Hi Romeo, Its not only "*" I am getting any junk character, and these junk character some time turned as new line character coming prior to "T000100N" this data and that come in between the fixed length data. Few More Examples are below ▒T000100N▒ @*T000100N▒ ▒@T000100N▒ – user241383 Jul 19 '17 at 6:43
  • My name is not Romeo, but what character do you want to have there instead? And are you sure you can't heal the cause instead of curing the symptom? – Philippos Jul 19 '17 at 6:46
  • Hi Romeo, Its not only "*" I am getting any junk character, and these junk character some time turned as new line character coming prior to "T000100N" this data and that come in between the fixed length data. Few More Examples are below ▒T000100N▒ @*T000100N▒ ▒@T000100N▒ – user241383 Jul 19 '17 at 6:50
  • Don't repeat your comment, answer the question if you want help – Philippos Jul 19 '17 at 6:51
0

In your example sed succeeds:

sed '/^.\{216\}$/!N;s/./?/59' file

The address /^.\{216\}$/! executes the next command on lines that don't have the required length of 216 chars. In this case we assume the line is split, so the N reads the rest of the line.

Then the s command replaces the 59th char of that record (either the * or the newline by a question mark (change that as you need it).

This works for your example, but I'm not sure whether some other random bytes or multi-byte chars will break the script.

So, if sed doesn't work in all cases, use the nice tool called bbe which is perfect for processing fixed-length records including binary data:

bbe -b ":219" -e 'r 58 ?' file

-b ":219" defines the blocksize to be 219 (inclusing newline and multibyte-char) and the r 58 ? replaces the byte at offset 58 (so it's the 59th char) with the question mark.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.