-3

working with a script I've got this issue, most of the times when I run the script this is the output file I've got:

device_id,ip_address,serial_number
 SEP0c1167231746, 148.000.000.32
 SEP0c1167223fa5, 148.000.000.30
 SEP0c1167224170, 148.000.000.30
 SEP0c1167231d2e, 148.000.000.194
 SEP0c1167233b9f, 148.000.000.31
 CUV, 148.000.000.254
 SEP0c1167231d32, 148.000.000.34
 SEP501cbffcfa9c, 148.000.000.24
 SEP00082fb67d5f, 148.000.000.21
 SEP00082fb67701, 148.000.000.22

And this is exactly what I expect, but occasionally the file looks like this:

device_id,ip_address,serial_number
 SEP0c1167231746
, 148.000.000.32
 SEP0c1167223fa5
, 148.000.000.30
 SEP0c1167224170
, 148.000.000.30
 SEP0c1167231d2e
, 148.000.000.194
 SEP0c1167233b9f
, 148.000.000.31
 CUV
, 148.000.000.254
 SEP0c1167231d32
, 148.000.000.34
 SEP501cbffcfa9c
, 148.000.000.24
 SEP00082fb67d5f
, 148.000.000.21
 SEP00082fb67701
, 148.000.000.22

I tried to figure out what's going on, but it does not seem to be something regular, now I only want to handle with this, using Ghex I've identified the character that causes the problem.

enter image description here

Now I would like to replace all "0D" with Null and keep all "0A"

just as a note, I have tried using "dos2unix" but it didn't work.

Could you help me?

UPDATE: Using: sed -n -e '/,/!{N;s/\n//;}; /,/p' input

with a file like this:

device_id,ip_address,serial_number
 SEP0c1167231746
, 148.000.000.32
,
 SEP0c1167223fa5
, 148.000.000.30
,
 SEP0c1167224170
, 148.000.000.30
,
 SEP0c1167231d2e
, 148.000.000.194
,
 SEP0c1167233b9f
, 148.000.000.31
,
 CUV
, 148.000.000.254
,
 SEP0c1167231d32
, 148.000.000.34
,
 SEP501cbffcfa9c
, 148.000.000.24
,
 SEP00082fb67d5f
, 148.000.000.21
,
 SEP00082fb67701
, 148.000.000.22

I've got this output:

, 148.000.000.32
, 148.000.000.30
, 148.000.000.30
, 148.000.000.194
, 148.000.000.31
, 148.000.000.254
, 148.000.000.34
, 148.000.000.24
, 148.000.000.21
, 148.000.000.22
|improve this question

closed as unclear what you're asking by Jeff Schaller♦, Anthony Geoghegan, Archemar, Anthon, thrig Apr 10 '17 at 13:46

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    you say "null" but I suspect you mean "with nothing" -- so that the lines are rejoined? – Jeff Schaller♦ Apr 7 '17 at 17:52
  • That's right. I would like to delete that Hex character "0D" – Cesar Alejandro Villegas Yepez Apr 7 '17 at 17:54
  • 1
    You need to show your script which is causing this behavior of printing \r sometimes and other times \n. As otherwise, all fixes will be just patches without knowing for sure what's causing it, – user218374 Apr 7 '17 at 18:10
  • 1
    you've changed your input format; that would require a different solution – Jeff Schaller♦ Apr 7 '17 at 20:05
2

There may be a better sed option, but here's one:

sed -n -e '/,/!{N;s/\n//;}; /,/p' input > output

It says, (by default, not printing lines): if there's a comma in the line, then read the next line in and replace the newline. Then, if there is (now, or already) a comma in the line, print the line. It reads from input and writes to output. With some seds, you can use sed's -i flag to edit the file in-place.

Sample input:

device_id,ip_address,serial_number
 SEP0c1167231746, 148.000.000.32
 SEP0c1167223fa5, 148.000.000.30
 SEP0c1167231746
, 148.000.000.32
 SEP0c1167223fa5
, 148.000.000.30

Sample output:

device_id,ip_address,serial_number
 SEP0c1167231746, 148.000.000.32
 SEP0c1167223fa5, 148.000.000.30
 SEP0c1167231746, 148.000.000.32
 SEP0c1167223fa5, 148.000.000.30
|improve this answer
  • Upvoted because this answers the question as originally asked. – Anthony Geoghegan Apr 10 '17 at 8:37
2

Looking at your hexdump, it would seem that this should fix your issue:

tr -d '\015' < input > log

As the octal \015 is the carriage return ^M character.

Why dos2unix didn't help is coz dos2unix looks at the sequence \r\n which is not present in your case.

|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.