1

I have an RFID reader which sends the Unique ID of the RFID tag to the serial port of my linux (raspberry pi) box every time it gets a successful read. The problem is that it is sending it without any newline characters. Here's an example of the output:

root@scalepi:~/scale# ./grabserial -d "/dev/ttyUSB4" -b 9600 
$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#

I'd like to write each entry to a text file with a new line, like this:

$A0112OKD9990001200005643E#
$A0112OKD9990001200005643E#
$A0112OKD9990001200005643E#

As you can see each entry begins with $ and ends with #

I have tried just piping the outout to sed using this line to add a new line after the #, but I get zero output, because I think sed is looking for a new line before it processes the data:

./grabserial -d "/dev/ttyUSB4" -b 9600 | sed 's/#/#\r\n/g'

If I redirect the output to a file, and then cat the file, piped through that same sed command, I get what I expect, but I need to be able to write the data to a file in this format, so I can read it in realtime.

root@scalepi:~/scale# ./grabserial -d "/dev/ttyUSB4" -b 9600 > /tmp/test2.txt
^C
root@scalepi:~/scale# cat /tmp/test2.txt | sed 's/#/#\r\n/g'
$A0112OKD9990001200005643E#
$A0112OKD9990001200005643E#
$A0112OKD9990001200005643E#
root@scalepi:~/scale# 

If anyone has any suggestions for me, I'd appreciate it. Thanks!

EDIT - UPDATED

After trying the "tr" solution below, I was able to get real time output, but if I try to redirect via pipe, or via stdout/stderr redirection, I get buffered results of 50-60 lines at a time instead of getting a line by line result.

The following command will give me real time output as it comes in on the serial port:

./grabserial -d "/dev/ttyUSB4" -b 9600 | tr '#' '\n'

The following command will give me output in chunks of 50 or 60 lines at a time (not real time):

./grabserial -d "/dev/ttyUSB4" -b 9600 | tr '#' '\n' | sed -u 's/$/#\r/g'

The following command will give me output to a text file in chunks of 50 or 60 lines at a time (not real time):

./grabserial -d "/dev/ttyUSB4" -b 9600 | tr '#' '\n'  > /tmp/test1.txt

FWIW, I get the same results using cat to read the serial port, instead of "grabserial".

The following command gives me realtime results:

cat /dev/ttyUSB4 | tr '#' '\n' 

But when I try to redirect output via either pipe or stdout redirection, I get buffered chunks of data in the text file:

cat /dev/ttyUSB4 | tr '#' '\n' > /tmp/test4.txt

2 Answers 2

1

You can translate (or transliterate) the # characters into newlines with

 tr '#' '\n'

so

./grabserial -d "/dev/ttyUSB4" -b 9600 | tr '#' '\n'

should give you

$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
            ︙

in real time.  If you need the pound signs and/or the carriage returns, you can pipe the above into

        … sed 's/$/#\r/'

(using the fact that sed (and regular expressions in general) use $ to represent the end of a line.

Gotcha

You may have a problem that tr buffers its output when that output is going to a pipe or a file.  Perhaps the best way to handle this is to follow αғsнιη’s advice to turn off buffering with stdbuf; e.g.,

./grabserial -d "/dev/ttyUSB4" -b 9600 | stdbuf -oL tr '#' '\n' | sed 's/$/#\r/' > RFIDs.txt

In case it’s unclear, the option to stdbuf is lower-case o (oh) and upper-case L (ell), which says that the standard output should be line-buffered.

Bottom line (TL;DR) #1:

So you should be able to use

./grabserial -d "/dev/ttyUSB4" -b 9600 | stdbuf -oL tr '#' '\n' > RFIDs.txt

to get the entries without the # at the end, or

./grabserial -d "/dev/ttyUSB4" -b 9600 | stdbuf -oL tr '#' '\n' | sed 's/$/#/' > RFIDs.txt

to get the entries with the # at the end, but without the \r, or

./grabserial -d "/dev/ttyUSB4" -b 9600 | stdbuf -oL tr '#' '\n' | sed 's/$/#\r/' > RFIDs.txt

to get the entries with the # and the \r at the end.

Bottom line (TL;DR) #2:

You haven’t said how you’re terminating the command.  If you’re typing Ctrl+C, that might be causing a problem1.  If so, try:

  • putting one of the above command lines into the background with &, and then
  • when you’re ready to quit, do a ps and get the PID of the grabserial process, and
  • kill it with the kill command.

Bottom line (TL;DR) #3:

You say you need to be able to read the data in real time, without going into specifics.  If you have some program / script that is capable of reading from stdin, try

./grabserial -d "/dev/ttyUSB4" -b 9600 | stdbuf -oL tr '#' '\n' | sed 's/$/#\r/' | tee RFIDs.txt | your_program 

This should write the data to the output file while simultaneously piping it to your program.

Bottom line (TL;DR) #4:

If, in fact, cat works as well as grabserial, then you don’t actually need either one; things like

stdbuf -oL tr '#' '\n' < /dev/ttyUSB4 | sed 's/$/#\r/' > RFIDs.txt

should work just as well.
____________
1 Based on your update, this is probably not your problem, but I’m leaving this here in case it helps somebody else.


Note that tr basically operates on a one-to-one basis.  (There are some options to do some slightly fancier stuff.)  So, for example,

tr 'abc…' 'ABC…'

will replace a with A, b with B, c with C, etc.  So you can’t do something like

tr '#' '#\r\n'
4
  • Thanks. that at least got me closer, but I still can't get it to a text file for some reason. No matter how I redirect the output, either to a file, or to another pipe, I just get no output.
    – Danny F
    Sep 23, 2017 at 19:04
  • I updated the main question with the results of your suggestion.
    – Danny F
    Sep 23, 2017 at 20:06
  • I added a whole bunch of stuff to my answer in response to your comment.  Now that I have a clearer idea of your problem (from your update to the question), a lot of it is irrelevant, redundant, and / or superfluous, but I’m leaving it there as it may help others. Sep 23, 2017 at 20:50
  • 1
    Thanks for your help. FWIW, in your Bottom Line #1 example, I had to use the "-u" flag for sed, to get it to send to a text file without buffering (if I wanted the # at the end of the line).
    – Danny F
    Sep 23, 2017 at 21:20
1

Because sed normally buffering its output until a buffer size fills up ( Probably 4K bytes), you could use -u as GNU specific option to unbuffer the output.

Please see Turn off buffering in pipe and related answer for sed.

1
  • Thank you for your suggestion here. I felt like the bulk of the answer came from G-Man, so I awarded him the "solve", but yours was helpful, and useful in this scenario.
    – Danny F
    Sep 23, 2017 at 21:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .