1

I have an RFID reader which sends the Unique ID of the RFID tag to the serial port of my linux (raspberry pi) box every time it gets a successful read. The problem is that it is sending it without any newline characters. Here's an example of the output:

root@scalepi:~/scale# ./grabserial -d "/dev/ttyUSB4" -b 9600 
$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#$A0112OKD9990001200005643E#

I'd like to write each entry to a text file with a new line, like this:

$A0112OKD9990001200005643E#
$A0112OKD9990001200005643E#
$A0112OKD9990001200005643E#

As you can see each entry begins with $ and ends with #

I have tried just piping the outout to sed using this line to add a new line after the #, but I get zero output, because I think sed is looking for a new line before it processes the data:

./grabserial -d "/dev/ttyUSB4" -b 9600 | sed 's/#/#\r\n/g'

If I redirect the output to a file, and then cat the file, piped through that same sed command, I get what I expect, but I need to be able to write the data to a file in this format, so I can read it in realtime.

root@scalepi:~/scale# ./grabserial -d "/dev/ttyUSB4" -b 9600 > /tmp/test2.txt
^C
root@scalepi:~/scale# cat /tmp/test2.txt | sed 's/#/#\r\n/g'
$A0112OKD9990001200005643E#
$A0112OKD9990001200005643E#
$A0112OKD9990001200005643E#
root@scalepi:~/scale# 

If anyone has any suggestions for me, I'd appreciate it. Thanks!

EDIT - UPDATED

After trying the "tr" solution below, I was able to get real time output, but if I try to redirect via pipe, or via stdout/stderr redirection, I get buffered results of 50-60 lines at a time instead of getting a line by line result.

The following command will give me real time output as it comes in on the serial port:

./grabserial -d "/dev/ttyUSB4" -b 9600 | tr '#' '\n'

The following command will give me output in chunks of 50 or 60 lines at a time (not real time):

./grabserial -d "/dev/ttyUSB4" -b 9600 | tr '#' '\n' | sed -u 's/$/#\r/g'

The following command will give me output to a text file in chunks of 50 or 60 lines at a time (not real time):

./grabserial -d "/dev/ttyUSB4" -b 9600 | tr '#' '\n'  > /tmp/test1.txt

FWIW, I get the same results using cat to read the serial port, instead of "grabserial".

The following command gives me realtime results:

cat /dev/ttyUSB4 | tr '#' '\n' 

But when I try to redirect output via either pipe or stdout redirection, I get buffered chunks of data in the text file:

cat /dev/ttyUSB4 | tr '#' '\n' > /tmp/test4.txt
1

You can translate (or transliterate) the # characters into newlines with

 tr '#' '\n'

so

./grabserial -d "/dev/ttyUSB4" -b 9600 | tr '#' '\n'

should give you

$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
$A0112OKD9990001200005643E
            ︙

in real time.  If you need the pound signs and/or the carriage returns, you can pipe the above into

        … sed 's/$/#\r/'

(using the fact that sed (and regular expressions in general) use $ to represent the end of a line.

Gotcha

You may have a problem that tr buffers its output when that output is going to a pipe or a file.  Perhaps the best way to handle this is to follow αғsнιη’s advice to turn off buffering with stdbuf; e.g.,

./grabserial -d "/dev/ttyUSB4" -b 9600 | stdbuf -oL tr '#' '\n' | sed 's/$/#\r/' > RFIDs.txt

In case it’s unclear, the option to stdbuf is lower-case o (oh) and upper-case L (ell), which says that the standard output should be line-buffered.

Bottom line (TL;DR) #1:

So you should be able to use

./grabserial -d "/dev/ttyUSB4" -b 9600 | stdbuf -oL tr '#' '\n' > RFIDs.txt

to get the entries without the # at the end, or

./grabserial -d "/dev/ttyUSB4" -b 9600 | stdbuf -oL tr '#' '\n' | sed 's/$/#/' > RFIDs.txt

to get the entries with the # at the end, but without the \r, or

./grabserial -d "/dev/ttyUSB4" -b 9600 | stdbuf -oL tr '#' '\n' | sed 's/$/#\r/' > RFIDs.txt

to get the entries with the # and the \r at the end.

Bottom line (TL;DR) #2:

You haven’t said how you’re terminating the command.  If you’re typing Ctrl+C, that might be causing a problem1.  If so, try:

  • putting one of the above command lines into the background with &, and then
  • when you’re ready to quit, do a ps and get the PID of the grabserial process, and
  • kill it with the kill command.

Bottom line (TL;DR) #3:

You say you need to be able to read the data in real time, without going into specifics.  If you have some program / script that is capable of reading from stdin, try

./grabserial -d "/dev/ttyUSB4" -b 9600 | stdbuf -oL tr '#' '\n' | sed 's/$/#\r/' | tee RFIDs.txt | your_program 

This should write the data to the output file while simultaneously piping it to your program.

Bottom line (TL;DR) #4:

If, in fact, cat works as well as grabserial, then you don’t actually need either one; things like

stdbuf -oL tr '#' '\n' < /dev/ttyUSB4 | sed 's/$/#\r/' > RFIDs.txt

should work just as well.
____________
1 Based on your update, this is probably not your problem, but I’m leaving this here in case it helps somebody else.


Note that tr basically operates on a one-to-one basis.  (There are some options to do some slightly fancier stuff.)  So, for example,

tr 'abc…' 'ABC…'

will replace a with A, b with B, c with C, etc.  So you can’t do something like

tr '#' '#\r\n'
  • Thanks. that at least got me closer, but I still can't get it to a text file for some reason. No matter how I redirect the output, either to a file, or to another pipe, I just get no output. – Danny F Sep 23 '17 at 19:04
  • I updated the main question with the results of your suggestion. – Danny F Sep 23 '17 at 20:06
  • I added a whole bunch of stuff to my answer in response to your comment.  Now that I have a clearer idea of your problem (from your update to the question), a lot of it is irrelevant, redundant, and / or superfluous, but I’m leaving it there as it may help others. – G-Man Sep 23 '17 at 20:50
  • 1
    Thanks for your help. FWIW, in your Bottom Line #1 example, I had to use the "-u" flag for sed, to get it to send to a text file without buffering (if I wanted the # at the end of the line). – Danny F Sep 23 '17 at 21:20
1

Because sed normally buffering its output until a buffer size fills up ( Probably 4K bytes), you could use -u as GNU specific option to unbuffer the output.

Please see Turn off buffering in pipe and related answer for sed.

  • Thank you for your suggestion here. I felt like the bulk of the answer came from G-Man, so I awarded him the "solve", but yours was helpful, and useful in this scenario. – Danny F Sep 23 '17 at 21:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.