2

I have an input stream which I do get with netcat. The input stream DOES NOT contain any newline characters. instead there are 4 spaces where there should be newline. So I need to replace each occurrence of 4 spaces with the newline character.

I was trying to use

netcat xxxx | sed 's/    /\n/g' | myscript.sh

so I am trying to pipe through sed or awk but without success. Both commands seem to be waiting indefinitely for newline character in input stream before processing replacement. Both sed and awk work perfectly on file, but NOT when they are part of the pipe. I was trying awk with

RS='    '

or

-F'    '

but no success either. both sed and awk seem to swallow everything until real newline found in input file which will never happen.

1
  • 1
    Do you have GNU awk (check awk --version)? Setting RS to 4 blanks as you say you tried should do what you want with GNU awk but not others. edit your question to show us the awk script you tried that had RS=' '. Setting FS=' ' doesn't make sense.
    – Ed Morton
    Nov 13, 2021 at 14:37

1 Answer 1

1

Yeah, you're getting killed by buffering: sed is trying to read a line.

This may help:

input=""
netcat xxx | while IFS= read -r -d "" -n 1 char; do
    input+=$char
    if [[ $input == *"    " ]]; then
        echo "${input%    }"
        output=""
    fi
done | yourscript.sh

On second though, would be better: define the input record separator to be 4 spaces and print every record (multi-character RS is a GNU awk feature)

netcat xxx | gawk -v RS="    " 1 | yourscript.sh

This never-ending one-line input can be simulated by a bash function:

netcat() { while true; do printf '%d    ' $RANDOM; done; }
3
  • 2
    You should mention that awk script requires GNU awk for multi-char RS. It is the right approach if you have it but according to the OP they already tried that and it didn't work for them so I suspect they just haven't tried it with GNU awk yet.
    – Ed Morton
    Nov 13, 2021 at 14:38
  • Although it does work with with mawk and the default BSD awk on my Mac. Nov 13, 2021 at 16:30
  • I could believe it works with newer versions of mawk as it now has added several gawk enhancements, but I've never heard of a BSD awk supporting multi-char RS before. Are you sure about that one and it's not just treating it like RS=" "?
    – Ed Morton
    Nov 13, 2021 at 19:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.