# Read from and append to file at the same time while preserving end-of-line

Conceptually I have an easy task... I have [loosely] structured data in a file:

Testing: debug, default CXXFLAGS
Testing: release, default CXXFLAGS
...


I try to summarize it in a log file:

echo "Configurations tested:" | tee -a "$TEST_RESULTS" echo$($GREP 'Testing: ' "$TEST_RESULTS" | $SED 's/Testing: / * /g') | tee -a "$TEST_RESULTS"


Configurations tested:
* debug, default CXXFLAGS
* release, default CXXFLAGS


I get:

Configurations tested:
1 3way.cpp 3way.h CMakeLists.txt CMakeLists.txt.diff Doxyfile Filelist.txt GNUmakefile


I think I am wreaking havoc on the file buffer $TEST_RESULTS because its being read from in the grep, and written to with the tee. When I attempt to put the result of $GREP 'Testing: ' "$TEST_RESULTS" |$SED 's/Testing: / * /g' in a shell variable, I loose the line endings which results in one big concatenation:

* debug, default CXXFLAGS    * release, default CXXFLAGS ... <30 additional configs>


How do I read from and append to a file at the same time while preserving the end-of-lines?

ESCAPED=$($GREP 'Testing: ' "$TEST_RESULTS" |$AWK -F ": " '{print "  -" $2 "$"}')
echo $ESCAPED | tr$ '\n' | tee -a "$TEST_RESULTS"  However, it can't use * as a bullet point, and it seems to drop leading space: Configurations tested: -debug, default CXXFLAGS -release, default CXXFLAGS  I'm not using sed because swapping-in a new line is an absolute pain across platforms. Platforms include BSD, Cygwin, Linux, OS X, Solaris. ## 2 Answers I would try something like: awk -v pattern="Testing:" '$0 ~ pattern { sub(pattern, "  *"); print }'


This should work with any version of sed, since it contains no obvious extensions. It is not necessary to handle the newlines explicitly as long as you take care to quote your variables properly, in order to prevent word splitting.

• "Frankly, I don't see why you are messing around with the newlines...." - I told you why: "When I attempt to put the result of <command above> in a shell variable, I loose the line endings which results in one big concatenation". – user56041 Jun 29 '16 at 1:06
• Well, if you don't want the shell to do its thing, then use quotes. echo $var is not the same as echo "$var". – Michael Vehrs Jun 29 '16 at 6:00
• Thanks. " if you don't want the shell to do its thing, then use quotes" - OK, could you provide it in your answer? Also, does your Awk work everywhere? I try to avoid Sed because of the different behavior on BSDs, Linux, OS X, Solaris and Unix. Its been my experience that Awk can be a pain point too. – user56041 Jun 29 '16 at 6:07

Assuming $TEST_RESULTS is the name of a file that contains the test output. If I understand correctly, you want to append the list of tested configurations to that same file. So, use a temporary file. Then append that file to the original. tmpfile="$( mktemp )"
echo "Configurations tested:" >"$tmpfile" sed -n 's/^Testing: $$.*$$/ * \1/p' "$TEST_RESULTS" >>"$tmpfile" cat "$tmpfile" >>"$TEST_RESULTS" rm -f "$tmpfile"


If you don't have mktemp, you could use

tmpfile="$TEST_RESULTS.tmp"  or tmpfile="/tmp/$( basename "\$TEST_RESULTS" ).tmp"


or something similar... assuming that file doesn't already exist (in which case it will be overwritten and then removed).