3

If I create a file descriptor in bash and point it at a file and put some data into it:

exec 5>>file
for i in {1..10000}; do
    echo "$i" >&5
done

File descriptor 5 holds open "file" and whenever I write data to that descriptor, it appends the data and keeps its current place in the file.

If I re-point that same descriptor to the same file again:

exec 5>>file

Does it close the current fd and recreate a new one pointing back to that same file? So the next time I write data, it has to find the end of the file before appending onto the end? Or, is it smart enough to realize the descriptor is already pointed to that file, and do nothing. What is going on at a low level?

edit: More specifically I am logging data to a file using file descriptors to hold the file open. Some relevant bits:

recv_udp    () {    # Receives a udp stream, timestamps and removes commas and the period from the seconds field, deletes blank lines.
        socat -L "$LOCKFILE" -u udp-recv:"$UDP_IN",reuseaddr STDIO | ts '%Y %j %H %M %.S' | sed -u -e 's/,/ /g' -e 's/\./ /' -e '/^$/d' &
        }

file_handle () {    # Points file descriptor 9 to the desired log file
        exec 9>>"$DATADIR"/"$CRUISEID"/"$STAMP"/"$STAMP"_"$JDY"_raw
        }

parse_lci90i    () {    # Parse lci90i winch controller mtnw2 data stream.
        echo "Started $STAMP $CRUISEID $(date +'%c')" >> "$LOGFILE"
        while read YR JDY HR MIN SEC uS GARBAGE GARBAGE TENS SPD LINE CHKSUM; do
            printf '%4.0f %03.0f %02.0f %02.0f %02.0f %.3s %s %.0f %.1f %.0f\n' "$YR" "$JDY" "$HR" "$MIN" "$SEC" "$uS" "$STAMP" "$SPD" "$LINE" "$TENS"
            check_path
            file_handle
            printf '%4.0f %03.0f %02.0f %02.0f %02.0f %.3s %s %.0f %.1f %.0f\n' "$YR" "$JDY" "$HR" "$MIN" "$SEC" "$uS" "$STAMP" "$SPD" "$LINE" "$TENS" >&9
        done
        }

Socat reads a 20hz udp feed and pumps it into a parser via process substitution. You can see here that every time a line of data gets read, the file descriptor gets remade. So yes I am curious about whether or not this is any more efficient than just taking the stdout from the parser and just >> right into the log file.

Thanks for the replies so far, you are all pretty insightful. The filename of the data file contains the julian day from the data timestamp, so the log file rolls over every day when the date changes.

  • It might not be the same file at that point, someone could have renamed or deleted it. – ilkkachu Aug 3 '16 at 7:13
  • Is your concern about the speed of finding the end of the file? – Kusalananda Aug 3 '16 at 7:47
1

bash handles redirections of the standard file descriptors 1 (output) and 2 (error) by first flushing them. Aside from that, it does no special actions (does not care what that file descriptor may have been used for), simply calling dup2 to establish a new connection to the file.

As long as you have the file descriptor open, that will follow the same file through a rename. But opening a new file descriptor after the file has been renamed will write to a new file rather than continuing to write to the old one. Likewise, a subprocess writing to the old file will continue until the descriptor is closed

Further reading (source code):

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.