17

I just read the answers to "Removing a newline character at the end of a file" and everyone said to delete the last character. My question is, isn't the eof character the last one?

  • 10
    EOF is not a character. – Soren Bjornstad Oct 8 '16 at 23:02
  • 1
    @SorenBjornstad I'd also like to add that when there is a newline at the end of a Unix text file, it's there because it terminates the last line. An empty text file has no newline at the end: it's a sequence of zero characters. – Kaz Oct 8 '16 at 23:17
  • 3
    To be slightly pedantic, CPM and DOS did use ^Z as the EOF character, and you may still sometimes encounter files that end with ^Z. – Edward Falk Oct 11 '16 at 21:07
12

A file does not end with an End of File character, as the previous answers correctly state. But I think the answers and comments contain some inaccuracies worth pointing out:

  • The ASCII character set does not contain an exact EOF character. There are several "end" control characters: End of Text (3), End of Transmission (4), End of Transmission Block (23), End of Medium (25). File Separator (28) maybe comes closest to an EOF character. Code 26 is "Substitute", not EOF.

  • Ctrl-D is only associated with terminal input. For example the command cat filea fileb filec > outfile does not involve Ctrl-D. By the way, you can change the terminal EOF character to something else than Ctrl-D using the stty command.

  • Strictly speaking, Ctrl-D (or whatever you have changed to) is not an EOF key code. What it does is make the read system call return with what input is available, just like pressing return makes the read system call return a line of characters to the caller. By convention a return value of zero from the read system call (i.e. zero characters read) signals an end of file condition. However, the input file is not closed automatically, and, if the input comes from the terminal, it is not put in an "end of file" state. You can write a program that continues reading from the terminal even after an "end of file" and the read call can return non-zero for the next input line.

  • The analogy between the eof and eol characters can be seen if Ctrl-D is pressed when some input has already been written on the line. For example, if you write "abc" and the press Ctrl-D the read call returns, this time with a return value of 3 and with "abc" stored in the buffer passed as argument. Because read does not return 0, this is not interpreted as an EOF condition by the convention above. Similarly, pressing return to makes the read call return with the whole input line (including newline). You can try this out with the cat command: write some characters on the line and press Ctrl-D. You'll see the characters echoed back to you and cat waiting for more input.

  • All the above only applies when the terminal is in the "cooked" mode, as opposed to "raw" mode, in which line input processing is minimized. In raw mode a Ctrl-D character really is delivered to the input buffer.

18

ASCII control characters have definitions from the 1960s (actually preceding what you might consider a network). Not all of those control characters are used in the way that they were defined for telecommunications equipment back then.

On Unix-like systems, there is no need for an EOF character; none is used. The system can tell applications how many bytes are in a file:

  • On some other systems (seen in VMS, DOS, Windows), a control-Z may act as an end-of-file marker because in older versions the system could not tell some applications how many bytes are in the file.

    In the case of VMS, the limitation was due to the way the C runtime worked. Assembly-language applications could (and did) get the correct file size.

  • Unix systems in the shell conventionally use control-D to tell an application that an end of input (file) has been reached, but the control-D is not stored in the file.

In C, EOF is purposely made -1 to indicate that it is not a valid character. Standard I/O returns EOF when an end-of-file condition is detected — not a special character.

By the way, files need not end with a newline (ASCII line-feed) character. Text editors can cope with files which are all printable text but lack a trailing newline.

  • 8
    POSIX defines a text file as a file containing a sequence of lines and in turn each line as a sequence of non-newline characters followed by one newline. Thus a file ending with anything but 0x0A is not a conforming text file. – Damian Yerrick Oct 8 '16 at 22:07
  • 2
    I'm aware of that, which is why I pointed out that text editors work. (Binary files have no such constraint). – Thomas Dickey Oct 8 '16 at 22:14
  • It's really worth noting that files intended to be handled as text which don't have a trailing newline are still arguably bad form (even if typical text editors have been coded to compensate for such files), at least if you actually want it to be broadly user-friendly/compatible, because the lack of a trailing newline can add additional difficulties in various circumstances (concatenating/printing multiple text files, parsing with typical command-line tools, minimal editors like busybox's vi, etc). – mtraceur Oct 8 '16 at 22:58
  • (1) Before VMS, RT-11 RSX-11 TOPS-10 had filesystems only precise to a block and needed an EOF character. So did CP/M, which apparently copied it from DEC and was in turn copied by early MS-DOS and then passed down to Windows. (2) In Unix it's the tty driver not the shell, as described in more detail by JohanM, although people usually run shells on tty devices. – dave_thompson_085 Oct 9 '16 at 12:58
  • Sure - DEC was back there (and note that I mentioned older versions). Whether it was the origin of the CP/M feature would be an interesting topic to explore (not here); I mentioned those cases to give some background to the alternatives. – Thomas Dickey Oct 9 '16 at 14:18
7

EOF is not a character. It is a state which indicates no more characters to read from a file stream. When you enter EOF command from the terminal, you are signalling the OS to close the input stream, not putting in a special character.

  • 1
    Yes but in ASCII table EOF is 26 so I thought the last byte was the binary representation of 26. So how could a program which reads an input know where it ends? – sworwitz Oct 8 '16 at 17:10
  • ASCII was meant for passing information over a network. In that case, you need an EOF character. (ASCII had a lot of control codes as well. Not everything was printable.) In case of file streams, the size of the file is already known through the file system so the OS can tell when there is no more data to read. – Munir Oct 8 '16 at 17:14
  • @sworwitz: With regards to C, input reading functions that return a character per call return an int (usually a 32 bit number but must be minimum 16 bits) not a char. The function signals and EOF by returning -1 (0xffffffff) which is not a valid 8 bit value so it won't be confused by any ASCII character, not even 0xff. Functions that return a string also returns the length of the read data. This length may be used to signal no data or end of data (again, length can be -1). Finally, there is also a function that you can call that will tell you if a stream has reached the end – slebetman Oct 8 '16 at 19:09
  • Ok thank you! So when in bash I press Ctrl+d I give in input the ASCII character, right? – sworwitz Oct 8 '16 at 19:17
  • @sworwitz Not exactly. Before bash gets its hands on the input, it is massaged by the TTY driver. This driver intercepts Ctrl-D and sends an EOF to bash (Where EOF is not a character, but a special file status) – Stig Hemmer Oct 10 '16 at 7:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.