/dev/console behavior that makes no sense to me

Question

[Context: After I read Linux: Difference between /dev/console , /dev/tty and /dev/tty0, I invented a few tests to check my understanding. As a result, I learned that my understanding of /dev/console is NON-EXISTANT :). It's possible that some points raised here should also be added as comments to the question above]

I am running arch linux, and there is no "console=" kernel parameter at boot time. The output of dmesg contains the following three lines, written during boot:

[    0.183964] printk: console [tty0] enabled
...
[    4.908815] fbcon: Deferring console take-over
...
[    5.217652] fbcon: Taking over console

Question 1: The QA I link to above asserts that

/dev/console is a virtual set of devices which ... by default points to /dev/tty0.

What is the actual meaning of "points to"? is that the same as "is a link to"? If so, is it a hard link or a soft link?

---

Now I start X on my system. I use openbox, and as is usual these days, the desktop is shown on the FIRST virtual console. (That is, Alt-Ctl-F2 gets me a new login terminal, and Alt-Ctl-F1 gets me back to my gui-desktopi). I get the following by typing into the first xterm that opens on the desktop:

crw------- 1 root root 5,  1 Nov 30 01:07 /dev/console
crw-rw-rw- 1 root tty  5,  0 Nov 29 23:23 /dev/tty
crw--w---- 1 root tty  4,  0 Nov 30 01:07 /dev/tty0
crw-rw---- 1 root uucp 4, 64 Nov 29 23:15 /dev/ttyS0
[root@scott-dell7577 ~]# tty
/dev/pts/0

So in this case /dev/console seems not to point to anything.

Question 2: Is it because "fbcon took over the console" that /dev/console no longer points to /dev/tty0? Does /dev/console point to some other 'underlying' raw device?

---

Now I have a REAL problem: I can't look at the output of /dev/console. In the old days, X put its graphical interface on terminal 7, and I could see output to /dev/console by choosing Alt-Ctl-F1. But now, the GUI is itself on Terminal 1, and I can't see how to "peek underneath" the deskop to see the console character table. The only two ways I can think of to view /dev/console are 1. to kill X via Alt-Ctrl-Backspace; or 2. use netconsole to view the console output on a remote machine.

Question 3: Am I correct that when i kill X, I will then see all the messages that were written to the console while X was running. Is there some way to view console output by "peeking" at the underlying terminal on which the GUI is being displayed? Or perhaps I can modify the startx script so that the GUI will display on the second virtual terminal and not the first? Is there some reason that this big change -- moving the GUI to the first VT where it now covers the console -- was instituted? It seems to have the considerable disadvantage of making it next to impossible to see the console output, so I'd like to understand what benefit it brings.

---

Now is when things get truly weird for me. I know that only root can write to /dev/console. Following on from the QA cited above, I typed the following into a non-root xterm on the dekstop. Can you explain the response to the last command?

[scott@scott-dell7577 ~]$ echo hello >/dev/console
bash: /dev/console: Permission denied
[scott@scott-dell7577 ~]$ su 
Password: 
[root@scott-dell7577 scott]#  echo hello >/dev/console
[root@scott-dell7577 scott]# exit
exit
[scott@scott-dell7577 ~]$ sudo echo hello  > /dev/console
bash: /dev/console: Permission denied

As an ordinary user, I get permission denied when trying to write to the console. I su to root, and now I get no error. (Of course, i also get no echoed output, I believe that's because as I stated above, the console output is "hidden underneath" the GUI.) I then exit root, and run the same command via sudo. WTF?

Question 5: What is the difference between running the command as root and then running it via sudo, that one should be allowed and the other forbidden?

---

As a final exercise, I did set up the netconsole facility, following the directions here. Everything seemed to connect, all diagnostic/verbose output suggested that the console was in fact being redirected. But when I would enter the command

echo hello > /dev/console

on the local machine, there was no error. But no "hello" appeared either. Could it be some sort of simple buffering issue that causes writes to /dev/console not to appear at the remote end of the pipe?

Sorry to drone on, but i found it upsetting to see reality agreeing so poorly with what I expected. The two biggest issues being, of course, 1. that there is a command which the root can perform but returns "Permission denied " when run with sudo; and 2. that using the new default of having X display on /dev/ttyS0 seems to make it impossible to see /dev/console output.

Thanks for any correct answers to these five questions!

Answer 1

~ Where did the echo hello >/dev/console go?

That will go either to:

1. /dev/tty0 (the active virtual terminal: the last of /dev/tty1, /dev/tty2, etc that was switched to, which may be hidden behind your GUI).
2. a serial or virtual terminal specified by the last console=ttyXXX kernel boot command line argument.
3. another tty where /dev/console was later redirected with ioctl(fd, TIOCCONS). That redirection will only affect userland output to /dev/console (kernel messages will not be redirected). Also, to make it more fun, there's absolutely no way to determine where it has been redirected ;-)

In any case, echoing stuff to /dev/console does not cause it to become a kernel message and appear in dmesg (and wherever kernel messages are redicted via netconsole or other facility).

[/dev/console] by default points to /dev/tty0.

What is the actual meaning of "points to"? is that the same as "is a link to"? If so, is it a hard link or a soft link?

No, /dev/console is a separate character device (just like /dev/tty => controlling terminal or /dev/tty0 => active virtual terminal). All the magic happens in the kernel. To get the actual terminal /dev/console points to (not the one it was redirected to as per point 3. above), you could use the TIOCGDEV ioctl. Unlike /dev/tty, that could be used reliably with /dev/console, since /dev/console cannot refer a pseudo-tty.

Is there some way to view console output by "peeking" at the underlying terminal on which the GUI is being displayed?

There's /dev/vcs*, but vts aren't properly updated while behind a GUI; new data keeps overwriting the last line, instead of causing them to scroll.

# perl -0777 -CO -e '
   $c = unpack 'xC', <>; close ARGV;
   $t = join "", map chr $_, unpack "L*", <>;
   s/(.{$c})/$1\n/g, s/ +$//mg, s/\n+$/\n/ for $t;
   print $t
' /dev/vcs{a,u}
...
content of the active vt
...