-t option forces [the standard file descriptors] to refer to a TTY device. Is this the correct way of reasoning in order to understand what this option does?
-t option will run the command on the remote machine with its stdin/out/err connected to a pseudo-tty slave instead of a pair of pipes.
Running it in a pty is the default if (a) no explicit command is given and (b) the stdin of the ssh client is itself a tty. You need a single
-t to force tty allocation even when the (a) condition is not satisfied, and two of them (
-tt) when (b) is not satisfied.
By which mechanism does ssh allocate that TTY?
By some system-dependent mechanism. Nowadays, it's mostly the standard
master_fd = posix_openpt() followed by
slave_fd = open(ptsname(master_fd)) 
Does ssh create new TTY on server or on client?
ssh always create the new pseudo-tty on the server, ie on the remote machine. If a pseudo-tty is allocated on the server and the local (client's) stdin is a tty, the local tty will be set to raw mode.
How to check this? (a new node in
/dev/ must appear or something
Not necessarily. But on a regular linux machine of 2019, a new file will appear under
/dev/pts for each new pseudo-tty.
And how this new TTY is tied to existing STDIN, STDOUT and STDERR?
Just like any other file descriptor, with
dup2(slave_fd, 1), etc.
dup2(newfd, oldfd) will close whatever file
oldfd was referring to. If a pty was allocated, it will also be made the controlling tty of the remote session.
An example of use case of
-t option is welcome.
ssh -t /bin/bash on a system where your login shell is
csh. If you leave out the
/bin/bash will run without prompt, job control, line editing capabilities, etc.
And what is so special about TTY which may not be achieved using ordinary STDIN, STDOUT and STDERR?
See above ;-)
And a pipe is no more of an "ordinary" stdin than a tty or some other kind of file.
 that itself has a lot of problems (with multiple devpts mounts and mount namespaces) that's why the
TIOCGPTPEER ioctl was added to Linux, which returns a fd referring to the slave without going through the file system.