POSIX thread (pthread) and OpenMP are both libraries for thread programming. But is it right that they are not native to Linux, i.e. they have to be installed by user later?
If yes, what are the native library or functions in Linux? Are they used to implement pthread and OpenMP?
To draw a parallel comparison to process programming, if I am correct, functions fork(), exec*(), waitpid() and pipe() are offered natively by Linux, while MPI is not. Is MPI implemented in those native functions for process programming?
What do you meant by “non-native”? There isn't a clear definition of “installed by default”, since each distribution has its own default set of installed packages and it's very easy to tune that set.
POSIX threads are part of GNU libc, which is a fundamental part of any non-embedded Linux systems (there are substitutes for small systems, I think the major ones also include pthread support). The Linux kernel itself includes support for threads. It's not exactly pthread, but the distinction between what's supported by the kernel alone and what's supported by the standard library on top of the kernel is rarely useful.
OpenMP comes through GOMP, which is part of gcc. Unlike the standard library (Glibc), it's possible to have a normal Linux system without libgomp installed.
Everything is ultimately implemented on top of system calls, i.e. the functionality provided by the kernel. MP, MPI and other libraries are implemented in terms of system calls for process management, interprocess communication (pipes, sockets, shared memory, …) and multithreading (locks, conditions, …).