The protocol is plain TCP/IP.
From posgresql documentation about "frontends" and "backends" protocol:
PostgreSQL uses a message-based protocol for communication between frontends and backends (clients and servers). The protocol is supported over TCP/IP and also over Unix-domain sockets. Port number 5432 has been registered with IANA as the customary TCP port number for servers supporting this protocol, but in practice any non-privileged port number can be used.
So regarding iptables use the
tcp protocol, as Unix-domain socket are not meant to be used over network.
iptables <other_options> -p tcp -dport 5432 -j ACCEPT
As pointed out by Lekensteyn, it is especially wise to consider activating SSL over that network connection (see postgresql documentation regarding using TCP-over-SSL). The iptables rule would not change in that case: same port (5432), same protocol (tcp).