PING is indeed a client.
The ping command also uses a part of the ICMP protocol, namely echo-reply (ICMP type 0 message) and echo-request (ICMP message type 8).
Many professionals and network monitoring software use echo-request/reply successful ICMP messages processing as an indicator a system is up/down. However this is a convention, and not strictly mandatory. For instance, I can define in Nagios that I will monitor my Linux servers with the SSH TCP/22 port instead of using PINGs.
The concept as an established connection does not exist per se as in TCP connections. ICMP is a not a connection oriented protocol.
As stated earlier, the concept of port also does not exist - the Linux kernel processes an ICMP packet, and throws an answer to it accordingly, forgetting then about the deed (ignoring other mechanisms as rate limiting, for instance).
Ping is also available for users without special privileges, or has to be a setuid binary, as it uses RAW sockets to manufacture a ICMP packet.
You can also define if the kernels answer or not to an ICMP echo-request message with a sysctl/proc setting.
To disable answers to ICMP pings do:
echo "1" > /proc/sys/net/ipv4/icmp_echo_ignore_all
Or to enable it again:
echo "0" > /proc/sys/net/ipv4/icmp_echo_ignore_all
As for the part of the kernel responsible for handling ICMP messages, it is found in
icmp.c in the Linux kernel sources as in https://github.com/torvalds/linux/blob/master/net/ipv4/icmp.c
As for the ICMP packet see this image:
As for more kernel definitions:
#define ICMP_ECHOREPLY 0 /* Echo Reply */
#define ICMP_ECHO 8 /* Echo Request */