A (home/small office) router's DNS functionality is usually limited to forwarding any DNS requests that arrive to it from the local side to the Internet Service Provider's DNS servers, and passing any responses back to the client that made the request.
There are no guarantees that a router could always act as a fully-featured DNS server for the local network. Some routers might be able to do that, most aren't.
Even so, a DNS service isn't plug-and-play: classically, the administrator of a DNS server would have to edit the appropriate DNS zone files on the DNS server to add DNS records for the hosts that should be accessible by name. The modern addition of dynamic DNS updates allows a host to register itself by name, or the host can tell a DDNS-aware DHCP server to do it on the host's behalf.
(Since a home/small office router usually also has a built-in DHCP server, it might be able to do this as a single integrated package.)
But an unsecured dynamically-updateable DNS service is asking for trouble, so such functionality is not likely to be enabled by default. Secure dynamic DNS updates would need either some rules defining which host can do what, or some form of authentication. As a result, this cannot be done automatically.
There are other services developed for hostname resolution that don't need explicit configuring.
Multicast DNS (mDNS) is a relatively new solution, implemented by Apple Bonjour, open-source Avahi packages in Linux and in a very limited sense (network printer discovery only) by Windows 10.
Link-Local Multicast Name Resolution (LLMNR) is another name resolution protocol that requires no explicit configuration and allows systems in the local network to find each other by name. It is implemented in Windows Vista and above, and in Linux by systemd-resolved.
If none of these services are available (or the system whose name you would need to resolve isn't compatible), then the traditional fall-back in small networks is to add the name and IP address of the host(s) you wish to be resolvable by name to /etc/hosts. Of course, you'll need to do this on every host in the local network and keep the hosts files in sync to make it truly useful.
(Fun fact: in the early days of the internet, before DNS was developed, there used to be a single, centrally-managed master hosts file for the whole internet. The fact that maintaining the hosts file became unworkable was one of the main reasons of the development of DNS.)
