A lot of peripherals are removable (USB, Firewire, PCMCIA, hotpluggable PCI, etc.). Furthermore a system installation might be moved to new hardware if the old hardware failed or had to be upgraded. Linux distributions tend to provide all the drivers that you might possibly need, preferring to waste a few megabytes of disk space rather than tell you “keyboard not recognized, press F1 to continue”¹ or “network adapter not recognized; downloading driver failed: network not reachable”².
Updates are offered for all the software you have installed. The package manager can't predict that you're never going to use some piece of software. The way to tell a package manager you don't want t piece of software is to install it.
If you're sure you aren't going to need some driver or some firmware — for example wifi on a server that's sitting on a rack or running in a virtual machine, then you can safely remove packages that provide add-on drivers and firmware. Never remove individual files, only RPM packages. If removing a package would also pull out some package whose role you don't understand, leave it alone.
¹ Seen on many BIOSes (paraphrased).
² Seen on Windows (paraphrased).