What internally limits these things?
ZFS's limits are based on fixed-size integers because that's the fastest way to do arithmetic in a computer.
The alternative is called arbitrary-precision arithmetic, but it's inherently slow. This is why arbitrary-precision arithmetic is an add-on library in most programming languages, not the default way of doing arithmetic. There are exceptions, but these are usually mathematics-oriented DSLs like
bc or Wolfram Language.
If you want fast arithmetic, you use fixed-size words, period.
The speed hit from arbitrary precision arithmetic is bad enough inside a computer's RAM, but when a filesystem doesn't know how many reads it needs to make in order to load all of the numbers it needs into RAM, that would be very costly. A filesystem based on arbitrary sized integers would have to piece each number together from multiple blocks, requiring a lot of extra I/O from multiple disk hits relative to a filesystem that knows up front how big its metadata blocks are.
Now let's discuss the practical import of each of those limits:
Max. volume size
2128 bytes is effectively infinite already. We can write that number instead as roughly 1038 bytes, which means in order to hit that limit, you'd have to have a single Earth-sized ZFS pool where every one of its 1050 atoms is used to store data, and each byte is stored by an element no larger than 1012 atoms.
1012 atoms sounds like a lot, but it's only about 47 picograms of silicon.
The density of data in grams is 2.5×10-13 g/byte for microSD storage, as of this writing: the largest available SD card is 1 TB, and it weighs about 0.25g.¹ A microSD card isn't made of pure silicon, but you can't ignore the packaging, because we'll need some of that in our Earth-computer, too; we'll assume that the low density of the plastic and the higher density of the metal pins average out to about the same density as silicon. We also need some slop here to account for inter-chip interconnects, etc.
A pico-anything is 10-12, so our 47 pg and 2.5×10-13 g/B numbers above are about an order of magnitude apart. That means that to a first approximation, to construct a single maximally-sized ZFS pool out of the current largest-available microSD cards, you might have to use an entire Earth-sized planet's worth of atoms, and then only if you start off with something close to the right mix of silicon, carbon, gold, etc. such that you don't end up with so much slag that you blow the estimate.
If you think it's unfair that I'm using flash storage here instead of something denser like tape or disk, consider the data rates involved, as well as the fact that we haven't even tried to consider redundancy or device replacement. We have to assume that this Earth-sized ZFS pool will be composed of vdevs that never need replacing, and that they can transfer data fast enough that you can fill the pool in a reasonable time. Only solid-state storage makes sense here.
The approximation above is quite rough, and storage densities continue to climb, but keep things in perspective: in the future, to pull off this stunt of constructing maximally-sized ZFS pools, we'll still need to use the total crust-to-core resources of small planets.
Max. file size
So we've got a filesystem the size of a planet now. What can we say about the size of the files stored within it?
Let's give every person on the planet their own equally-sized slice of that pool:
1038 ÷ 1010 ≈ 1028 ÷ 1019 ≈ 109
That's the size of the pool divided by the population of Earth² divided by the maximum file size, in round numbers.
In other words, every person can store about a billion maximally-sized files in their tiny personal slice of our Earth-sized ZFS storage array.
(If it's bothering you that our storage array is still the size of a planet here in this example, remember that it had to be that big in order to hit the first limit above, so it is fair to continue to use it for this example here.)
That per-file maximum file size is 16 EiB under ZFS, which is 16× larger than the maximum volume size of ext4, which is considered ridiculously large today in its own right.
Imagine someone using their slice of Planet ZFS (formerly known as Earth) to store backups of maximally-sized ext4 disk images. Further, this demented customer (there's always one) has decided to
tar them up, 16 per file, just to hit the ZFS maximum file size limit. Having done so, that customer will still have room to do that again about a billion more times.
If you're going to worry about this limit, that's the sort of problem you have to imagine needing to solve. And that's without even getting into the required data bandwidth needed to transfer that file to the online backup service once.
Let's also be clear about how improbable that Earth-computer is. First you'd have to figure out how to construct it without allowing it to collapse in on itself under the force of gravity and become molten at the center. Then you'd have to figure out how to manufacture it using every single atom on Earth without any leftover slag.
Now, since you've turned the surface of the Earth-computer into a hellscape, all the people trying to make use of that computer would have to live somewhere else, a place where you'd frequently hear people cursing the speed-of-light delays that add latency to every transaction between between the Earth-computer and wherever they live now. If you think your ~10ms Internet ping time is a problem today, imagine putting 2.6 light-seconds between your keyboard and the computer if we move the population of Earth to the moon so we can make this Earth-computer.
ZFS's volume and file size limitations are science fiction big.
Max. number of files per directory
248 is roughly 1014 files per directory, which is only going to be a problem for applications that try to treat ZFS as a flat filesystem.
Imagine an Internet researcher who is storing files about each IP address on the Internet. Let's say there are exactly 232 IPs being tracked after first subtracting the slack spaces in the old IPv4 space and then adding in the hosts now using IPv6 addresses to make the arithmetic come out nice. What problem is this researcher trying to tackle which requires him to construct a filing system that can store more than 216 — 65536! — files per IP?
Let's say this researcher is storing files per TCP port as well, so that with just one file per IP:port combination, we've eaten up our 216 multiplier.
The fix is simple: store the per-IP files in a subdirectory named after the IP, and store the per-port files in a subdirectory of the directory holding the per-IP files. Now our researcher can store 1014 files per IP:port combination, sufficient for a long-term global Internet monitoring system.
ZFS's directory size limit isn't what I'd call "science fiction big," as we know of real applications today that can hit this limit, but the power of hierarchy means you can just add another directory layer if you run up against the limit.
This limit is probably set as low as this purely to avoid making the data structures needed to find files in a given directory too big to fit into RAM. It encourages you to organize your data hierarchically to avoid this problem in the first place.
Max. filename length
While this one limit does seem stringent, it actually makes sense.
This limit doesn't originate with ZFS. I believe it dates back to FFS in 4.2BSD. I can't find the quote, but when this limit was young, someone pointed out that this is enough space for "a short letter to grandma."
So, that begs the question: why do you need to name your files more descriptively than that? Any true need greater than that probably calls for hierarchy, at which point you multiply the limit by the number of levels in the hierarchy, plus one. That is, if the file is buried 3 levels deep in the hierarchy, the limit on the name of the full path is 4 × 255 = 1020 characters.
Ultimately, this limit is a human limit, not a technological limit. File names are for the human's use, and humans really don't need more than 255 characters to usefully describe the content of a file. A higher limit simply wouldn't be helpful. The limitation is old (1983) because humans haven't acquired the ability to cope with longer file names since then.
If you're asking where the odd-looking "255" value comes from, it's some limitation based on the size of an 8-bit byte. 28 is 256, and the N-1 value used here probably means they're using a null terminator to mark the end of the file name string in a 256-byte field in the per-file metadata.
Practically speaking, what limits?
I measured this using a scale specified with an accuracy of 0.01g.
7.55 billion, as of this writing. Above, we're rounding this off to 1010, which we should hit by mid-century.