xz has become a bit of a standard when it comes to package management in recent years.
the LZMA2 compression algorithm
xz uses, is exceptionally efficient with text (and "text-like") data. I personally use it for pretty much anything, where computational encoding speed is not an issue (
xz decompresses much faster than compressing, it's actually one of its design targets).
So it's great for things like archiving, or backing up (generally speaking, files that get decompressed a lot, but where data isn't compressed all the time, or changed, etc.). It's not such a great decision for stream compression - at least not all that often, because LZMA2 isn't exactly light on resources, when compressing.
But I'd like to comment on one other statement of yours: "should".
What you should or should not use, depends on what you need your data for. Even though
xz and LZMA2 has been around for years, people still consider compressing with
gz as fall-back for compatibility reasons. Things like Trac or cgit offer
.tar.xz. With the
.tar.gz option being there, because it has been "around forever", and it's a good fall-back measure on any sort of device, no matter how minimal or old.
Just as a counter example,
lzop is a fast and resource saving compressor, which is often used on spacecraft, but it otherwise not very efficient. I couldn't say it's not widely used, though. It's often used on systems where it's more important to quickly compress something with little power and computing space.
So, what you should or should not use, is what makes the most sense to you and the people interacting with your data.
One thing that popped into my mind recently:
man pages are routinely compressed with
gzip. If you look into
/usr/share/man/man1/, for instance, you'll see a lot of files similar to
mv.1.gz, if not all of them are compressed. Now, it would arguably make sense to switch to
.xz files because of the better compression, but the space gains are minimal, compared to the negative effects from ditching
.xz. The reason for that is the almost universal compatibility. Even opening the
.gz compressed files directly in Vim works fine, and acts as transparent compression.