7z and lzma are the same compression algorithm, with a different container. 7z with solid archive mode enabled should do about as well as tar.7z, and provide not as bad random random access to a single file. (Still bad, though.)
pdf uses gzip internally, which makes it not very compressible. Same for most image formats (although the choice of entropy coder varies; IIRC JPEG's entropy coder is simpler).
In theory, and I don't know of any implementation of this idea, you could have an archiver that undoes the simple gzip or other entropy coding of pdf, png, jpeg, and various other already-compressed file formats. Then compress that stream with something good like LZMA. On extraction, you'd extract data from the LZMA stream, and redo the pdf-internal compression on the parts that needed it. Your output would be the same pdf/jpg/whatever, but the files would potentially have different checksums/hashes. (So it would be lossless with respect to the final rendered pixels, but not wrt the file bytes.)
PNG and jpeg optimizers sort of implement this idea for a single file: redo the internal compression with the equivalent of gzip -9 instead of the default.
Your best bet is to try compressing a sample of 1GB or so with various compressors, and see what does well. 1GB should be representative, because that's about as big as is reasonable for a dictionary size.
