On our server, we have some issues with mime types. I need to add the mime type for Microsoft Publisher files, so that file -b -i theFile.pub prints application/x-mspublisher to the command line. I have tried adding it in /etc/mimes.etc, but that does not work. It continues to say application/x-msword.

I have also tried to add an xml via xgd-mime, but that didn't help either.

I have extracted the magic bytes at the start of the file via hexedit, however, I am not sure how to add this to the mime data of file. Can anyone assist please?

Again, this is about command line only, not about icons.

edit to elaborate:

# Adobe ESP Script
0               string  %\!PS-Adobe-3.1\ EPSF-3.0    Adobe EPS file
32              string  %\!PS-Adobe-3.1\ EPSF-3.0    Adobe EPS file
!:strength +50
!:mime application/postscript

I have put this into /etc/magic, ran file -C and this one works. Reads the string at the start of the file or after 32 Bytes.

# Mirosoft Powerpoint
1152    string/b        x50/x00/x6F/X00/x77&x00/x65/x00    Power
!:mime application/vnd.ms-powerpoint

# Microsoft Publisher
0               quad    0xd0cf11e0a1b11ae1           Publisher Magic Number
4512            string  Microsoft\ Publisher\ 3.0    Microsoft Publisher
!:strength +50
!:mime application/vnd.ms-publisher

Those do not work. I have been sitting over hex editors and the man pages for hours, I can't find the fault. I have tried hex, I have tried strings, etc pp.


Short answer:

curl -Ls https://git.io/fj9xx > ole2compounddocs
file -m ole2compounddocs -e cdf -b --mime-type file.pub


Long answer:

First (for posterity, some background):

The file utility (that is, the implementation used today by most systems) is Ian Darwin's "Fine Free File Command" (n some package managers, this is called file; in others, it is called libmagic. The file command is a CLI utility wrapping libmagic. Basically, they're the same thing.

The libmagic library itself typically relies on a "database" (a compressed concatenated list, usually called "magic.mgc", installed in various places depending on OS) of "magic" files that contain regex and other patterns that the library can use fingerprint the "type" of unknown files (or data streams) from samples of raw binary.

The filetype of a file is determined through a list of various criteria based on internal rules, the filename/extension (if any), the "main" compressed magic file, and any magic files added on the command line with the -m flag.

The tool has to strike a balance between what is a file's "type", and what is information contained in it. In many cases, a file extension is of little value, but some; take, for example, the classic case of a .zip file and a .jar file. Both are "zip files", i.e., i.e., they are files compressed with Info-Zip specification. In this case, the file extension should be enough for it to know this is a Java executable, not just a zip archive. However, with Microsoft Publisher's '.pub' extension, there is little value to this, as the most common use of '.pub' on *nix platforms is for SSH public keys.

Futher complicating matters is the fact that Microsoft uses several different versions of its "Object Linking and Embedding (OLE)" formats within its "Compound Document File" archives for a myriad of very different applications. These themselves are usually zipped XML files. All of this adds up to some pretty difficult (and computationally expensive) logic to drill down through multiple binary/compression formats to be able to distinguish a "Publisher" document from, say, a Microsoft Outlook database.

Finally, its important to note that MIME types (technically, IANA media types) are notoriously unreliable markers of "content disposition" for precisely the reasons above, and because many developers don't bother to register their "vendor" (vdn.xxxxxxx) types: although Microsoft has registered a variety of media types, application/vnd.ms-office (which is what file uses for CDF/CDF2 files) is unregistered, as is application/ms-publisher. In fact the mime.types file is never checked by libmagic, and its existance is not at all standard on most platforms. Instead, MIME types are simple hardcoded in "magic" files, and application/vnd.ms-publisher has only recently been added.

Its also important to note that the -i flag's functionality often differs greatly from what's expected, and it operates differently in various versions of file, or may not exist at all. However, in general, it's intent is machine readability and (importantly), in some versions, it changes the heuristics in favor of more machine-relevant results. Therefore, in some versions of file and its database, you'll get a result from file -bi that is simply: application/octet-stream; charset=binary (which is a valid, non-vendor MIME type) instead of application/vnd.ms-office. And if does return application/vnd.ms-office (CDF), it requires the flags -e cdf to effectively "pass through" that result, because (assumedly), is more relevant to another program/algorthm that can read CDF files than the fact that it is a Publisher file. Ignoring the CDF result to keep searching and get any of the ODF2 identifiers and their MIME types is significantly more computationally expensive — the result of application/vnd.ms-office takes six magic test operations, while application/vnd.ms-publisher takes hundreds or thousands (you can see this with the -d flag). Another reason its best to avoid -i: in most newer versions of file, that flag returns the MIME encoding as well as the MIME type, so its best to use --mime-type if you do not want the result to be application/vnd.ms-publisher; charset=binary.

In conclusion, to get the results you want, my recommendation would be to:

  1. Recompile a new version of file from bleeding-edge source, and
  2. Use the flags: file -b -e cdf --mime-type instead of file -bi.

For file to know about it, you need to record the magic number stored at the beginning of the file in /etc/magic. The man page for ‘file’ describes the way that file works, and the man page for ‘magic’ describes the syntax to use in the magic file.

  • I've read those files before I've opened the post, it just does not want to work. – BadSnowflake Aug 2 at 18:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.