6

I occasionally get tarballs where the filenames are encoded in ISO-8859-1 or some other pre-Unicode scheme. My system uses UTF-8, so when I untar these archives with the usual options (tar xvf foo.tar) I end up with a directory full of mojibake filenames.

Until now I've been using convmv to convert the filenames to UTF-8 after they've been extracted. This is a bit incovenient, because I either need to invoke convmv on each affected file, or else untar the file into a new directory, run convmv on the entire directory, and then move the files to where I wanted them originally. Short of coding this functionality into a shell script, is there some way of converting the archived filenames to UTF-8 on the fly, as they are being untarred?

9

Here is a little tar file extractor that modifies the names in memory before extracting:

#!/usr/bin/python27

import tarfile

def transform(data):
    u = data.decode('latin1')
    return u.encode('utf8')

tar = tarfile.open('archive.tar')
for m in tar.getmembers():
    m.name = transform(m.name)

tar.extractall()

Warning: unlike GNU tar, this extractor is not stripping leading /. Either add checking logic to this extractor or check each tar file before extracting with tar -t.

  • This is OK as long as you don't need to pass any additional options to tar. I don't suppose there's any way of generalizing your solution so that it supports the same options as GNU tar…? – Psychonaut Sep 9 '15 at 7:48
  • @Psychonaut Compression is handled by tarfile transparently. Options that handle on the fly manipulation (change UID, change paths) can be done in the same way as in the example. Filtering (extract only a subset) is also easy: create a list of all member, filter them, iterate over the list with tar.extract(). So it's all doable, just work. However, as GNU tar has tons of options, some of them really obscure, a full compatible version is not worth the effortd. – Thomas Erker Sep 9 '15 at 10:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.