2

I need to store a large number of files (> 100,000) in a tar archive and have run into the error: /bin/sh: /bin/tar: cannot execute [Arg list too long].

Are there limits with UNIX Solaris tar or GNU tar? Or, is this a ZFS issue? I am running Solaris 11.3 with ZFS.

  • 1
    How do you make the archive? – NickD Nov 20 '17 at 1:35
  • 2
    Appearances are a limit to tar’s argument list, not contents. – Jeff Schaller Nov 20 '17 at 1:38
  • As the error seems to be related to the number of arguments that have been passed on the command line, I think it would be useful to see what was the command you used for this purpose. – Scantlight Nov 20 '17 at 1:57
  • The error message is from /bin/sh (failing on execve), not from tar (as you can guess). If you replace tar by ls -l you'll get the same error – Basile Starynkevitch Nov 20 '17 at 12:01
5

This is not a limitation of tar. You are exceeding the maximum size of arguments that can be passed to a program. If the list of file names is available in a file, then you can use GNU tar's -T option:

tar -c -f output.tar -T filelist.txt

Tar can also receive the file list from standard input by using - as the file name. For example, if you want to create an archive containing all files and subdirectories of mydir, you can pass the output of find to tar:

find mydir | tar -c -f output.tar -T -
  • 1
    You are exceeding the maximum size of arguments that can be passed to a program. To be specific, that's likely a sh limit. csh has a much larger limit. Running csh<enter>host% tar -cvf tarfile.tar * can work when running the same command under sh fails with an arg list is too long error. – Andrew Henle Nov 20 '17 at 12:58
  • I've usually had the issue with the csh and better luck with the sh shell for this. But in this case I think creating an include file with the -I option would work better than passing the files as args. – sleepyweasel Nov 20 '17 at 19:04
  • Please read the tar man page before answerig questions, the -T option switches on Trusted Extensions and this is definitely not what you like. You probably ment the -I option...or did you refer to GNU tar? Then you should know that the official name for GNU tar is gtar, – schily Jun 28 '18 at 7:48
1

As others have said, the issue is (probably) the length of the command and not the size of the archive.  You should be able to do

tar cf tarfile file001 file002 … file100
tar uf tarfile file101 file102 … file200
tar uf tarfile file201 file202 … file300
                  ︙
Alternatively, do
tar cf tarfile file001
to create the archive, and then
tar cf tarfile         file002 … file100
tar uf tarfile file101 file102 … file200
tar uf tarfile file201 file202 … file300
                  ︙
(possibly with xargs) to add to it.

  • Just be aware that updating a tar file with the number of entries in question can be excruciatingly slow. – Andrew Henle Nov 27 '17 at 17:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.