2

I am running Linux version 3.4.8 on an at91sam9g20.

I want to take a large record and split it into multiple files. I have tried a number of methods, but none appear to function correctly, e.g.

tar -c -M --tape-length=102400 --file=disk1.tar mytest.tar.g
z
tar: invalid option -- M
BusyBox v1.20.2 (2012-09-24 16:21:25 CEST) multi-call binary.

Usage: tar -[cxthvO] [-X FILE] [-T FILE] [-f TARFILE] [-C DIR] [FILE]...

Create, extract, or list files from a tar file

Operation:
        c       Create
        x       Extract
        t       List
        f       Name of TARFILE ('-' for stdin/out)
        C       Change to DIR before operation
        v       Verbose
        O       Extract to stdout
        h       Follow symlinks
        exclude File to exclude
        X       File with names to exclude
        T       File with names to include

It appears busybox has a slimmed down version of tar that does not allow certain parameters.

When I try split, I get the following:

/:# split -sh: split: not found

Is there any method of splitting a large file to multiple files using the busybox command set?

Currently defined functions:
        [, [[, addgroup, adduser, ar, arping, ash, awk, basename, blkid,
        bunzip2, bzcat, cat, catv, chattr, chgrp, chmod, chown, chroot, chrt,
        chvt, cksum, clear, cmp, cp, cpio, crond, crontab, cut, date, dc, dd,
        deallocvt, delgroup, deluser, devmem, df, diff, dirname, dmesg, dnsd,
        dnsdomainname, dos2unix, du, dumpkmap, echo, egrep, eject, env,
        ether-wake, expr, false, fdflush, fdformat, fgrep, find, fold, free,
        freeramdisk, fsck, fuser, getopt, getty, grep, gunzip, gzip, halt,
        hdparm, head, hexdump, hostid, hostname, hwclock, id, ifconfig, ifdown,
        ifup, inetd, init, insmod, install, ip, ipaddr, ipcrm, ipcs, iplink,
        iproute, iprule, iptunnel, kill, killall, killall5, klogd, last, less,
        linux32, linux64, linuxrc, ln, loadfont, loadkmap, logger, login,
        logname, losetup, ls, lsattr, lsmod, lsof, lspci, lsusb, lzcat, lzma,
        makedevs, md5sum, mdev, mesg, microcom, mkdir, mkfifo, mknod, mkswap,
        mktemp, modprobe, more, mount, mountpoint, mt, mv, nameif, netstat,
        nice, nohup, nslookup, od, openvt, passwd, patch, pidof, ping,
        pipe_progress, pivot_root, poweroff, printenv, printf, ps, pwd, rdate,
        readlink, readprofile, realpath, reboot, renice, reset, resize, rm,
        rmdir, rmmod, route, run-parts, runlevel, sed, seq, setarch,
        setconsole, setkeycodes, setlogcons, setserial, setsid, sh, sha1sum,
        sha256sum, sha512sum, sleep, sort, start-stop-daemon, strings, stty,
        su, sulogin, swapoff, swapon, switch_root, sync, sysctl, syslogd, tail,
        tar, tee, telnet, test, tftp, time, top, touch, tr, traceroute, true,
        tty, udhcpc, umount, uname, uniq, unix2dos, unlzma, unxz, unzip,
        uptime, usleep, uudecode, uuencode, vconfig, vi, vlock, watch,
        watchdog, wc, wget, which, who, whoami, xargs, xz, xzcat, yes, zcat
4

You can use busybox's dd applet with its bs, count and skip arguments to split a large file into chunks.

dd manpage part from busybox:

dd [if=FILE] [of=FILE] [ibs=N] [obs=N] [bs=N] [count=N] [skip=N]
[seek=N] [conv=notrunc|noerror|sync|fsync]

       Copy a file with converting and formatting

               if=FILE         Read from FILE instead of stdin
               of=FILE         Write to FILE instead of stdout
               bs=N            Read and write N bytes at a time
               ibs=N           Read N bytes at a time
               obs=N           Write N bytes at a time
               count=N         Copy only N input blocks
               skip=N          Skip N input blocks
               seek=N          Skip N output blocks
               conv=notrunc    Don't truncate output file
               conv=noerror    Continue after read errors
               conv=sync       Pad blocks with zeros
               conv=fsync      Physically write data out before finishing

So basically you would do something like this:

$ dd if=bigfile of=part.0 bs=1024 count=1024 skip=0
$ dd if=bigfile of=part.1 bs=1024 count=1024 skip=1024
$ dd if=bigfile of=part.2 bs=1024 count=1024 skip=2048

For each part.X file dd writes count * bs bytes ignoring first skip bytes from input file.

A very basic one-liner (combining sed, xargs and dd applet from busybox) could look like this:

seq 0 19 | xargs -n1 sh -c 'dd if=bigfile of=part.$0 bs=1024 count=1024 skip=$(expr $0 \* 1024)'

producing 20 part.X files with at most 1048576 bytes in size.

Example splittig bigfile:

$ ls -l
total 2940
-rw-rw-r-- 1 user user 3000000 Apr 27 13:21 bigfile

$ seq 0 20 | xargs -n1 sh -c 'dd if=bigfile of=part.$0 bs=1024 count=1024 skip=$(expr $0 \* 1024)'
1024+0 records in
1024+0 records out
1024+0 records in
1024+0 records out
881+1 records in
881+1 records out
0+0 records in
0+0 records out
[...]

$ ls -l
total 5968
-rw-rw-r-- 1 user user 3000000 Apr 27 13:21 bigfile
-rw-rw-r-- 1 user user 1048576 Apr 27 13:43 part.0
-rw-rw-r-- 1 user user 1048576 Apr 27 13:43 part.1
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.10
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.11
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.12
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.13
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.14
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.15
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.16
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.17
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.18
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.19
-rw-rw-r-- 1 user user  902848 Apr 27 13:43 part.2
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.3
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.4
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.5
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.6
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.7
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.8
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.9

Restoring can easily be done with cat (or dd again with the seek parameter). 0 byte files can be skipped:

$ cat part.0 part.1 part.2 > bigfile.res
$ diff bigfile bigfile.res

Depending on your needs you shouldn't use seq and calculate the specific size of your bigfile and do all the stuff in a shell script.

1

I believe, that you can download statically linked busybox binary for your ARM-based microcontroller: http://www.busybox.net/downloads/binaries/latest/

As far as I know, all the binaries provided there include all the utilities which are described here: http://www.busybox.net/downloads/BusyBox.html

So, you'll be able to use split utility in a very simple way.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.