4

I am running Linux version 3.4.8 on an at91sam9g20.

I want to take a large record and split it into multiple files. I have tried a number of methods, but none appear to function correctly, e.g.

tar -c -M --tape-length=102400 --file=disk1.tar mytest.tar.g
z
tar: invalid option -- M
BusyBox v1.20.2 (2012-09-24 16:21:25 CEST) multi-call binary.

Usage: tar -[cxthvO] [-X FILE] [-T FILE] [-f TARFILE] [-C DIR] [FILE]...

Create, extract, or list files from a tar file

Operation:
        c       Create
        x       Extract
        t       List
        f       Name of TARFILE ('-' for stdin/out)
        C       Change to DIR before operation
        v       Verbose
        O       Extract to stdout
        h       Follow symlinks
        exclude File to exclude
        X       File with names to exclude
        T       File with names to include

It appears busybox has a slimmed down version of tar that does not allow certain parameters.

When I try split, I get the following:

/:# split -sh: split: not found

Is there any method of splitting a large file to multiple files using the busybox command set?

Currently defined functions:
        [, [[, addgroup, adduser, ar, arping, ash, awk, basename, blkid,
        bunzip2, bzcat, cat, catv, chattr, chgrp, chmod, chown, chroot, chrt,
        chvt, cksum, clear, cmp, cp, cpio, crond, crontab, cut, date, dc, dd,
        deallocvt, delgroup, deluser, devmem, df, diff, dirname, dmesg, dnsd,
        dnsdomainname, dos2unix, du, dumpkmap, echo, egrep, eject, env,
        ether-wake, expr, false, fdflush, fdformat, fgrep, find, fold, free,
        freeramdisk, fsck, fuser, getopt, getty, grep, gunzip, gzip, halt,
        hdparm, head, hexdump, hostid, hostname, hwclock, id, ifconfig, ifdown,
        ifup, inetd, init, insmod, install, ip, ipaddr, ipcrm, ipcs, iplink,
        iproute, iprule, iptunnel, kill, killall, killall5, klogd, last, less,
        linux32, linux64, linuxrc, ln, loadfont, loadkmap, logger, login,
        logname, losetup, ls, lsattr, lsmod, lsof, lspci, lsusb, lzcat, lzma,
        makedevs, md5sum, mdev, mesg, microcom, mkdir, mkfifo, mknod, mkswap,
        mktemp, modprobe, more, mount, mountpoint, mt, mv, nameif, netstat,
        nice, nohup, nslookup, od, openvt, passwd, patch, pidof, ping,
        pipe_progress, pivot_root, poweroff, printenv, printf, ps, pwd, rdate,
        readlink, readprofile, realpath, reboot, renice, reset, resize, rm,
        rmdir, rmmod, route, run-parts, runlevel, sed, seq, setarch,
        setconsole, setkeycodes, setlogcons, setserial, setsid, sh, sha1sum,
        sha256sum, sha512sum, sleep, sort, start-stop-daemon, strings, stty,
        su, sulogin, swapoff, swapon, switch_root, sync, sysctl, syslogd, tail,
        tar, tee, telnet, test, tftp, time, top, touch, tr, traceroute, true,
        tty, udhcpc, umount, uname, uniq, unix2dos, unlzma, unxz, unzip,
        uptime, usleep, uudecode, uuencode, vconfig, vi, vlock, watch,
        watchdog, wc, wget, which, who, whoami, xargs, xz, xzcat, yes, zcat

2 Answers 2

5

You can use busybox's dd applet with its bs, count and skip arguments to split a large file into chunks.

dd manpage part from busybox:

dd [if=FILE] [of=FILE] [ibs=N] [obs=N] [bs=N] [count=N] [skip=N]
[seek=N] [conv=notrunc|noerror|sync|fsync]

       Copy a file with converting and formatting

               if=FILE         Read from FILE instead of stdin
               of=FILE         Write to FILE instead of stdout
               bs=N            Read and write N bytes at a time
               ibs=N           Read N bytes at a time
               obs=N           Write N bytes at a time
               count=N         Copy only N input blocks
               skip=N          Skip N input blocks
               seek=N          Skip N output blocks
               conv=notrunc    Don't truncate output file
               conv=noerror    Continue after read errors
               conv=sync       Pad blocks with zeros
               conv=fsync      Physically write data out before finishing

So basically you would do something like this:

$ dd if=bigfile of=part.0 bs=1024 count=1024 skip=0
$ dd if=bigfile of=part.1 bs=1024 count=1024 skip=1024
$ dd if=bigfile of=part.2 bs=1024 count=1024 skip=2048

For each part.X file dd writes count * bs bytes ignoring first skip bytes from input file.

A very basic one-liner (combining sed, xargs and dd applet from busybox) could look like this:

seq 0 19 | xargs -n1 sh -c 'dd if=bigfile of=part.$0 bs=1024 count=1024 skip=$(expr $0 \* 1024)'

producing 20 part.X files with at most 1048576 bytes in size.

Example splittig bigfile:

$ ls -l
total 2940
-rw-rw-r-- 1 user user 3000000 Apr 27 13:21 bigfile

$ seq 0 20 | xargs -n1 sh -c 'dd if=bigfile of=part.$0 bs=1024 count=1024 skip=$(expr $0 \* 1024)'
1024+0 records in
1024+0 records out
1024+0 records in
1024+0 records out
881+1 records in
881+1 records out
0+0 records in
0+0 records out
[...]

$ ls -l
total 5968
-rw-rw-r-- 1 user user 3000000 Apr 27 13:21 bigfile
-rw-rw-r-- 1 user user 1048576 Apr 27 13:43 part.0
-rw-rw-r-- 1 user user 1048576 Apr 27 13:43 part.1
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.10
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.11
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.12
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.13
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.14
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.15
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.16
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.17
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.18
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.19
-rw-rw-r-- 1 user user  902848 Apr 27 13:43 part.2
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.3
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.4
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.5
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.6
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.7
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.8
-rw-rw-r-- 1 user user       0 Apr 27 13:43 part.9

Restoring can easily be done with cat (or dd again with the seek parameter). 0 byte files can be skipped:

$ cat part.0 part.1 part.2 > bigfile.res
$ diff bigfile bigfile.res

Depending on your needs you shouldn't use seq and calculate the specific size of your bigfile and do all the stuff in a shell script.

1

I believe, that you can download statically linked busybox binary for your ARM-based microcontroller: http://www.busybox.net/downloads/binaries/latest/

As far as I know, all the binaries provided there include all the utilities which are described here: http://www.busybox.net/downloads/BusyBox.html

So, you'll be able to use split utility in a very simple way.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .