csplit not recognizing provided regexp

Question

I'm working on this big file (DATA.DAT, ~900MB) which contains several other files. It's from a PS2 game.

Sound samples (which are in .AIFF format), precisely what I'm after, make up most of its size.

After searching the web for PS2 .DAT extractors I found out that they're basically developer dependent and since this game/tool is rather obscure and not finding much about it online, I thought about automating the process myself.

Inspecting the file on a hex editor I came across some .AIFF headers, cloned the chunks to new .AIFF files and without any further work, they were playable.

Having spent a while getting the rust out of my VERY limited bash knowledge and having read similar questions here, I came up with this expression:

gcsplit -f "sample-" -b "%04d.aif" DATA.DAT /FORM/ '{*}'

(I'm on OSX using coreutils, hence the g- prefix on csplit)

Given that .AIFF files start with the string "FORM" and given that basically all samples in the file are next to each other (spaced apart by disregardable amounts of data that won't generate unwanted end noise on the samples), I thought that the regexp

/FORM/

would suffice to split the files up.

However, every split file is being output with junk data that sits in between sound samples before the .AIFF header, rendering it unplayable.

Screenshots of the hex data of a split sound sample below:

This actual sample begins roughly around the 1500 bytes mark:

What's making this expression split the files with an offset?

Answer 1

Csplit is a text utility. It is line-based. A pattern /FORM/ means “a line containing FORM”. A line is a sequence of bytes other than LF (line feed, also known as newline, which can be written \n, ^J, …), followed by an LF byte (or by the end of the file, with GNU utilities). Thus the “junk” that you observe is whatever is between the previous LF character and the FORM substring.

The man page and the --help short description assume that you already know what the command does, so they just mention “pieces” without explanation. You need to read the full documentation to get a description of what the pieces are.

You can't do what you want with csplit. You can do it with GNU awk. (Other versions of awk may not have the requisite features — support of arbitrary record separators and coping with null bytes.) Untested:

gawk -v RS='FORM' -v ORS='' '{
    print "FORM" $0 >sprintf("sample-%04d.aif", n++)
}' DATA.DAT

But this may cut at spurious places if the compressed data just happens to contain the four bytes FORM. This may be good enough for one-time operation with manual review, but you would be better off with a format-aware tool if you need something reliable.

Answer 2

A text-based utility is not appropriate for manipulating binary files.

You're likely to obtain better results with Lib/aifc, PySoundFile, or the ffmpeg command line app.