There are many tools for this:
dd
is what you want to use if you want to block a file off - reliably read only a certain number of bytes only a certain number of times. It portably handles blocking and unblocking file streams:
tr -dc '[:graph:]' </dev/urandom |
dd bs=32 count=1 cbs=8 conv=unblock,sync 2>/dev/null
###OUTPUT###
UI(#Q5\e
BKX2?A:Z
RAxGm:qv
t!;/v!)N
I also use tr
above because it can handle converting any ASCII byte to any other (or, in this case, deleting any ASCII byte that isn't a not-space printable character). It's what I used in answer to your other question this morning, in fact, when I did:
tr '>\n' '\n>' | sed 's/^>*//' | tr '\n>' '>\n'
There are many similar. That list should provide a lowest common-denominator subset with which you might become familiar.
But, if I were going to do text processing on 2.5gbs of binary file, I might start with od
. It can give you an octal dump
or any of several other formats. You can specify all kinds of options - but I'll just do one byte per line in a \C
escaped format:
The data you'll get from od
will be regular at whatever interval you specify - as I show below. But first - here's an answer to your question:
printf 'first\nnewline\ttab spacefoobar\0null' |
od -A n -t c -v -w1 |
sed 's/^ \{1,3\}//;s/\\$/&&/;/ /bd
/\\[0nt]/!{H;$!d};{:d
x;s/\n//g}'
That little bit above delimits on \n
ewlines, \0
nulls, \t
abs and <spaces>
while preserving the \C
escaped string for the delimiter. Note the H
and x
functions used - every time sed
encounters a delimiter it swaps out the contents of its memory buffers. In this way sed
only retains as much information as it must to reliably delimit the file and does not succumb to buffer overruns - does not, that is, so long as it actually encounters its delimiters. For so long as it does, sed
will continue to process its input and od
will continue to provide it until it encounters EOF
.
As is, its output looks like this:
first
\nnewline
\ttab
spacefoobar
\0null
So if I want foobar
:
printf ... | od ... | sed ... |
sed 's/foobar/\
&\
/g'
###OUTPUT###
first
\nnewline
\ttab
space
foobar
\0null
Now if you want to make use of the C
escapes it's pretty easy - because sed
has already double \\
backslash escaped all of its single input backslashes, so printf
execed from xargs
will have no issues producing the output to your specification. But xargs
eats shell quotes so you'll need to double quote it again:
printf 'nl\ntab\tspace foobarfoobar\0null' |
PIPELINE |
sed 's/./\\&/g' |
xargs printf %b |
cat -A
###OUTPUT###
nl$
tab^Ispace $
foobar$
$
foobar$
^@null%
That could have as easily been saved to a shell variable and output later in identical fashion. The last sed
inserts a \
backslash before every character in its input, and that's all.
And here's what it all looks like before ever sed
gets hold of it:
printf 'nl\ntab\tspace foobarfoobar\0null' |
od -A n -t c -v -w1
n
l
\n
t
a
b
\t
s
p
a
c
e
f
o
o
b
a
r
f
o
o
b
a
r
\0
n
u
l
l
perl
orpython
? – iruvar Jun 16 '14 at 14:44gsar
(home.online.no/~tjaberg) which I will try. – MattBianco Jun 16 '14 at 14:49