I've created a disk image which was zeroed, formatted to ext4 file system, and then had several various files copied to it.
I'm trying to extract these files using a combination of xxd and dd, but with certain files I find their size has increased and cannot be properly interpreted by their default viewer/player.
The reason behind this is because the file system is inserting several almost identical chunks of what I guess is some form of metadata periodically on the image. The data starts off as, or pretty similar to the hex code below, followed by what seems to be a random assortment of characters. Each of these chunks is exactly 8MBs (8388608 bytes) in length and appears 8 times on this 300MB image.
Offset 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
00000000 70 2C 01 00 00 B0 04 00 00 0C 00 00 D9 C3 00 00 p, ° ÙÃ
00000016 59 2C 01 00 01 00 00 00 00 00 00 00 00 00 00 00 Y,
00000032 00 20 00 00 00 20 00 00 E8 07 00 00 19 0F 3E 51 è >Q
00000048 5D 0F 3E 51 01 00 FF FF 53 EF 01 00 01 00 00 00 ] >Q ÿÿSï
00000064 FD 0E 3E 51 00 00 00 00 00 00 00 00 01 00 00 00 ý >Q
00000080 00 00 00 00 0B 00 00 00 80 00 00 00 3C 00 00 00 € <
00000096 42 02 00 00 79 00 00 00 26 09 09 F0 95 77 41 E7 B y & ð•wAç
00000112 81 0F 90 77 76 86 93 C1 00 00 00 00 00 00 00 00 wv†“Á
Is there a command or tool available to allow me to simply ignore these chunks of metadata or allow me to remove them from the file? I've already attempted to extract the non metadata chunks of information out and cat them back together but this seems to waste time as well as causing some of the files to carved incorrectly even if they were carvable beforehand. The code for that looks like this;
inputFile="zeroedImage.ext4
signature="70 2c 01 00 00 b0 04 00"
counter=0
occurance=`xxd -g 1 $inputFile | grep -c "$signature"`
startPosition[0]=0
while [[ $counter < $occurance ]]
do
signatureLocation=`xxd -g 1 -s +${startPosition[$counter]} $inputFile | grep -m 1 "$signature"`
signatureLocationShort=${signatureLocation%$signature*}
signatureByteSpaces=`echo $signatureLocationShort | awk -F " " "{print NF}"`
signatureHexOffset=`echo $signatureLocationShort | cut -f 1 -d ":" | tr [:lower:] [:upper:]`
signatureDecOffset=`echo "ibase=16;$signatureHexOffset" | bc`
endPosition[$counter]=$(($signatureDecOffset-1))
if [[ ${startPosition[$counter]} -gt ${endPosition[$counter]} ]]
then
echo "No space between chunks"
else
difference=$((${endPosition[$counter]}-${startPosition[$counter]}+1))
xxd -g 1 -s +${startPosition[$counter]} -l $difference $inputFile > $counter.temp
cat *.temp > all.temp
fi
counter=$(($counter+1))
startPosition[$counter]=`expr $((${endPosition[$((counter-1))]}+8388609))`
done
xxd -r all.temp > adjustedImage.zeroed
rm *.temp
exit 0
Thank you.