Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I've created a disk image which was zeroed, formatted to ext4 file system, and then had several various files copied to it.

I'm trying to extract these files using a combination of xxd and dd, but with certain files I find their size has increased and cannot be properly interpreted by their default viewer/player.

The reason behind this is because the file system is inserting several almost identical chunks of what I guess is some form of metadata periodically on the image. The data starts off as, or pretty similar to the hex code below, followed by what seems to be a random assortment of characters. Each of these chunks is exactly 8MBs (8388608 bytes) in length and appears 8 times on this 300MB image.

Offset      0  1  2  3  4  5  6  7   8  9 10 11 12 13 14 15

00000000   70 2C 01 00 00 B0 04 00  00 0C 00 00 D9 C3 00 00   p,   °      ÙÃ  
00000016   59 2C 01 00 01 00 00 00  00 00 00 00 00 00 00 00   Y,              
00000032   00 20 00 00 00 20 00 00  E8 07 00 00 19 0F 3E 51           è     >Q
00000048   5D 0F 3E 51 01 00 FF FF  53 EF 01 00 01 00 00 00   ] >Q  ÿÿSï      
00000064   FD 0E 3E 51 00 00 00 00  00 00 00 00 01 00 00 00   ý >Q            
00000080   00 00 00 00 0B 00 00 00  80 00 00 00 3C 00 00 00           €   <   
00000096   42 02 00 00 79 00 00 00  26 09 09 F0 95 77 41 E7   B   y   &  ð•wAç
00000112   81 0F 90 77 76 86 93 C1  00 00 00 00 00 00 00 00      wv†“Á   

Is there a command or tool available to allow me to simply ignore these chunks of metadata or allow me to remove them from the file? I've already attempted to extract the non metadata chunks of information out and cat them back together but this seems to waste time as well as causing some of the files to carved incorrectly even if they were carvable beforehand. The code for that looks like this;

inputFile="zeroedImage.ext4
signature="70 2c 01 00 00 b0 04 00"
counter=0
occurance=`xxd -g 1 $inputFile | grep -c "$signature"`
startPosition[0]=0

while [[ $counter < $occurance ]]
do
    signatureLocation=`xxd -g 1 -s +${startPosition[$counter]} $inputFile | grep -m 1 "$signature"`
    signatureLocationShort=${signatureLocation%$signature*}
    signatureByteSpaces=`echo $signatureLocationShort | awk -F " " "{print NF}"`
    signatureHexOffset=`echo $signatureLocationShort | cut -f 1 -d ":" | tr [:lower:] [:upper:]`
    signatureDecOffset=`echo "ibase=16;$signatureHexOffset" | bc`


    endPosition[$counter]=$(($signatureDecOffset-1))

    if [[ ${startPosition[$counter]} -gt ${endPosition[$counter]} ]]
    then
        echo "No space between chunks"
    else
        difference=$((${endPosition[$counter]}-${startPosition[$counter]}+1))
        xxd -g 1 -s +${startPosition[$counter]} -l $difference $inputFile > $counter.temp
        cat *.temp > all.temp

    fi

    counter=$(($counter+1))
    startPosition[$counter]=`expr $((${endPosition[$((counter-1))]}+8388609))`
done

xxd -r all.temp > adjustedImage.zeroed
rm *.temp
exit 0

Thank you.

share|improve this question
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.