Store a custom serial number on a USB flash drive, immune to overwriting

Question

I'm working with massive amounts of USB flash memory. I'd like to be able to track events regarding the USB drives. Specifically: I perform f3 tests, format the drives and write data to them.

I'm looking for a way to mark the drives digitally (for example with a UUID) so I can perform various actions with the drive while having the serial numbers intact. Hopefully also have other people use and format the drives, still allowing me to read the serial number off of them later.

I have tried using the serial number reported by the USB device, but I've soon have found out that the majority of the drives have no uniquely identifying information stored in them.

Hence - I have tired generating a UUID concatenated with it's own CRC sum so I can write that string to the drive and read it back, verifying if it was stored correctly.

Writing the serial number to the filesystem however means that once the drive is filled up the every copy of the serial number will be overwritten.

My latest idea is to partition the drive to leave 1 MB of unpartitioned empty space so I can store the serial number there.

This is going to have a marginal impact on a drive's usefullness but should allow me to somewhat reliably identify drives, as long as the drive will not be repartitioned.

Ideally each drive would have a UUID written to the Mass Storage driver chip and would report that - but I don't manufacture the drives.

I thought about generating the serial numbers a bit like this (Bash):

UUID=$(uuidgen)
CK=$(echo "$UUID" | cksum | cut -c -8)
SN="SN:$UUID:$CK;"
echo $SN

This generates a UUID, calculates a CRC sum of that UUID, leaves just 8 first characters of that sum (because the length can vary and we need a constant length to effectively search for the pattern later). Then it concatenates that with some markers that can be used to extract the number form a data stream and verify if it wasn't damaged.

Example serial numbers:

SN:7199e42e-465b-4baf-96b6-a109247d3f4b:39272476;
SN:aa795142-fa2e-4929-b93b-e991f95a74cd:40619286;
SN:43f93702-eda9-4089-9a81-d21172e051bf:37174302;

I can then verify the serial number integrity like this:

SN="SN:7199e42e-465b-4baf-96b6-a109247d3f4b:39272476;"
UUID=$(echo "$SN" | cut -d':' -f2)
CK=$(echo "$SN" | cut -d':' -f3 | cut -d';' -f1)
if [[ $(echo "$UUID" | cksum | cut -c -8) == $CK ]]; then
    echo "SN is fine"
else echo "SN is corrupted"; fi

I can also extract a serial number from a drive with grep:

cat /dev/sdX | grep -E "SN:.{8}-.{4}-.{4}-.{4}-.{12}:.{8};" -o

Then I'd get a list of serial numbers, and evaluate them until I find one that has a matching checksum.

Because I work with potentially faulty drives I need to write the serial number in multiple copies so at least one copy will survive.

I wonder if there's a better approach to what I'm trying to do - without of course resorting to manufacturing my own flash memory,

EDIT: UDEVADM

I can try to get a unique identifier using various read-only information from the drive itself:

$ udevadm info /dev/sdd | grep -e "ID_MODEL" -e "ID_SERIAL" -e "ID_VENDOR"
E: ID_MODEL=Flash_Disk
E: ID_MODEL_ENC=Flash\x20Disk\x20\x20\x20\x20\x20\x20
E: ID_MODEL_ID=6387
E: ID_SERIAL=Generic_Flash_Disk_97C06F44-0:0
E: ID_SERIAL_SHORT=97C06F44
E: ID_VENDOR=Generic
E: ID_VENDOR_ENC=Generic\x20
E: ID_VENDOR_ID=058f

$ udevadm info /dev/sdd | grep -e "ID_MODEL" -e "ID_SERIAL" -e "ID_VENDOR" | md5sum | cut -d' ' -f1
7b7f63b829b6301cdf35d71eed9bb707

I am almost certain however that I'll encounter a lot of drives that return identical values here - making this approach useless.

EDIT2:

I have accumulated nearly 1600 datapoints on this, and most drives seem to produce unique hashes, but I got still around 400 drives that returned identical hashes, so this method alone is not enough.

Answer 1

The ID_SERIAL of the device usually does the trick for verifying the identity of a device.

It is persistent across formatting because it's a property of the device itself; however, afaik, you cannot change it due to the same reason.

You can get it by running: udevadm info /dev/sdX | grep ID_SERIAL=

Answer 2

Persistent device naming (a name that is the same across diferent mount points) is an elusive goal. There are several UUIDs created in several containers (filesystem, mount point, etc) as described in detail here.

An UUID (well, actually a World Wide Identifier (WWID)) given by the hardware of the USB device connected may be given in:

ls -l /dev/disk/by-id/

But understand that not all devices support such numbering.

Answer 3

Some flash storage is not uniquely identifiable after a format and the best you can do is set custom partition UUIDs or upgrade to better storage.

https://superuser.com/questions/1247972/change-uuid-of-vfat-partition