9

I would like to use Debian wherever I go by installing it on a USB flash drive, but I'm not sure how to make it save the settings and be able to install/update packages without loosing them on reboots.

So I downloaded a Hybrid ISO image from this link, and I followed the instructions in the FAQ:

dd if=image.iso of=/dev/sdb bs=4M; sync

This command copied the image to the flash drive by creating a partition which size is the same as the ISO image (1.3 GB out of 8GB), and the rest of the flash drive is unallocated.

I searched on how to do this, but every tutorial uses a different approach and some of them are outdated and talk about the old usb-hdd image.

So how should I install this hybrid image on the flash drive ?

How should I partition my flash drive to be able to install packages and save settings ? and how can I install this image without using dd ?

  • This dated tutorial (2011) described exactly how the result supposed to be in GParted; this was possible with Debian 6.0.0 (Squeeze) using a USB-HDD image. Then again, I can't seem to reproduce the result by using a hybrid ISO image for newer releases of Debian. – clearkimura Sep 1 at 5:54
  • @clearkimura Try UEFI + BIOS bootable live Debian stretch amd64 with persistence. You should award the bounty to user schlimmchen if the answer works for you. – Freddy Sep 1 at 8:20
  • Hybrid ISOs are images bootable from CD-ROM and bootable from USB devices as they also contain an MBR. These images are read-only (squashfs filesystem) and thus cannot be written to and they usually have one partition. So you will have to create a second writable partition for persistence, add a persistence.conf configuration file and add a "persistence" kernel boot parameter to your grub / syslinux config to detect your second partition. – Freddy Sep 2 at 13:03
  • @Freddy If you think you can provide a better explanation than posted answers so far--to explain why cloning (using dd or cp) will not work with persistence as described in the official documentation--you should convert your comment into a proper answer instead. – clearkimura Sep 2 at 15:54
  • 1
    Post-bounty homework: After I continue to read, read, and read various text on the web, and did another few rounds of trial and error, I managed to understand better of cloning (Yes, hybrid ISO can work with persistent on USB as per official documentation). Yet to download and test using Debian 10--I may reach my own answer sometime soon. – clearkimura Sep 6 at 9:50
5

Debian live with persistence.

  1. First try with official image from www.debian.org/CD/live/

    From SE site (standard live):

    wget https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/debian-live-10.0.0-amd64-standard.iso
    

    From automatic mirror selection (mate live):

    wget http://debian-cd.debian.net/debian-cd/10.0.0-live/amd64/iso-hybrid/debian-live-10.0.0-amd64-mate.iso
    

    Then checksum you download file with https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/SHA256SUMS...

    Ok ISO filesystem is read-only, but there is a little workaround: we could replace non vital bootparam by persistence in this way.

    • Once file validated !
    • you could alter them by using sed for replacing strings on binarie.

      LANG=C sed 's/splash quiet/persistence /;s/quiet splash/persistence /' \
          </tmp/debian-live-10.0.0-amd64-mate.iso \
          >/tmp/debian-live-10.0.0-amd64-mate-persist.iso
      

    This will create a modified copy of your live binary file, by strictly replacing splash quiet or quiet splash by persistence, everywhere. Ok this will work only while grub boot command do contain this two words together.

    But care to not miss the space after persistence:

    "splash quiet" -> 12 characters
    "persistence " -> 12 characters
    

    Or your binarie will be broken.

  2. Install on USB key

    dd if=debian-live-10.0.0-amd64-standard-persist.iso of=/dev/sdX
    

    Then add your third partition for persistence:

    fdisk /dev/sdX
    n           # new partition
    p           # primary
    <Return>    # default: 3
    <Return>    # default: next free sector
    <Return>    # default: last addressable sector
    w           # write and quit
    

    This could be run without interaction:

    fdisk /dev/sdX <<<$'n\np\n\n\n\nw'
    

    Format and prepare persistence with union:

    mkfs.ext4 -L persistence /dev/sdX3
    mount /dev/sdX3 /mnt
    echo '/ union' >/mnt/persistence.conf
    sync
    umount /mnt
    

    Then eject and try!

    If you use official, unmodifiel image, for using persistence, you have to interrupt boot selection:

    Once menu screen is displayed, choose your boot option, then instead of Return, hit Tab.

    The kernel commandline will be displayed, then add persistence with a space, after last word (quiet), then hit Return.

    Unfortunately, as 1st partition is bundled with UEFI and is ISO, you can't the modify boot command.

Customized Debian live with persistence

you just have to add persistence to boot command line, but nothing else!? There is a way, using FAT and syslinux, but you have a lot of data manipulations. It's long and I find this not so well. I prefer:

  1. Build your own Debian live

    More regular, but a little longer (at least for 1st image),

    Note: All this stuf was done under root user (this must work by using fakeroot, but this is not tested there and today).

    apt install live-build
    

    ... and all recommendations.

    I wrote a little XARGS function for dropping commented lines:

    XARGS() { sed -ne '/#/d;s/ \t//g;H;${x;s/\n/ /g;s/^ //;p}'; }
    

    First setting bootparams, with localisation and arguments for persistence:

    ExtraBootParams=$(XARGS <<eobp
        boot=live
        config
        locales=ed_WT
        keyboard-layouts=ed
        keyboard-variant=wt
        persistence
    eobp
    )
    

    Now your package list:

    PackageList=$(XARGS <<-eopl
        gnome
        gnome-core
        # gnome-full
        # debian-forensics
        debian-installer-launcher
    eopl
    )
    

    Very first step of lb: create initial tree:

    lb config --architectures amd64 -d buster --debian-installer-gui \
        true --archive-areas 'main contrib non-free' \
        --bootappend-live "$ExtraBootParams" 
    

    Now, you have a small tree, you could:

    printf "%s\n" > config/package-lists/standard.list.chroot \
        $Packages $PackageList
    

    Ok, next command will take a loooong time! (Approx 1 hour on my host)

    lb build
    

    If everything's ok, you may find your own Debian live:

    ls -l *.iso
    -rw-r--r--  1 root root 1511817216 sep  7 15:32 live-image-amd64.hybrid.iso
    
  2. Install on USB key (same operation than for downloaded binaries)

    The 'iso-hybrid' image contains two partitions for UEFI and live mixed in a way both EFI and bios could boot on.

    file live-image-amd64.hybrid.iso
    live-image-amd64.hybrid.iso: DOS/MBR boot sector; partition 2 : ID
    =0xef, start-CHS (0x3ff,254,63), end-CHS (0x3ff,254,63), startsect
    or 708, 5696 sectors
    

    You could simply put to your USB Key: (Note: ensure first your USB Key is not mounted!)

    dd if=live-image-amd64.hybrid.iso of=/dev/sdX
    

    Then add your third partition for persistence:

    fdisk /dev/sdX <<<$'n\np\n\n\n\nw'
    

    Format and prepare persistence with union:

    mkfs.ext4 -L persistence /dev/sdX3
    mount /dev/sdX3 /mnt
    echo '/ union' >/mnt/persistence.conf
    sync
    umount /mnt
    

    Eject and try...

Debian live with encrypted persistence

  1. Build your own Debian live, but with encrypted persistence.

    In order to be able to boot with rootfs / on encrypted persistence, you have to add dm-crypt module and related binaries to initrd (initial ram disk) by adding setting CRYPTSETUP=y into an /etc/initramfs-tools/hooks/...

    I wrote a little XARGS function for dropping commented lines:

    XARGS() { sed -ne '/#/d;s/ \t//g;H;${x;s/\n/ /g;s/^ //;p}'; }
    

    First setting bootparams, with localisation and arguments for persistence and cryptsetup:

    ExtraBootParams=$(XARGS <<eobp
        boot=live
        config
        locales=ed_WT
        keyboard-layouts=ed
        keyboard-variant=wt
        persistent=cryptsetup
        persistence-encryption=luks
        persistence
    eobp
    )
    

    Now your package list:

    PackageList=$(XARGS <<-eopl
        gnome
        gnome-core
        # gnome-full
        # debian-forensics
        debian-installer-launcher
    eopl
    )
    

    And your package selection:

    Packages=$(XARGS <<-eopk
        cryptsetup
        cryptsetup-initramfs
        debian-installer-launcher
        firmware-linux-nonfree
        firmware-linux-free
        less
        ssh
    #   openvpn
    #   xtightvncviewer
        gsmartcontrol
        smartmontools
        partclone
        ntfs-3g
        task-gnome-desktop
        user-setup
        sudo
        apt-utils
    eopk
    )
    

    Of course cryptsetup is required! ;-)

    Very first step of lb: create initial tree:

    lb config --architectures amd64 -d buster --debian-installer-gui \
        true --archive-areas 'main contrib non-free' \
        --bootappend-live "$ExtraBootParams" 
    

    Now, you have a small tree, you could:

    printf "%s\n" > config/package-lists/standard.list.chroot \
        $Packages $PackageList
    

    Ok, next two command will take a loooong time! (Approx 40' on my host)

    lb bootstrap ; lb chroot
    

    Now you could add your module and binaries:

    echo dm-crypt >> chroot/etc/initramfs-tools/modules
    
    sed '/CRYPTSETUP=/s/^#//;s/=.*/=y/' -i \
        chroot/etc/cryptsetup-initramfs/conf-hook 
    ln -s ../../cryptsetup-initramfs/conf-hook \
        chroot/etc/initramfs-tools/hooks/cryptsetup
    chroot chroot live-update-initramfs -u
    

    Then run final stage (will take some more lot of time ~25'):

    lb installer ; lb binary
    

    Note: If you read cryptsetup: WARNING: Couldn't determine root device, it's fine! This mean that cryptsetup is installed on your initrd.

    If everything's ok, you may find your own Debian live:

    ls -l *.iso
    -rw-r--r--  1 root root 1511817216 sep  7 15:32 live-image-amd64.hybrid.iso
    
  2. Install on USB key

    The 'iso-hybrid' umage do contain already two partitions for UEFI and live mixed in a way both EFI and BIOS could boot on.

    file live-image-amd64.hybrid.iso
    live-image-amd64.hybrid.iso: DOS/MBR boot sector; partition 2 : ID
    =0xef, start-CHS (0x3ff,254,63), end-CHS (0x3ff,254,63), startsect
    or 708, 5696 sectors
    

    You could simply put to your USB Key: (Note: ensure first your USB Key is not mounted!)

    dd if=live-image-amd64.hybrid.iso of=/dev/sdX
    

    Add new Linux partion by using free space.

    fdisk /dev/sdX <<<$'n\np\n\n\n\nw'
    

    This will create partition 3 using free space on your USB Key.

    Then prepare your crypted partition

    cryptsetup -q luksFormat /dev/sdX3
    

    Enter passphrase

    cryptsetup -q luksOpen /dev/sdX3 persist 
    

    Enter passphrase again

    mkfs.ext4 -L persistence /dev/mapper/persist
    mount /dev/mapper/persist /mnt
    echo '/ union' >/mnt/persistence.conf
    sync
    umount /mnt
    cryptsetup luksClose persist
    

That's all.

eject /dev/sdX
  • This is interesting: 1. Are there any bin files available for the current versions of Debian? 2. Please let us know, if and how your method works (with the current versions of Debian). – sudodus Sep 2 at 13:09
  • 1
    @sudodus Since Debian 7 (Wheezy), the USB-HDD images were no longer built for release. I can at least confirm that the USB-HDD images are available for Debian 5 (Lenny) and 6 (Squeeze). The USB-HDD images are .img and not .bin, but I guess these are essentially the same as per Debian Wiki. – clearkimura Sep 2 at 15:26
  • 2
    Ok, I will rewrite all this, but not today... Sorry. – F. Hauri Sep 3 at 5:40
  • @sudodus Rewite finished, and no, sorry: bin image published on debian.org don't boot with crypted persistence. – F. Hauri Sep 7 at 14:08
  • 1
    +1; @F.Hauri, I have tried to create the partition for persistence after the cloned part of a USB drive using parted and gparted without much luck. You show that fdisk works (does not destroy the booting of the cloned drive). And yes, I see that it will be necessary to add the boot option persistence at every boot because the file system iso9660 is read-only by design. And I agree, it can be cool (even very valuable) to have encrypted persistence for example while travelling. – sudodus Sep 8 at 2:52
4

Why don't you make a new installation of Debian choosing the usb device instead your HDD as the destination? What capacity does your usb have?

I have a Debian LXDE installed and runing in a 4GB usb memory stick, updating the system whenever I want and installing packages when I need them.

  • It's an 8GB flash drive, I want to use GNOME, and thought that it would take a lot of space, and the GNOME live image is 1.3 GB only, but my HDD install is larger but I don't know exactly how much, which installation image did you use ? – Pierre Mar 10 '14 at 12:15
  • @Peter - 8GB should be fine for a default install. – slm Mar 10 '14 at 13:00
  • I used cdimage.debian.org/debian-cd/current-live/i386/iso-hybrid/…, since I wanted speed I picked lxde. You have a gnome image there too. Gnome from the usb may be a little bit slow, but if you have plenty of RAM, it will only be slow opening the apps the first time, next time they will open faster. Regarding space, obviously you will have reduce the number of apps you install in the usb to the ones you really use/need, and leave the rest for your home computer. – YoMismo Mar 10 '14 at 14:09
  • 2
    By the way, you may want to create your /tmp on your ram instead on your disk, you will only need to add the line tmpfs /tmp tmpfs nodev,nosuid,size=2G 0 0 to your fstab (change 2G for the size you want it to be and have in mind that the amount you assign to tmp won't be available for the rest of the apps) – YoMismo Mar 10 '14 at 14:13
  • 2
    Using installed system is not suitable to USB key who have to boot on many different hardware. A real live is build in a way to permit multiple hardwares – F. Hauri Sep 2 at 8:12
3
+50

Cloning and Debian persistent live

Cloning (copying each byte as it is, creating a one-to-one copy) is a very simple and reliable method to create a boot drive (live drive or install drive) from a linux hybrid iso file.

There is no conventional way to make a cloned Debian iso file persistent, because it has a read-only file system, ISO9660, (and read-only 'partition table').

Using mkusb (BIOS/UEFI)

It is possible (and I would say rather simple) to use mkusb to create a persistent live drive from a Debian live iso file. mkusb supports Debian 8 or newer, and you should use the current version of mkusb (version 12.3.2 or newer).

mkusb creates a partition table and 5 partitions:

  1. NTFS partition for exchange of data with Windows computers (optional size)
  2. Extended partition or bios_grub partition
  3. FAT32 boot partition for booting with grub (both in UEFI and BIOS mode)
  4. ISO9660 partition containing clone of the iso file
  5. ext partition for persistence, where your modifications (and files) are stored

mkusb does the following tweaks automatically:

  • The boot option persistence (not persistent as in Ubuntu)
  • The label persistence on partition #5 (not casper-rw as in Ubuntu)
  • The file persistence.conf with the content / union at the top level of partition #5.

This structure is created by the bash shellscript dus-persistent, when you use mkusb version 12, alias mkusb-dus. If you want all the details, install mkusb and read the content of dus-persistent, or read it directly via phillw.net/isos/linux-tools/mkusb/dus-persistent.

The default settings of mkusb can be used for Debian 8 and 9, and the persistent live drive will work when booted in BIOS mode and UEFI mode.

But for Debian 10 the default settings work only in BIOS mode. Select the setting usb-pack-efi in mkusb. Then also a persistent live drive with Debian 10 will boot both in BIOS mode and UEFI mode (but not with secure boot).

I intend to look into this problem and fix the bug, but there is no time to do it within the near future.

enter image description here

Using manual setup (UEFI only)

If you want an UEFI only boot flash drive you do not need an installer at all.

You just need to format flash drive with FAT32 and set boot flag on. Then use whatever is your favorite extraction tool like 7zip to extract & copy ISO to FAT32 partition. See Do it yourself for more details.

I verified this method with the standard live iso file (small, no graphic desktop environment),

debian-live-10.0.0-amd64-standard.iso

So this way you can make a USB boot drive with Debian 10 (64-bit). It will be live-only and boot in UEFI mode.

  • Create a FAT32 partition big enough to store the files from the iso file (add approximately 5 % to the size of the iso file)
  • Mount the FAT32 partition
  • Extract the content (directory tree with all the files) of the iso file to the FAT32 partition

Now you have a live-only USB boot drive

  • Edit the word persistence to the end of the line(s) starting with linux in the file

    /path/to/mountpoint/boot/grub/grub.cfg
    
  • Create an ext2 partition in the unallocated space (behind the FAT32 partition)

  • Put the label persistence on the ext2 partition
  • Mount the ext2 partition
  • Write / union to the file persistence.conf in the ext2 file system

  • Unmount all partitions on the flash drive before you unplug it, or reboot

Now you have a persistent live USB drive with Debian 10.


Command output seen from within the persistent live drive:

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            7.8G     0  7.8G   0% /dev
tmpfs           1.6G  9.0M  1.6G   1% /run
/dev/sdb1       4.0G  826M  3.2G  21% /run/live/persistence/sdb1
/dev/loop0      610M  610M     0 100% /run/live/rootfs/filesystem.squashfs
tmpfs           7.8G     0  7.8G   0% /run/live/overlay
/dev/sdb2        11G   38M   11G   1% /run/live/persistence/sdb2
overlay          11G   38M   11G   1% /
tmpfs           7.8G     0  7.8G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           7.8G     0  7.8G   0% /sys/fs/cgroup
tmpfs           7.8G     0  7.8G   0% /tmp
tmpfs           1.6G     0  1.6G   0% /run/user/1000

$ lsb_release -a
Distributor ID: Debian
Description:    Debian GNU/Linux 10 (buster)
Release:    10
Codename:   buster

$ lsblk -fm /dev/sdb
NAME   FSTYPE LABEL       UUID                                 FSAVAIL FSUSE% MOUNTPOINT                            SIZE OWNER GROUP MODE
sdb                                                                                                                14.8G             brw-rw----
├─sdb1 vfat   USBBOOT     7176-C538                               3.2G    20% /usr/lib/live/mount/persistence/sdb1    4G             brw-rw----
└─sdb2 ext2   persistence 2b324439-d63e-4a19-bf57-d49ecb881828     10G     0% /usr/lib/live/mount/persistence/sdb2 10.8G             brw-rw----

You can see that the size of overlay matches that of /dev/sdb2 which indicates that the persistence is working.

  • 1
    General information for users: As of today (5 Sep 2019), mkusb 12.3.2-1ubuntu4 is available via PPA for Ubuntu 12.04 and newer. SparkyLinux, a Debian-based distro, has the same version in their repository as announced in March 2019. – clearkimura Sep 5 at 14:04
  • 1
    Post-bounty comment: The sed command is just a workaround, and yet nobody seems to comment or explain the limitations. I think copying the solution from other answer is doing more harm than good, unless you can clarify when that will fail to work. – clearkimura Sep 12 at 14:08
  • @clearkimura, I have tested 'all the way' F.Hauri's to 'binary edit' persistence into a Debian 10 iso file, create a partition behind it with fdisk and tweak the partition to become a container for persistent data. I can confirm that it works with Debian 10 both in UEFI and BIOS mode, but I cannot say anything yet about previous versions (Debian 8 and 9). I remove the text from the answer and make 'only' this comment about it. – sudodus Sep 12 at 15:12
  • 1
    Not sure if my comment was misinterpreted, but I just wanted to hint that the sed command will not work for everyone. Given that is only a workaround, it is not worth the time to do further testing. – clearkimura Sep 13 at 7:30
  • 1
    I have tested mkusb 2 years ago, it work perfectly on/for creating a persistent debian 8 and 9. here is my answer on U&L – GAD3R Sep 13 at 9:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.