2

I am trying to take a Azure cloud Ubuntu 14.04 image (with a single partition for /) and re-organize it such that /var and /srv are on a logical volume backed by a file on the root drive.

(why, you may ask? I'm making a packer image with customizations and preloaded data so that when the REAL servers boot, the data can be moved onto a data disk instead of the OS disk)

I'm doing the following:

dd bs=16384 count=1048576 if=/dev/zero of=/.img-loop0
sync
losetup /dev/loop0 /.img-loop0
pvcreate /dev/loop0
vgcreate data /dev/loop0
lvcreate -l 100%FREE data -n var
mkfs.ext4 -L var -E nodiscard /dev/data/var
echo "/dev/data/var /var ext4 defaults 1 1" >> /etc/fstab
cd /var
mount /var
echo Moving old /var to new /var
mv * /var
echo Moving /srv to /var/srv
mv /srv /var/
ln -s var/srv /

and then doing an losetup in local-bottom

#!/bin/sh               
PREREQ=""               
prereqs()
{   
  echo "$PREREQ"
}             

case $1 in             
prereqs)                        
  prereqs                       
  exit 0
  ;;                            
esac                            

log_begin_msg losetup beginning 
if [ -f /root/.img-loop0 ]; then
  #mount /root -o remount,nodiscard
  losetup /dev/loop0 /root/.img-loop0
else                            
  log_warning_msg No /root/.img-loop0
fi                              
log_end_msg losetup finished

This all works great up until the reboot when everything goes to 💩:

[   15.538463] EXT4-fs (sda1): INFO: recovery required on readonly filesystem
[   15.572786] EXT4-fs (sda1): write access will be enabled during recovery
[   15.854354] EXT4-fs (sda1): recovery complete
[   15.882127] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
[   16.028060] random: lvm urandom read with 115 bits of entropy available
[   16.114088] random: nonblocking pool is initialized
[   16.403521] EXT4-fs (sda1): re-mounted. Opts: (null)
[   16.502056] Buffer I/O error on dev dm-0, logical block 1606657, lost async page write
[   16.541149] Buffer I/O error on dev dm-0, logical block 0, lost async page write
[   16.760508] Buffer I/O error on dev dm-0, logical block 0, lost sync page write
[   16.760882] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null)
[   17.722159] EXT4-fs warning (device dm-0): ext4_end_bio:317: I/O error -5 writing to inode 264766 (offset 0 size 4096 starting block 1082509)
[   17.784249] Buffer I/O error on device dm-0, logical block 1082509
[   19.248031] EXT4-fs warning (device dm-0): ext4_end_bio:317: I/O error -5 writing to inode 264774 (offset 0 size 4096 starting block 1094480)
[   19.248035] Buffer I/O error on device dm-0, logical block 1094480
[   19.829079] EXT4-fs warning (device dm-0): ext4_end_bio:317: I/O error -5 writing to inode 262206 (offset 0 size 4096 starting block 1096192)
[   19.829083] Buffer I/O error on device dm-0, logical block 1096192
[   20.011549] EXT4-fs warning (device dm-0): ext4_end_bio:317: I/O error -5 writing to inode 264763 (offset 0 size 4096 starting block 1096193)
[   20.011553] Buffer I/O error on device dm-0, logical block 1096193
[   20.043613] EXT4-fs warning (device dm-0): ext4_end_bio:317: I/O error -5 writing to inode 264777 (offset 0 size 4096 starting block 1096194)
[   20.043616] Buffer I/O error on device dm-0, logical block 1096194
[   20.080044] EXT4-fs warning (device dm-0): ext4_end_bio:317: I/O error -5 writing to inode 264778 (offset 0 size 4096 starting block 1096195)

I don't know what's going wrong here… I think I've narrowed out any sort of thinning/discarding as being the problem.


While writing this up I started wondering: perhaps I'm doing this too early and the losetup is capturing a file that's on an overlayfs of a read-only root… that might explain these errors.

When should I be doing the losetup so /dev/loop0 gets picked up by LVM?

1

It turns out that the mistake I was making was setting up the loopback device during initramfs, during which time the root filesystem is still read-only. Thus, changes that the loop device attempts to write to the backing file fail.

Responsibility for remounting the root filesystem as read-write belongs to upstart/systemd/etc., so I needed to add the following to /etc/init/lvmloop.conf to get it to work.

description "custom local LVM loopback device and setup"

start on mounted MOUNTPOINT=/

task

console output

script
LOOP0=/.img-loop0

if [ -f $LOOP0 ]; then
  losetup /dev/loop0 $LOOP0  # set up loop device...
  vgscan                     # scan for new PVs
  vgchange -ay               # activate VGs
  vgmknodes -v               # makes missing device nodes in /dev
  mount /var                 # for some reason mountall fails to do this...
fi
end script

It would feel "nicer" if there was a spot I could put this such that I didn't need to do these steps manually, but at least IT WORKS.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.