join files/devices in linear mode together in a linux system

Question

In order to test joining 3 files linearly on my Linux system, I do:

fallocate -l 100M file1.image
fallocate -l 150M file2.image
fallocate -l  50M file3.image

and then I run losetup:

losetup /dev/loop1 file1.image
losetup /dev/loop2 file2.image
losetup /dev/loop3 file3.image

I found this script for joining two files:

#!/bin/sh
# Join 2 devices together
size1=`blockdev --getsz $1`
size2=`blockdev --getsz $2`
echo "0 $size1 linear $1 0
$size1 $size2 linear $2 0" | dmsetup create joined

And I start it with

./join.sh /dev/loop1 /dev/loop2

it works with two devices, they are joined together into one:

lsblk
NAME               MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
loop1                7:1    0   100M  0 loop
└─joined           253:10   0   250M  0 dm
loop2                7:2    0   150M  0 loop
└─joined           253:10   0   250M  0 dm

I then remove the "joined" device with dmsetup remove joined and adjust the script for 3 device files:

#!/bin/sh
# Join 3 devices together
size1=`blockdev --getsz $1`
size2=`blockdev --getsz $2`
size3=`blockdev --getsz $3`
echo "0 $size1 linear $1 0
$size1 $size2 linear $2 0
$size2 $size3 linear $3 0"  | dmsetup create joined

start it with

./join.sh /dev/loop1 /dev/loop2 /dev/loop3
device-mapper: reload ioctl on joined (253:10) failed: Das Argument ist ungültig
Command failed.

But it fails with the error above. What am I doing wrong?

dmesg

device-mapper: table: 253:10: linear: Gap in table (-EINVAL)
device-mapper: ioctl: error adding target to table

---

at

https://stackoverflow.com/questions/4519761/programming-a-loopback-device-consisting-of-several-files-in-linux

I found a way to use mdadm to join linearly and when I tried it, it works. I did:

dd if=/dev/zero bs=1M count=100 of=a.img
dd if=/dev/zero bs=1M count=50 of=b.img
dd if=/dev/zero bs=1M count=150 of=c.img
losetup -f
/dev/loop0
for i in a b c; do losetup -f $i.img; done
mdadm --build /dev/md0 -l linear -n 3 /dev/loop[012]
mdadm: array /dev/md0 built and started.
cat /proc/mdstat 
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] 
md0 : active linear loop2[2] loop1[1] loop0[0]
      307200 blocks super non-persistent 64k rounding

unused devices: <none>

lsblk
NAME               MAJ:MIN RM   SIZE RO TYPE   MOUNTPOINTS
loop0                7:0    0   100M  0 loop
└─md0                9:0    0   300M  0 linear
loop1                7:1    0    50M  0 loop
└─md0                9:0    0   300M  0 linear
loop2                7:2    0   150M  0 loop
└─md0                9:0    0   300M  0 linear

Why does the dmsetup way not work with 3 devices?

What is the difference between the dmsetup and mdadm ways?

What is dmsetup for ...?

And is mdadm for raid?

Have i done a raid0 with 3 different disk sizes? is that possible?

Answer 1

The format of linear device mapper table lines is as follows:

<target start> <mapping length> linear <source device> <source start>

Or in other words, from <source device>, starting from block position <source start> onward for <mapping length> blocks should be represented in the resulting mapping as a sequence of blocks starting from <target start>.

When joining devices together in order:

• on the first line, the <target start> must be 0
• on any subsequent lines, the <target start> must be equal to the sum of the sizes of all previous parts.

So, your script for three devices should be:

#!/bin/sh
# Join 3 devices together
size1=`blockdev --getsz $1`
size2=`blockdev --getsz $2`
sum12=`expr $size1 + $size2`
size3=`blockdev --getsz $3`
echo "0 $size1 linear $1 0
$size1 $size2 linear $2 0
$sum12 $size3 linear $3 0"  | dmsetup create joined

Or for an arbitrary number of devices:

#!/bin/sh
usage() {
    echo "Usage: ${0##*/} <target name> <component...>" >&2
    exit 64 # EX_USAGE
}
if [ "$1" = "" ] || [ "$1" = "-h" ] || [ "$1" = "-?" ] || [ "$1" = "--help" ]
then
    usage
fi

if [ -e "/dev/mapper/$1" ]
then
    echo "ERROR: target $1 already exists." >&2
    exit 73 # EX_CANTCREAT
fi
MAPNAME="$1"
shift

DMMAP=$(mktemp)
trap "rm -f $DMMAP" EXIT

sum=0
while [ "$1" != "" ]
do
    if [ ! -e "$1" ]
    then
        echo "ERROR: $1 does not exist"
        exit 66 # EX_NOINPUT
    fi
    if [ ! -b "$1" ]
    then
        echo "ERROR: $1 is not a block device"
        exit 69 # EX_UNAVAILABLE
    fi 
    srcN="$1"
    sizeN=$(blockdev --getsz "$srcN")
    echo "$sum $sizeN linear $srcN 0" >>$DMMAP
    sum=$(expr $sum + $sizeN)
    shift
done
dmsetup create $MAPNAME < $DMMAP

To use this script, specify the target name first, e.g.

sh ./dm-joinlinear joined /dev/loop1 /dev/loop2 /dev/loop3

Answer 2

Create executable script mapdevlist:

#!/bin/sh
mapname=$1
mapdev=/dev/mapper/$mapname
test -e "$mapdev" && {
    printf '%s already exists\n' "$mapdev"
    exit 1
}
shift
at=0
for dev do
    sz=$( blockdev --getsz "$dev" )
    printf '%u %u linear %s 0\n' \
           "$at" "$sz" "$dev"
    at=$(( $at + $sz )) # for POSIX sh
    # (( at += sz ))    # for Bash, Zsh etc
done |
dmsetup create "$mapname"

Call this like:

mapdevlist newmapdevname /dev/loop1 /dev/loop2 /dev/loop3 …