Compiling VDO for use on Debian

Question

I cannot find any packages for vdo for Debian, and my own attempts to compile and run the software have failed. Can anyone shed light on how to compile vdo for use with Debian as this is software released by RHEL after acquiring another company.

My current steps are:

apt-get update -y
apt-get install -y git sudo
sudo apt-get upgrade -y
sudo apt-get install -y build-essential libdevmapper-dev libz-dev uuid-dev
git clone https://github.com/dm-vdo/vdo.git
make
make install
sudo apt install -t stretch-backports linux-headers-$(uname -r)
git clone https://github.com/dm-vdo/kvdo.git
make -C /usr/src/linux-headers-`uname -r` M=`pwd`
cp vdo/kvdo.ko /lib/modules/$(uname -r)
cp uds/uds.ko /lib/modules/$(uname -r)
depmod
modprobe kvdo
modprobe uds
systemctl start vdo

// error with
Starting VDO volume services...
Traceback (most recent call last):
  File "/usr/bin/vdo", line 46, in <module>
    from vdo.utils import Command

Answer 1

OP's question is incomplete: the end of the error message which contains an important clue to solve this is not included. Here it is (on Debian buster. Debian 9 would instead search for python3.5):

# vdo status
Traceback (most recent call last):
  File "/usr/local/bin/vdo", line 46, in <module>
    from vdo.utils import Command
  File "/usr/local/lib/python3.7/dist-packages/vdo/utils/__init__.py", line 27, in <module>
    from .YAMLObject import YAMLObject
  File "/usr/local/lib/python3.7/dist-packages/vdo/utils/YAMLObject.py", line 33, in <module>
    import yaml
ModuleNotFoundError: No module named 'yaml'

So the python code needs a yaml module.

# apt-cache search python3 yaml | grep yaml | head -5
python3-pretty-yaml - module to produce pretty and readable YAML-serialized data (Python 3)
python3-xstatic-js-yaml - JavaScript yaml implementation - XStatic support
python3-xstatic-json2yaml - converts json or simple javascript objects into a yaml - XStatic support
python3-yamlordereddictloader - loader and dump for PyYAML keeping keys order
python3-yaml - YAML parser and emitter for Python3

# apt-get install python3-yaml

[...]

# vdo status
VDO status:
  Date: '2019-05-13 19:33:06+02:00'
  Node: somenode
Kernel module:
  Loaded: true
  Name: kvdo
  Version information:
    kvdo version: 6.2.0.293
Configuration:
  File: does not exist
  Last modified: not available
VDOs: {}

That's it. Note that without any configuration made, nothing would actually start. You should follow directions provided by Redhat there: 1.5. Creating a VDO volume.

Here's an example I ran:

# vdo create --name=vdo-data --device=/dev/md0 --vdoLogicalSize=8T
Creating VDO vdo-data
Starting VDO vdo-data
Starting compression on VDO vdo-data
VDO instance 0 volume is ready at /dev/mapper/vdo-data

Even without completely installing it, a peek at vdo.service gives enough informations:

ExecStart=/usr/bin/vdo start --all --confFile /etc/vdoconf.yml

So manually:

# vdo start --all --confFile /etc/vdoconf.yml
Starting VDO vdo-data
VDO instance 0 volume is ready at /dev/mapper/vdo-data

# ps -ef|grep vdo
root     11590     2  0 19:53 ?        00:00:00 [kvdo0:dedupeQ]
root     11593     2  0 19:53 ?        00:00:00 [kvdo0:journalQ]
root     11594     2  0 19:53 ?        00:00:00 [kvdo0:packerQ]
root     11595     2  0 19:53 ?        00:00:00 [kvdo0:logQ0]

[...]

# vdo status
VDO status:
  Date: '2019-05-13 19:54:46+02:00'
  Node: somenode
Kernel module:
  Loaded: true
  Name: kvdo
  Version information:
    kvdo version: 6.2.0.293
Configuration:
  File: /etc/vdoconf.yml
  Last modified: '2019-05-13 19:53:35'
VDOs:
  vdo-data:
    Acknowledgement threads: 1
    Activate: enabled
    Bio rotation interval: 64
    Bio submission threads: 4
    Block map cache size: 128M
    Block map period: 16380
    Block size: 4096
    CPU-work threads: 2
    Compression: enabled
    Configured write policy: auto
    Deduplication: enabled
    Device mapper status: 0 17179869184 vdo /dev/md0 normal - online online 1151960 242161600
    Emulate 512 byte: disabled
    Hash zone threads: 1
    Index checkpoint frequency: 0

[...]

Final note: to run it on kernel >= 4.20, which by default requires there is no variadic function in kernel, changes are needed for kvdo. The simplest is to ignore corresponding warnings, until the project itself corrects the affected functions. A 2x2 lines patched tree is available from an other RH employee there.