0

I would like to use the Oracle Linux UEK7 Kernel but UEK7 disabled DRBD , which we rely on heavily.

However, the .src.rpm is available so theoretically the drbd.ko module could be built against /lib/modules/$(uname -r)/build from their original .src.rpm kernel tree as out-of-tree modules using the kernel-uek-devel package.

One way to do this would be to tweak the .config and rpmbuild -bb the entire RPM, but I want to keep booting Oracle's original kernel and only build the drbd.ko module such that they will load cleanly into a kernel of the same version.

Basically, we want to build an in-tree kernel module as if it were an out-of-tree kernel module. (Note that the kernel version for the .ko's that we want and the kernel that we are using are exactly the same.)

Question:

  • How can I build specific kernel modules that were not originally built by the publisher for a kernel provided as a .src.rpm?
2

3 Answers 3

0

This is how I did it for UEK6

Install build dependencies:

yum group install Development\ Tools -y
yum install kernel-uek-devel kernel-rpm-macros kernel-abi-whitelists -y

Download DRBD source tar from Linbit and make rpm:

DRBD_VER="9.1.11"
curl -LO https://pkg.linbit.com//downloads/drbd/9/drbd-${DRBD_VER}.tar.gz
tar xf drbd-${DRBD_VER}.tar.gz
(cd ./drbd-${DRBD_VER} && make kmp-rpm)

Your rpm will be here: /root/rpmbuild/RPMS/x86_64

The only catch with building your own kernel modules is you may have to rebuild them everytime the kernel is upgraded (even minor upgrades)

1
  • Welcome to Stack Exchange! Ok, I but that doesn't answer the question: I want to build the 8.4 module that is in Linux 5.15.x from within the UEK7 .src.rpm. We don't plan to use DRBD 9.x (yet). The same procedure to build drbd.ko from that .src.rpm would solve building bcache.ko, too.
    – KJ7LNW
    Nov 3, 2022 at 21:45
0

It works almost same for kernel UEK R7, but you need also install gcc-toolset-11-annobin-plugin-gcc a build it with scl enable gcc-toolset-11 "make kmp-rpm"

2
  • Hi Jozef, welcome to SE! So you do you select drbd.ko and bcache.ko to be built as part of the resulting RPM?
    – KJ7LNW
    Dec 23, 2022 at 20:42
  • As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.
    – Community Bot
    Dec 24, 2022 at 20:51
0

There was a lot of troubleshooting to make this work. First, follow this link directions in another answer as a first pass because may work for you:

If that does not work, then make sure you run these prerequisites that need to run from your kernel source tree:

  • zcat /boot/symvers-5.15.0-7.86.6.1.el9uek.x86_64.gz > Module.symvers
  • cp /boot/System.map-5.15.0-7.86.6.1.el9uek.x86_64 System.map
  • cp Module.symvers vmlinux.symvers
  • ./scripts/extract-vmlinux /boot/vmlinuz-5.15.0-7.86.6.1.el9uek.x86_64 > vmlinux

Next, if you get errors in dmesg such as:

module: x86/modules: Skipping invalid relocation target, existing value is nonzero for type 1, loc 00000000fbd7a560, val ffffffffc0b33cf0

...then you need to pay close attention to what your .config file looks like compared to your distribution-provided config file.

In my case, I wanted to build drbd.ko which is not included in my distribution, and I wanted to build it from the same source tree. I started by copying the distribution-provided configuration into the distribution-provided Linux source tree (same version as what I was booted from) as follows:

cd /usr/src/linux
cp /boot/config-$(uname -r) .config

Then run make menuconfig, enable DRBD, and exit+save the new configuration to .config. Depending on what the Linux source tree finds, there may be configuration options that got turned on or off. Compare the generated configuration file with the distribution-provided configuration file as follows:

diff -uw /boot/config-$(uname -r) .config

I will go step by step in the relevant differences between my configuration and the distribution-provided configuration as follows:

-# Linux/x86_64 5.15.0-7.86.6.1.el9uek.x86_64 Kernel Configuration
+# Linux/x86 5.15.0 Kernel Configuration

There are two major differences here. The first obvious difference is that the version is wrong, so we have to pass EXTRAVERSION=-7.86.6.1.el9uek.x86_64 on the make command line. The second difference is slightly less obvious: notice that the first mine is x86_64, but the second line has x86 because you have to pass the architecture type as ARCH=x86_64. Thus, my make command begins as follows:

make EXTRAVERSION=-7.86.6.1.el9uek.x86_64 ARCH=x86_64 menuconfig

Now if you save and diff again you should notice that the version lines are identical.

While in my case I was using the same compiler, be sure that your compiler version did not change. If it did, you need to get same exact compiler that was used by your distribution. For example:

CONFIG_CC_VERSION_TEXT="gcc (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2.1.0.1)"

The next difference that I showed in my configuration was CTF missing:

-CONFIG_HAVE_CTF_TOOLCHAIN=y
...
-CONFIG_CTF=y

Because this is an Oracle-provided UEK kernel, dtrace was enabled. By downloading and installing the dtrace tooling, CTF was detected by the build chain and solved two different CTF diff problems in the configuration.

There were several config differences that were fixed by installing specific packages, so I will list them here as follows:

dnf install dwarves

  • -CONFIG_PAHOLE_HAS_SPLIT_BTF=y:

dnf install dtrace-devel

  • -CONFIG_DEBUG_INFO_BTF_MODULES=y
  • -CONFIG_PAHOLE_HAS_SPLIT_BTF=y
  • -CONFIG_DEBUG_INFO_BTF_MODULES=y
  • -CONFIG_MODULE_ALLOW_BTF_MISMATCH=y
  • -CONFIG_CTF=y

I also installed the binutils-x86_64-linux-gnu.x86_64 package, though I'm not sure if that solved any of the problems above. I mentioned it only because it was part of my testing. While maybe it was not necessary, at this point I do not know.

Now the command line to build DRBD looks as follows:

make \
  EXTRAVERSION=-7.86.6.1.el9uek.x86_64 \
  ARCH=x86_64 \
  drivers/block/drbd/drbd.ko

However, dmesg was still giving the following symbol errors

drbd: Unknown symbol lc_seq_printf_stats (err -2)
drbd: Unknown symbol lc_get_cumulative (err -2)
drbd: Unknown symbol lc_del (err -2)
drbd: Unknown symbol lc_committed (err -2)
drbd: Unknown symbol lc_get (err -2)
drbd: Unknown symbol lc_try_get (err -2)
drbd: Unknown symbol lc_element_by_index (err -2)
drbd: Unknown symbol lc_create (err -2)
drbd: Unknown symbol lc_try_lock (err -2)
drbd: Unknown symbol lc_destroy (err -2)
drbd: Unknown symbol lc_reset (err -2)
drbd: Unknown symbol lc_is_used (err -2)
drbd: Unknown symbol lc_seq_dump_details (err -2)
drbd: Unknown symbol lc_put (err -2)
drbd: Unknown symbol lc_find (err -2)

What I had overlooked during the diff of the .config from above was that a new module was included that DRBD depended upon:

+CONFIG_LRU_CACHE=m

When I noticed that, I did a bit of grabbing for the missing symbols and found that the kernel module lib/lru_cache.ko needed to be built as well. Here is my final kernel build command line that successfully built the DRBD module so that it would load into my running kernel from the distribution:

make \
  EXTRAVERSION=-7.86.6.1.el9uek.x86_64 \
  ARCH=x86_64 \
  lib/lru_cache.ko \
  drivers/block/drbd/drbd.ko

Finally, I was able to insert both modules, and it worked and correctly:

insmod lib/lru_cache.ko
insmod drivers/block/drbd/drbd.ko

Now simply copy them to your /lib/modules directory and run depmod, so they can be used like normal:

cp lib/lru_cache.ko drivers/block/drbd/drbd.ko /lib/modules/$(uname -r)/extra/
depmod -a
modprobe drbd
dmesg | tail

Finally, if you get errors like this:

Skipping BTF generation for drivers/md/raid456.ko due to unavailability of vmlinux

then you need to extract your distribution's vmlinux binary into your kernel source tree as follows:

./scripts/extract-vmlinux /boot/vmlinuz-5.15.0-7.86.6.1.el9uek.x86_64 > vmlinux

If it worked, then you should see the block device registered as follows:

drbd: initialized. Version: 8.4.11 (api:1/proto:86-101)
drbd: srcversion: 98E710E58B3041F3046305B 
drbd: registered as block device major 147

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .