Multiple stage servers in open build service

 The open build service publisher has a configuration variable in BSConfig.pm, where you can define a rsync server to publish the built repos to.

Unfortunately, the documentation apart from the actual code (in src/backend/bs_publish function sync_to_stage) seems scarce, so let's document one non-standard usage here.

"Standard" (IMO) usage, one rsync staging server:

our $stageserver = 'rsync://my-rsync/obs-repos-module';

 But now, for a transition phase, I need multiple rsync servers that are all synced to. The format for this is a perl array variable reference that contains pairs of "project name regex, array of sync servers". This also allows to sync different repositories to different rsync servers for example.

The simplest use of this is "sync all repos to multiple rsync servers" and is configured like this:

our $stageserver = ['.*', ['rsync://rsync1/module1', 'rsync://rsync2/module2', 'rsync://rsync3/module3']];

 With this configuration, bs_publish will send all repos to the three mentioned rsync URLs in turn.

Make Hibernation Great Again!

 I have now put the fixes from the last two blogposts into a github repository: https://github.com/seife/make-hibernate-great-again so that you can just clone it, examine what it does and then run

sudo make install

followed by

sudo dracut -f

This should do the following:

• let resume actually work (by enabling the dracut resume module)
• make hibernate more verbose (switch to an empty text console and increase the kernel's log level before hibernation, restore after resume)
• make resume more verbose

Have fun!

I have also packaged it in obs://home:seife:testing, but I do not really advise to use this repo on anything but my own machines ;-) 

 

 

"hibernation fix" part 2: verbose resume

Now that I fixed hibernation to resume at all, I found another thing I had added to my old install long ago.

The "problem" is, that resume from hibernation is very quiet. So all you see is the GRUB messages "loading kernel", "loading initrd", then nothing, just the disk light may be blinking for quite some time, and if resume is successful, you'll see the unlock prompt of your screenlock.

That's a bit too little "user interface" for my taste.

The reason it is like this is, that the kernel's progress messages (in 10% steps) are printed with level "pr_info":

    if (!(nr_pages % m))

        pr_info("Image loading progress: %3d%%\n",

                                     nr_pages / m * 10);

But the default "quiet" boot suppresses KERN_INFO messages, which is good because without that, you cannot find the prompt for unlocking your crypted filesystems amongst all those KERN_INFO messages during boot... so how to fix it?

We can just adapt the systemd-hibernate-resume.service to increase the kernel log level just before starting to resume, via a dropin file in /etc/systemd/system/systemd-hibernate-resume.service.d/10-resume-verbose.conf:

[Service]

ExecStartPre=/bin/sh -c "echo 7 > /proc/sys/kernel/printk"

ExecStartPost=/bin/sh -c "echo 1 > /proc/sys/kernel/printk"

Also, add the file to your dracut config, so that it is actually included in the initrd:

install_items+=" /etc/systemd/system/systemd-hibernate-resume@.service.d/10-resume-verbose.conf "

now rebuild your initrd with "dracut -f" and enjoy :-)

Oh, and of course this only helps if you do not use things like plymouth which actually hide everything interesting during boot. Update 2022-02-12: it also works with plymouth, because image loading kicks in before plymouth can hide the interesting bits.

dracut: fix hibernate after fresh install

Just for fun, I finally installed a fresh Tumbleweed on one of my machines to actually find out if I'm missing out on new features that are masked by just always updating the old installation.

One feature that I had missed was, that hibernation was no longer working. Or, to be more exact, hibernation was working, but resume was not. Investigating the issue, I found that dracut's "resume" module was not included in the initramfs, which in turn lead to initramfs not even trying to resume.

The dracut mechanism has some logic to actually find out if suspend and resume is configured, and when it decides it is not, it will just skip adding the "useless" resume module.

Unfortunately, the check is faulty IMHO. It checks, if the resume device is configured in the kernel, and if it is, it adds the module to dracut. The problem is, that this kernel config is written by the resume code in the initrd... so during installation this is not the case, and as a result the module is not added to initrd, which leads to the config not being set on next reboot... 

The trivial fix is to call dracut once with the option to include the resume module:

dracut -a resume -f

After a reboot, the kernel config will be set and in the future the resume module will always be included in the initrd.

For an even saver setup, adding

add_dracutmodules+=" resume "

to /etc/dracut.conf.d/99-local.conf might be even better.

Of course I wanted to report a bug about the issue, then found out that there are plenty already:

• openSUSE: https://bugzilla.opensuse.org/1192506
• upstream dracut: https://github.com/dracutdevs/dracut/issues/924
• Fedora: https://bugzilla.redhat.com/show_bug.cgi?id=1795422
• RedHat actually fixed it for RHEL9 by just removing the crazy "only include it if it has been included before"-logic: https://github.com/redhat-plumbers/dracut-rhel9/pull/12

Ethernet on the BananaPi M2 Zero

Even though it does not look like it does, the BPI-M2 Zero also has a wired ethernet interface. Unfortunately, it is disabled by default in its device tree blob.

But enabling it is easy:

# copy into /root
cp /boot/dtb/sun8i-h2-plus-bananapi-m2-zero.dtb .
# decompile
dtc -I dtb sun8i-h2-plus-bananapi-m2-zero.dtb \
    -O dts -o sun8i-h2-plus-bananapi-m2-zero.dts
# edit
vi sun8i-h2-plus-bananapi-m2-zero.dts # :-)
# compile
dtc -i dts sun8i-h2-plus-bananapi-m2-zero.dts \
    -O dtb -o /boot/custom.dtb

How do you need to edit the DTB file?

In the "ethernet@1c30000" block, you need to apply this "diff":

-       status = "disabled";
+       status = "okay";
+       phy-handle = <0x4d>;
+       phy-mode = "mii";
        phandle = <0x4b>;

The "phy-handle" value is taken from the "phandle" of the "ethernet-phy@1" block a few lines down. Then another change is adding an alias for ethernet0 to the "aliases" block:

       aliases { 

              serial0 = "/soc/serial@1c28000";
              serial1 = "/soc/serial@1c28400";
+             ethernet0 = "/soc/ethernet@1c30000";
       };

Ethernet will work without that, but we'll need this for setting the MAC address later.

Now reboot and try it out in a temporary way. Either by:

• interrupting u-boot and doing setenv fdtfile custom.dtb (no "saveenv" yet!), then "boot"
• editing the GRUB menu, adding an additional line after "initrd..." that reads devicetree /boot/custom.dtb

Check if the ethernet device eth0 appears after boot. If it does, the u-boot method can be persisted by issuing a saveenv command after the setenv.

Persisting the MAC address:

At my first tries I noticed that the MAC address was randomly assigned on every boot, in later experiments I could not reproduce this anymore but it looked like the MAC was at least reassigned on every deployment. I have no idea if this has to do with saving the environment in u-boot or if there is something in the userspace (a systemd service maybe?) that makes the MAC address semi-persistent. I decided to fix the MAC address once and for all, since I'm running a static DHCP setup, this is pretty important for my use case.

Because of the "ethernet0" alias in the device tree, persisting the MAC address is easy. All you need to do is setenv ethaddr 22:33:44:55:66:77 in u-boot and then persist this with saveenv and you are done. Note that u-boot has some precautions to disallow changing the MAC address later by accident so you are not able to easily undo this later. You can always remove "/boot/efi/uboot.env" in your booted system or by inserting the SD card into another machine and start from scratch.

GRUB, u-boot, kernels and DTB loading (on the BPi M2 Zero and others)

While I was experimenting with the BananaPi M2 Zero board, I soon needed to adopt its device tree file (dtb).

Fortunately, the friendly members of the openSUSE:Factory:ARM community quickly hinted me at the grub2 "devicetree" command which can be specified similar to "linux" or "initrd" to name a file that's loaded as device tree.

Unfortunately, there is no way to make this really persistent, short of editing the grub generator scripts which will get lost on every grub2 update.

The other option would be to decompile the board's DTB file ("/boot/dtb/sun8i-h2-plus-bananapi-m2-zero.dtb" in my case), change and then recompile it, replacing the original file. This has two downsides: first, it will get overwritten with every update of the "dtb-sun8i" package (no idea how often this will be the case) and second, you might want to have the original file as fallback ready. In general, editing package managed files is not a good idea in my book, if it can be avoided it should be.

So I looked into the "who loads which device tree file" and found out that actually the dtb is loaded by u-boot, even before grub starts. U-boot has the name of the board built-in and thus the file name it is looking for. Additionally it has a search list of directories that it searches to find the dtb file. So the simplest way to apply your own dtb file would probably be to put it, with the original file name into the search path after the original file. I tried this approach at first, but then went for explicitly specifying a different filename, which is just not as subtle in case you need to debug this years later ;-)

So the method is relatively easy:

1. put your modified dtb file into /boot, I named it custom.dtb.
2. boot into u-boot, interrupt booting on the serial console
3. in u-boot console, enter setenv fdtfile custom.dtb
4. in u-boot console, enter saveenv

That's it. Now boot and verify that your dtb file is used.

Note that the u-boot environment is saved on the EFI partition of the SD card (first partition, FAT format) as file "uboot.env". If you need to reset the environment to the built-in defaults, then you can always mount the SD card in another machine and move away or delete uboot.env.

openSUSE Tumbleweed on the Banana Pi BPI-M2 ZERO

This is the first post of a small series about openSUSE on the Banana Pi BPI-M2 ZERO.

Preface

I recently got myself a Banana Pi M2 Zero board while ordering other stuff at an electronics distributor. The M2 zero is the same form factor and feature set as the Raspberry Pi Zero W (the GPIO pin headers are said to be compatible, it has WiFi and Bluetooth built in and an USB OTG port). The CPU is an Allwinner H2+, a quad-core ARM processor running a 1GHz clock speed, RAM size is 512MB. Processing power is probably comparable to a Raspberry Pi 2 board.

I bought the M2 Zero to use it with an RTLSDR stick to receive the signal of my outside RF temperature sensor. This worked with the Raspberry Pi Zero W, but was a bit too much for the slower CPU which has other more important things to do anyway (playing internet radio ;-), so the M2 Zero was a cheap, more powerful alternative. The box will be running headless and thus I do not care about support for graphics and multimedia anyway.

In the end, I switched the RF receiver to a RaspyRFM board whih is using less energy and simpler to use than an RTLSDR stick just to receive some sensors and now the M2 Zero board is free for tinkering...

openSUSE on the M2 Zero

There was already an image available for the Banana Pi M2 Plus (called "sinovoipbpim2plus"), which is the "bigger brother" of the M2 zero, but that image did not boot. Experimenting with the u-boot image from armbian lead to success in "openSUSE Tumbleweed boots with armbian u-boot". Thanks to the friendly openSUSE ARM community, a matching u-boot version for the M2 Zero was built quickly and I could submit the updated image in OBS to get an image ready for the board (called "sinovoipbpim2zero").

Some small things are still to be sorted out, this is why I would suggest you use the image from the home:seife:bananapi repository for now. Since the board has only WiFi networking (more on that in a later post...), you'll need a serial console wired up for initial setup and I strongly suggest to use at least the "openSUSE-Tumbleweed-ARM-X11-..." image and not the JeOS image, since JeOS does not contain NetworkManager and using WiFi with wicked (actually using anything with wicked) is not fun.

So put the image on the SD card, connect the serial console, boot the box up. Log in as root. WiFi connection is easily established then:

nmcli dev wifi connect YOUR_SSID password YOUR_PASSWORD

That's it, have fun! ;-)

Addon note: When I started to write this post, my home:seife:bananapi repo was necessary for actually getting WiFi to work (contained a fixed kernel-firmware package). This has all been submitted to upstream or openSUSE:Factory:ARM now. All that's "special" in my repo now is a slightly extended package selection in the image (the "dtc" package is included) and a fixed config.sh script that makes NetworkManager actually work correctly, including name resolution, see boo#1180234 for details), so the image from openSUSE:Factory:ARM should be "good enough" for most uses already and in the near future I'll probably retire the home:seife:bananapi project or use it just for development stuff).