Tuesday, June 23, 2020

Brother MFC-L2710DW Printer / Scanner and Linux

I needed a new scanner, my old HP PSC-1510 finally broke..
I wanted a multi-function device with ethernet, non-color laser printer with duplex printing and a scanner with ADF. For ease of use, I wanted a "scan to network drive" and "scan to email" capable device, so that my users at home can easily help themselves without me having to scan every document for them.

The Brother MFC-L2710DW seemed to fit the bill pretty well.
Only after having it all set up, I found out that the "Scan to email" only works with a Windows PC, and apparently only with the device connected via usb, by somehow firing up Outlook to send the mail, and the "scan to network drive" also seems to work the same way.

The next thing I found out is, that the printer works with CUPS, but you need a binary only driver from Brother (at least if you follow all the HOWTOs on the internet) and the scanner unit also seemed to need a binary only sane backend. Not much more and I would have almost immediately put up the device on ebay just to get rid of it as fast as possible.

But then I noticed that the thing is AirPrint and AirScan capable and I found out that no MacOS drivers were included...

Long story short: AirScan is some http based protocol (named eSCL) for basic scanning. With some curl magic you can scan (from standard glass or ADF) to either (at least) JPEG or PDF. When scanning from ADF, you even get a multi-page PDF directly. There is an "escl" backend in latest SANE (but not in Leap 15.2). However, this did not work well for me. Then I found the excellent "sane-airscan" project on github which works fine, and which serves both as a working SANE backend and as a good documentation on the protocol, which I used to write a standalone tool "simple-airscan" in python, which I'll probably put into a small simple webfrontend, so that my users can help theirselves with their scanning needs.

Ok, scanning works. What about printing?
Airprint has some "driverless" mode, which I was unable to configure with YaST, but it was easy to configure after I enabled the CUPS web frontend. This works without any ugly brother binary only drivers and prints just fine.

So now both the scanner and the printer work just fine without any brother software ;-)

https://github.com/seife/airscan-simple - my simple python scanning tool.
https://github.com/alexpevzner/sane-airscan - the excellent sane-airscan backend for SANE.
https://wiki.debian.org/CUPSDriverlessPrinting - the debian wiki has extensive information on driverless printing

Thursday, March 26, 2020

Windows 10 update error 0x800f0922

First: sorry for the OT ;-)

A Thinkpad of mine that has Windows 10 co-installed was refusing all cumulative Windows updates since about 6 months, always performing everything, rebooting, counting up to 99%, then failing with error 0x800f0922 and rolling back.
Now this Windows instance is not really used and thus not booted on a regular base, but I'd still rather keep it up to date in case I somewhen really need it for something.

So I searched the internet for error 0x800f0922... and tried almost everything that was mentioned as a possible fix:

  • resetting windows update
  • uninstalling various pieces of software
  • in general, random changing of different settings ;-)
Nothing helped. Until I found a short comment under some blog post hidden in the vast voids of the internet, that mentioned that the problem could be solved by not booting via grub but instead directly activating the windows boot partition.

Could it be that easy? Yes. Rebooted into Linux, deactivated /dev/sda3 and activated /dev/sda1 with fdisk, rebooted and Windows is now updating happily ever after...

Sunday, March 22, 2020

Leap 15.2 Beta: mariadb upgrade woes

I'm running a server at home with openSUSE Leap, and since Leap 15.2 is now in beta, I thought it was a good idea to update and take part in the beta testing effort.
This machine is running an Owncloud instance, serving some internal NFS shares and used as a development machine for (cross-)compiling stuff, packaging etc.

The update went pretty smooth, only the mariadb instance used by Owncloud was hosed afterwards. There was no login possible and generally database users were gone.
Fortunately, I always have recent backups available, both a mysqldump and a complete file system backup.

So I tried to just restore the mysqldump on the updated database. This did not work, Bug#1166786.
Then I did just restore the filesystem backup of /var/lib/mysql and the database worked again.
Unfortunately, as I found out reproducing and investigating the issue, it would just get killed again by the next update, Bug#1166781. (Extra kudos to openSUSE Product Management which decided that this is not a bug, but instead regarded a feature!).

Finally I found the upstream bug in mariadb JIRA bugtracker, (which also does not look like there is much interest in fixing this), but with the information given there, I was able to fix this for me.

So all of you who are stuck with
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)
after updating a mariadb instance to 10.4, this might help:

  1. restart mysqld with option --skip-grant-tables, to disable user authentication
  2. in the mysql database, execute
    • ALTER TABLE user CHANGE COLUMN `auth_string` `authentication_string` text;
    • DROP TABLE global_priv;
  3. now run the mariadb-upgrade command
  4. restart mariadb.service with default options
This fixed my instance and owncloud is working again.

Note that I am by no means a database expert. Take backups before performing these steps.

Tuesday, January 21, 2020

Fun with grub2-install "not a directory"

These days, I came across spontaneous kiwi image build failures in a private OBS instance.
The images were SLES15-SP1, they had not been touched for quite some time, rebuilds were only triggered due to new packages in the update channel.
The error was grub2-install failing with the error message "not a directory".
Looking at the recent changes in the update repo showed no obvious reason (some python packages that had nothing to do with grub2-install), so I started to investigate...

... 3 days later, after following some detours, I finally found the issue.

grub2-install scans the installation device for filesystems, and probes all known (to grub2) fs types. The probe of "minix_be" fails fatally. Sometimes.

After building my own grub2 package with lots of debug-printf's, I finally found out, that the minix fs detection of grub2 is a little "fragile". It does the following (pseudo code):
  • grub_mount_minix(device) || return "not minix fs"
  • grub_minix_find_file("/") || fatal_error()
The problem is, that grub_mount_minix() only does pretty simple magic numbers checks, which can lead to false positives.

Comparing the superblock structures of ext[234] and minix filesystems (from the grub2 source code) side by side, you see this:

struct grub_minix_sblock         |struct grub_ext2_sblock
{                                |{
  grub_uint16_t inode_cnt;       |  grub_uint32_t total_inodes;
  grub_uint16_t zone_cnt;        |
  grub_uint16_t inode_bmap_size; |  grub_uint32_t total_blocks;
  grub_uint16_t zone_bmap_size;  |
  grub_uint16_t first_data_zone; |  grub_uint32_t reserved_blocks;
  grub_uint16_t log2_zone_size;  |
  grub_uint32_t max_file_size;   |  grub_uint32_t free_blocks;
  grub_uint16_t magic;           |  grub_uint32_t free_inodes;

This already hints at the issue: at the same disk location where ext2 stores the free inodes number, minix stores its magic number, which is used by grub to detect if it is a minix file system.

Now if you happen to have an ext3 file system with a free_inodes number whose lower 16 bits resemble one of the GRUB_MINIX_MAGIC numbers, chances are grub_mount_minix() will succeed, but then the attempt to acces the root directory will fail with a fatal error.

This is a plain grub2 bug, which I will probably report upstream and try to get fixed.
However, I need a fix to have my images build again, and the chances of getting a fix into SLES15-SP1 are ... low (and it is a daunting task, even if you are reporting this bug as a big SLES customer), so I built a workaround in my (locally built, lucky me...) python-kiwi package.

It basically does the following, before calling the "chroot grub2-install ...".

  • statvfs() to get the free_inodes number
  • check if the lower 16 bits resemble one of the MINIX_MAGIC numbers
    • if it does, touch a temporary file in
    • unmount and mount again to update the superblock (I missed this at first and wondered why it did not work)
    • unlink the temporary file
  • continue as before
This workaround is ugly as hell, but it does work for me.

P.S.: the detours included first noticing that almost every change I made to the image, like wrapping grub2-install into a wrapper script for debugging) made the issue go away (because of a different free_inodes number), so I always needed to check after every change that the issue was still present, then finding that copying the locales in grub2-install actually triggers an ENOTDIR - "Not a directory", because it misses special handling the /usr/share/locale/locale.alias file. Of course I thought "this is the issue" and patched it out of grub2, just to find that the original problem still persisted... then overnight package updates in SLES15-SP1 making this problem go away and reappear seemingly random... you guess it 😄

Thursday, February 28, 2019

KIWI repository SSL redirect pitfall

Today, I was wondering why my local kiwi image builds got download problems:
[ DEBUG ]: 09:52:15 | system: Retrieving: dracut-kiwi-lib-9.17.23-lp150.1.1.x86_64.rpm [.error] [ DEBUG ]: 09:52:15 | system: Abort, retry, ignore? [a/r/i/...? shows all options] (a): a
[ INFO ]: Processing: [########################################] 100%
[ ERROR ]: 09:52:15 | KiwiInstallPhaseFailed: System package installation failed: Download (curl) error for 'http://download.opensuse.org/repositories/Virtualization:/Appliances:/Builder/openSUSE_Leap_15.0/x86_64/dracut-kiwi-lib-9.17.23-lp150.1.1.x86_64.rpm':
Error code: Curl error 60
Error message: SSL certificate problem: unable to get local issuer certificate
SSL error for an http:// URL? Digging for the zypper logs in the chroot found, that this request got redirected to https://provo-mirror.opensuse.org/repositories/... today, which failed due to missing SSL certificates.

After I had debugged the issue, the solution was quite simple: in the <packages type="bootstrap"> section, just replace ca-certificates with ca-certificates-mozilla and everything works fine again.

Of course, jut building the image in OBS would also have solved this issue, but for debugging and developing, a "native" kiwi build was really necessary today.

Sunday, February 03, 2019

Raspberry Pi: "Bluetooth: hci0 link tx timeout"

I'm right now playing around with an old Raspberry Pi, to use it as a bluetooth speaker / DLNA renderer for audio. I spent almost a whole day trying to get bluetooth a2dp audio sink to work, but it almost always failed already at the pairing / connection stage. A similar thing had worked fine last week on a Raspi 3, but I want to try it with the old version as eventually the "Appliance" will be built with a Raspi Zero W, which is more like the first model. In the kernel log buffer I found the following:
Bluetooth: hci0 link tx timeout
Bluetooth: hci0 killing stalled connection xx:xx:xx:xx:xx:xx
This happened with different USB bluetooth dongles. Googling the problem found mostly unrelated articles, or advice that was obviously plain wrong.

Long story short: moving the dongle to a powered USB hub solved the issue.

(Just for the record: the raspi is powered by a good power supply...)

Friday, June 01, 2018

GRUB2 submenus revisited

After my last post quite some time ago, I had to revisit this issue again and found that there is a much easier way to "fix" the problem: Just add

in /etc/default/grub, recreate your grub.cfg and voila: the submenus are gone.