System Administration, Reconfigurable Computing and Other Random Topics

“touch -r $< $@” in Makefiles – Please don’t

Recently I had a bad memory module in my main storage system which is also the backup target of many of my systems. To be on the safe side, I validated all backed up files against their origin via checksums. Interestingly, multiple systems showed differing checksums for all DejaVu font files, i.e. the files in the backup did not match those in the system. Clearly, this was not caused by the bad memory module. It looked very much like a systematic issue.

Read More

Figuring out the used space_cache version of a btrfs file system

Btrfs contains different implementations of a free space cache. The default currently is space_cache=v1. A newer tree-based cache can be activated by the mount option space_cache=v2. As soon as the new version has been used once, it will automatically be used for any future mount. But how do you figure out which version your file system is actually using?

There are multiple ways to determine the used space cache version. The easiest one is to just mount the file system and look into the kernel log. If the default space cache v1 is used, it will say something like

BTRFS info (device yourdevice): disk space caching is enabled

However, if it already has been configured to use the new space cache v2, it will say something like

BTRFS info (device yourdevice): using free space tree

One can also determine the used space cache version without mounting the file system by peeking into the superblock:

btrfs inspect-internal dump-super /dev/yourdev

If the file system already uses space_cache=v2, the compat_ro_flags will contain the flag FREE_SPACE_TREE.

For completeness: When playing with space_cache=v2, be aware of the risks (see the remarks on the space_cache mount option in btrfs(5)).

Debugging a flaky cpu steal time counter on a paravirtualized Xen guest

I recently noticed a strange phenomenon on a Debian Stretch server running as a paravirtualized guest on a Xen host: top showed the CPU either be 100% idle or 100% stolen. User, system, nice and waiting times were stuck at 0%. I cross-checked with vmstat and it showed 0% for all cpu time counters. Both tools are getting their information from /proc/stat which looked like the following:

cpu 5322 0 4376 12720669 37879 0 59 1198368772563 0 0
cpu0 5322 0 4376 12720669 37879 0 59 1198368772563 0 0

The third to last value is the steal time, denoting “stolen time, which is the time spent in other operating systems when running in a virtualized environment” [procfs(5)]. This value looked way too high and, in particular, it was counting backwards. So, if I wanted to put this system into production, some debugging was required before…

Read More

Migrating from ownCloud 9.1.6 to Nextcloud 10

If you are stuck with PHP 5.4 (e.g. because you are still running Debian Wheezy) and want to migrate from ownCloud to Nextcloud, you are probably facing a minor issue. Nextcloud 11 and newer require PHP 5.6 so you have to stick to version 10 instead. Nextcloud 10 reached its end-of-life with version 10.0.5, which internally corresponds to ownCloud version 9.1.5. The most recent version of ownCloud 9 is version 9.1.6 though, so when trying to migrate to Nextcloud you will face the following error:

Downgrading is not supported and is likely to cause unpredictable issues (from to

Looking at the git commits between ownCloud 9.1.5 and 9.1.6 shows that there were no changes to the database layout. So, as a workaround, you can just edit your config/config.php and set version to or lower. Afterwards, you should be able to run the normal upgrade procedure.

Fixing Roundcube’s command execution vulnerability in Debian Wheezy

Recently a quite serious vulnerability (CVE-2016-9920) in Roundcube was reported. Until now (7th Dec) this vulnerability is unfixed in Debian’s roundcube packages (see the corresponding entry in the Debian Security Tracker).

The upstream patch is not directly applicable to version 0.7 which is used in Debian Wheezy but with a little modification it is. Following you find a corresponding patch*.
Read More

How systemd-timesyncd handles leap seconds

This night we get another leap second, meaning that the last second of today is not 23:59:59 but 23:59:60 instead. The last time this happened was on June 30, 2012 and lead to issues on several servers due to bugs in the Linux kernel. Since then these bugs were fixed and also methods of hiding the leap second altogether were implemented. Here you can find a good overview over different configurations of kernel, ntpd and tzdata and how a leap second will be handled using these configurations. What is missing here is how a setup using systemd-timesyncd instead of ntpd will handle this situation.
Read More

XPS on RHEL 7 / CentOS 7

Officially the ISE suite is not supported to run on RHEL / CentOS 7. But since time moves on and Vivado nowadays is supported on those systems you may be tempted to run the ISE tools on this system as well.

If you face the issue that XPS just shows a blank window and even XPS project files are opened as text files, the following will probably solve this issue:

yum install mesa-libGLU

By now this was also mentioned on a Xilinx forum thread but I thought I’d post it anyway.