[config-package-dev] usrmerge and dpkg-divert

[Geoffrey Thomas]

>From yesterday's debian-devel thread
https://lists.debian.org/debian-devel/2021/07/msg00128.html
I learned that the dpkg maintainer does not consider the current 
implementation of usrmerge supported by dpkg, and usrmerge is on by 
default in Debian 11. Following some links,
https://wiki.debian.org/Teams/Dpkg/MergedUsr
claims that "dpkg-divert is currently broken by this approach."

As background, "usrmerge" is the situation where /bin and /usr/bin refer 
to the same directory (and same for /lib and /usr/lib, etc.). On a 
freshly-installed Bullseye system, this is implemented via symlinks:

geofft at bullseye:~$ ls -ld /*bin /lib*
lrwxrwxrwx 1 root root  7 Jul 15 21:21 /bin -> usr/bin
lrwxrwxrwx 1 root root  7 Jul 15 21:21 /lib -> usr/lib
lrwxrwxrwx 1 root root  9 Jul 15 21:21 /lib32 -> usr/lib32
lrwxrwxrwx 1 root root  9 Jul 15 21:21 /lib64 -> usr/lib64
lrwxrwxrwx 1 root root 10 Jul 15 21:21 /libx32 -> usr/libx32
lrwxrwxrwx 1 root root  8 Jul 15 21:21 /sbin -> usr/sbin

The reason the dpkg maintainer says it's not supported is that dpkg 
retains its original idea of whether files are in /bin or /usr/bin, for 
example:

geofft at bullseye:~$ dpkg -S /bin/ls
coreutils: /bin/ls
geofft at bullseye:~$ dpkg -S /usr/bin/ls
dpkg-query: no path found matching pattern /usr/bin/ls
geofft at bullseye:~$ dpkg -S /bin/less
dpkg-query: no path found matching pattern /bin/less
geofft at bullseye:~$ dpkg -S /usr/bin/less
less: /usr/bin/less

The practical consequence of this is that it's possible to write a package 
that diverts a file by its wrong name, which leads to confusing results. 
For instance, here's a variant of debathena-bin-example-1.0 (from the 
examples shipped with config-package-dev) where I've displaced /bin/less 
instead of /usr/bin/less:

geofft at bullseye:~$ sudo dpkg -i debathena-bin-example_1.0_all.deb
Selecting previously unselected package debathena-bin-example.
(Reading database ... 35842 files and directories currently installed.)
Preparing to unpack debathena-bin-example_1.0_all.deb ...
Unpacking debathena-bin-example (1.0) ...
Setting up debathena-bin-example (1.0) ...
Adding 'diversion of /usr/bin/elinks to /usr/bin/elinks.debathena-orig by 
debathena-bin-example'
Adding 'diversion of /bin/less to /bin/less.debathena-orig by 
debathena-bin-example'
Adding 'diversion of /usr/share/man/man1/elinks.1.gz to 
/usr/share/man/man1/elinks.debathena-orig.1.gz by debathena-bin-example'
Adding 'diversion of /usr/share/man/man1/less.1.gz to 
/usr/share/man/man1/less.debathena-orig.1.gz by debathena-bin-example'
geofft at bullseye:~$ sudo apt install --reinstall less
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not 
upgraded.
Need to get 133 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://deb.debian.org/debian bullseye/main amd64 less amd64 551-2 
[133 kB]
Fetched 133 kB in 0s (4,100 kB/s)
(Reading database ... 35854 files and directories currently installed.)
Preparing to unpack .../archives/less_551-2_amd64.deb ...
Unpacking less (551-2) over (551-2) ...
Setting up less (551-2) ...
Processing triggers for mailcap (3.69) ...
Processing triggers for man-db (2.9.4-2) ...
geofft at bullseye:~$ ls -l /usr/bin/less*
-rwxr-xr-x 1 root root 175816 Jul  5  2020 /usr/bin/less
-rwxr-xr-x 1 root root    105 Sep 11  2012 /usr/bin/less.debathena
-rwxr-xr-x 1 root root 175816 Jul  5  2020 /usr/bin/less.debathena-orig
[...]

As you can see, because the diversion was of the wrong path, dpkg didn't 
prevent the less package from updating /usr/bin/less, and so it overwrote 
the file we thought we owned, making the config package useless.

At least config-package-dev warns when you remove the package:

Removing debathena-bin-example (1.0) ...
Removing 'diversion of /usr/share/man/man1/less.1.gz to 
/usr/share/man/man1/less.debathena-orig.1.gz by debathena-bin-example'
Removing 'diversion of /usr/share/man/man1/elinks.1.gz to 
/usr/share/man/man1/elinks.debathena-orig.1.gz by debathena-bin-example'
*** OMINOUS WARNING ***: /bin/less is not linked to either less.debathena 
or less.debathena-orig
Not removing diversion of /bin/less by debathena-bin-example
Removing 'diversion of /usr/bin/elinks to /usr/bin/elinks.debathena-orig 
by debathena-bin-example'

but it leaves a mess to clean up by hand:

geofft at bullseye:~$ dpkg -S /bin/less
diversion by debathena-bin-example from: /bin/less
diversion by debathena-bin-example to: /bin/less.debathena-orig
geofft at bullseye:~$ ls -l /usr/bin/less*
-rwxr-xr-x 1 root root 175816 Jul  5  2020 /usr/bin/less
-rwxr-xr-x 1 root root 175816 Jul  5  2020 /usr/bin/less.debathena-orig
[...]

This only affects the case where you use the wrong path. As far as I can 
tell, if you use the right path, everything is fine. However, a 
consequence of usrmerge is that people aren't really supposed to care 
about which path is "right."

I'm trying to figure out what, if anything, we should do about this. Some 
options are:

- Add a NEWS entry + some docs saying that people need to be careful about 
using the right path.

- Automatically try to determine whether you're using the wrong path. This 
is trickier than it seems, because we do support the case where a config 
package diverts a file that doesn't yet exist.

- For usrmerge-affected directories, on a usrmerged system, divert both 
paths, and clean them both up on removal/undisplace. (I actually think 
this might Just Work...)

- Push for a change into dpkg to sort this out, somehow. I do expect that 
dpkg can be taught to figure its way around usrmerge, but I also expect 
that the dpkg maintainer is very much uninterested in implementing it. 
It's not clear to me whether the dpkg maintainer would be interested in 
accepting a patch for it.

- Do nothing (or maybe update the website and the version in git with some 
docs, but not the version in Bullseye).

Keep in mind that hard freeze is expected for this Saturday (see 
https://release.debian.org/), so we'd very much need a freeze exception 
for any option other than "do nothing."

Thoughts?

I'll go ahead and file a request to pre-approve some sort of change so we 
get on the release team's radar, at least.

-- 
Geoffrey Thomas
https://ldpreload.com
geofft at ldpreload.com