[SOLVED] Distribution upgrade Stretch to Buster fails

[Roy Terhaar]

Software
XBian version:1.0 (knockout) (kernel: Linux 4.19.75+)
XBMC/Kodi version:118.4-Git:20191006-104b22a-dirty
Overclock settings:default

Hardware
Device type and model (e.g. Raspberry Pi Model A/B 256/512 MB, CuBox-i i4Pro, ...):RPI 3b+
Power supply rating:Original power supply adapter
SD card size and make/type:32Gb
Network (Ethernet or wireless):Kodi using Ethernet; wireless adapter active too on different network
Connected devices (TV, USB, network storage, ...):Philips TV, External USB harddisks Seagate and Seagate Backup Plusi

Log files
-

Problem description:
Trying to upgrade the distribution of Stretch to Buster via the Kodi/XBian gui fails displaying on the screen somewhere halfway the installation process:
Errors were encountered while processing:
util-linux
mount
initscripts
sub-process /usr/bin/dpkg returned an error code (1)

I have already tried the standard things found on the internet regarding the dpkg error, but none seems to work solving the problem.

I have checked contents of sources.list:
deb http://http.debian.net/debian stretch main non-free
And xbian.list:
deb http://apt.xbian.org/ stable main rpi2-stretch armv7l-stretch
Looks ok I guess...and during upgrade process 'stretch' gets replaced by 'buster', so that seems to be ok.

I have tried to reinstall the three problem packages:
sudo apt-get install --reinstall util-linux (this one shows a lot of warnings)
sudo apt-get install --reinstall mount (no warnings here)
sudo apt-get install --reinstall initscripts (this one shows a lot of warnings too)
The warnings refer to missing symbolic links in /etc/rcS.d
I am not sure if this can be the cause of the upgrade problem, but it is a possibility right?
This is part of the warnings shown on screen:
sudo apt-get install --reinstall util-linu x
[sudo] password for xbian:
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded.
Need to get 0 B/917 kB of archives.
After this operation, 0 B of additional disk space will be used.
Create a snapshot of '/run/xbian-storage/root/@' in '/run/xbian-storage/root/@bt rfs-auto-snap_apt-run-2019-11-03-1824'
Delete subvolume (no-commit): '/run/xbian-storage/root/@last_good_known'
Create a snapshot of '/run/xbian-storage/root/@btrfs-auto-snap_apt-run-2019-11-0 3-1824' in '/run/xbian-storage/root/@last_good_known'
Delete subvolume (no-commit): '/run/xbian-storage/root/@btrfs-auto-snap_apt-run- 2019-11-03-1743'
@btrfs-auto-snap_apt-run-2019-11-03-1824, 1 created snapshots, 0 sent snapshots, 1 destroyed snapshots, 0 created filesystems, 0 warnings.
(Reading database ... 40271 files and directories currently installed.)
Preparing to unpack .../util-linux_2.29.2-1+deb9u1_armhf.deb ...
Unpacking util-linux (2.29.2-1+deb9u1) over (2.29.2-1+deb9u1) ...
Setting up util-linux (2.29.2-1+deb9u1) ...
initctl: Unknown job: K01nfs-common
insserv: warning: script 'K01nfs-common' missing LSB tags and overrides
initctl: Unknown job: K01nfs-common
initctl: Unknown job: S02dbus
insserv: warning: script 'S02dbus' missing LSB tags and overrides
initctl: Unknown job: S02dbus
initctl: Unknown job: S02dbus
initctl: Unknown job: S02dbus
initctl: Unknown job: K01nfs-common
initctl: Unknown job: S04nfs-common
initctl: Unknown job: S04checkroot-bootclean.sh
insserv: warning: script 'S04checkroot-bootclean.sh' missing LSB tags and overrides
initctl: Unknown job: S04mountnfs-bootclean.sh
insserv: warning: script 'S04mountnfs-bootclean.sh' missing LSB tags and overrides
initctl: Unknown job: S04mountdevsubfs
insserv: warning: script 'S04mountdevsubfs' missing LSB tags and overrides
initctl: Unknown job: S04mountkernfs
insserv: warning: script 'S04mountkernfs' missing LSB tags and overrides
initctl: Unknown job: S04mtab

Right now I am kinda stuck resolving this problem.

Anybody any suggestions?

[Nachteule]

Hmmm, seems that your sources (sources.list & xbian.list) does not point to buster

Please check this first

Quote:(Reading database ... 40271 files and directories currently installed.)
Preparing to unpack .../util-linux_2.29.2-1+deb9u1_armhf.deb ...
Unpacking util-linux (2.29.2-1+deb9u1) over (2.29.2-1+deb9u1) ...
Setting up util-linux (2.29.2-1+deb9u1) ...

This file definitely comes from Debian Stretch!

[Roy Terhaar]

(4th Nov, 2019 10:05 PM)Nachteule Wrote:  Hmmm, seems that your sources (sources.list & xbian.list) does not point to buster

Please check this first

Quote:(Reading database ... 40271 files and directories currently installed.)
Preparing to unpack .../util-linux_2.29.2-1+deb9u1_armhf.deb ...
Unpacking util-linux (2.29.2-1+deb9u1) over (2.29.2-1+deb9u1) ...
Setting up util-linux (2.29.2-1+deb9u1) ...

This file definitely comes from Debian Stretch!

Sorry for not being clear, but after noticing that the upgrade was not working, I restored the Stretch image and tried to reinstall the 3 problem packages, so that is what's I have pasted in my problem description. It shows a lot of warnings no errors. Running the distribution upgrade again ends up in the same dpkg error...

[Nachteule]

Could you paste the log to pastebin or whatever?

If you run the XBian upgrade procedure, full log is in file /tmp/aptstatus

[Roy Terhaar]

(5th Nov, 2019 04:46 AM)Nachteule Wrote:  Could you paste the log to pastebin or whatever?

If you run the XBian upgrade procedure, full log is in file /tmp/aptstatus

Log is in:
https://pastebin.com/CfCsPs9n

[Nachteule]

Thanks, I'll investigate it tomorrow

[Nachteule]

You have apache2 and dnsmasq installed

Code:

insserv: FATAL: service dnsmasq is missed in the runlevels 2 3 4 5 to use service apache2

That's causing the error.

Could you try

A) set package dnsmasq and apache2 on hold before running upgrade
sudo apt-mark hold apache2 && sudo apt-mark hold dnsmasq and try again

or

B) temporarly deinstall both packages, run the upgrade and reinstall them after upgrade has been done

[Roy Terhaar]

(5th Nov, 2019 05:56 AM)Nachteule Wrote:  You have apache2 and dnsmasq installed

Code:

insserv: FATAL: service dnsmasq is missed in the runlevels 2 3 4 5 to use service apache2

That's causing the error.

Could you try

A) set package dnsmasq and apache2 on hold before running upgrade
sudo apt-mark hold apache2 && sudo apt-mark hold dnsmasq and try again

or

B) temporarly deinstall both packages, run the upgrade and reinstall them after upgrade has been done

Both options did not work out; in the end I am still ending up in the same error.
I checked in option B if dnsmasq or apache2 were still running or installed, but that was not the case.
So I don't understand that insserv is still checking upon apache2 and dnsmasq
I will call it a day for now and dig into it the day after tomorrow...

[Nachteule]

(5th Nov, 2019 07:44 AM)Roy Terhaar Wrote:  

(5th Nov, 2019 05:56 AM)Nachteule Wrote:  You have apache2 and dnsmasq installed

Code:

insserv: FATAL: service dnsmasq is missed in the runlevels 2 3 4 5 to use service apache2

That's causing the error.

Could you try

A) set package dnsmasq and apache2 on hold before running upgrade
sudo apt-mark hold apache2 && sudo apt-mark hold dnsmasq and try again

or

B) temporarly deinstall both packages, run the upgrade and reinstall them after upgrade has been done

Both options did not work out; in the end I am still ending up in the same error.
I checked in option B if dnsmasq or apache2 were still running or installed, but that was not the case.
So I don't understand that insserv is still checking upon apache2 and dnsmasq
I will call it a day for now and dig into it the day after tomorrow...

Yes indeed, it's very strange. Maybe there are some old links in the rc folders. Who knows

I'll see if I can find a Stetch installation which I can upgrade to Buster. So, if I still have one, I'll install dnsmasq and apache before upgrading and see what happens ...

[Nachteule]

(5th Nov, 2019 07:55 AM)Nachteule Wrote:  I'll see if I can find a Stetch installation which I can upgrade to Buster. So, if I still have one, I'll install dnsmasq and apache before upgrading and see what happens ...

I did that upgrade 2 times today, both with installed apache and dnsmasq

Upgrade 1) apache and dnsmasq enabled and running

Terminal

root@kmxbilr2 ~ # find /etc -type l \( -name \S*apache* -o -name \S*dnsmasq* \) -ls
169432 4 lrwxrwxrwx 1 root root 17 Nov 5 12:59 /etc/rc2.d/S04dnsmasq -> ../init.d/dnsmasq
169673 4 lrwxrwxrwx 1 root root 17 Nov 5 13:00 /etc/rc2.d/S05apache2 -> ../init.d/apache2
169433 4 lrwxrwxrwx 1 root root 17 Nov 5 12:59 /etc/rc3.d/S04dnsmasq -> ../init.d/dnsmasq
169675 4 lrwxrwxrwx 1 root root 17 Nov 5 13:00 /etc/rc3.d/S05apache2 -> ../init.d/apache2
169434 4 lrwxrwxrwx 1 root root 17 Nov 5 12:59 /etc/rc4.d/S04dnsmasq -> ../init.d/dnsmasq
169677 4 lrwxrwxrwx 1 root root 17 Nov 5 13:00 /etc/rc4.d/S05apache2 -> ../init.d/apache2
169435 4 lrwxrwxrwx 1 root root 17 Nov 5 12:59 /etc/rc5.d/S04dnsmasq -> ../init.d/dnsmasq
169679 4 lrwxrwxrwx 1 root root 17 Nov 5 13:00 /etc/rc5.d/S05apache2 -> ../init.d/apache2
root@kmxbilr2 ~ #

Upgade 2) apache and dnsmasq disabled (all links shown above removed) and not running

Both upgrade sessions completed sucessfully without any problems

So I'm afraid, without being able to reproduce the problem, I can't help you any more at the moment

[Roy Terhaar]

(6th Nov, 2019 03:29 AM)Nachteule Wrote:  

(5th Nov, 2019 07:55 AM)Nachteule Wrote:  I'll see if I can find a Stetch installation which I can upgrade to Buster. So, if I still have one, I'll install dnsmasq and apache before upgrading and see what happens ...

I did that upgrade 2 times today, both with installed apache and dnsmasq

Upgrade 1) apache and dnsmasq enabled and running

Terminal

root@kmxbilr2 ~ # find /etc -type l \( -name \S*apache* -o -name \S*dnsmasq* \) -ls
169432 4 lrwxrwxrwx 1 root root 17 Nov 5 12:59 /etc/rc2.d/S04dnsmasq -> ../init.d/dnsmasq
169673 4 lrwxrwxrwx 1 root root 17 Nov 5 13:00 /etc/rc2.d/S05apache2 -> ../init.d/apache2
169433 4 lrwxrwxrwx 1 root root 17 Nov 5 12:59 /etc/rc3.d/S04dnsmasq -> ../init.d/dnsmasq
169675 4 lrwxrwxrwx 1 root root 17 Nov 5 13:00 /etc/rc3.d/S05apache2 -> ../init.d/apache2
169434 4 lrwxrwxrwx 1 root root 17 Nov 5 12:59 /etc/rc4.d/S04dnsmasq -> ../init.d/dnsmasq
169677 4 lrwxrwxrwx 1 root root 17 Nov 5 13:00 /etc/rc4.d/S05apache2 -> ../init.d/apache2
169435 4 lrwxrwxrwx 1 root root 17 Nov 5 12:59 /etc/rc5.d/S04dnsmasq -> ../init.d/dnsmasq
169679 4 lrwxrwxrwx 1 root root 17 Nov 5 13:00 /etc/rc5.d/S05apache2 -> ../init.d/apache2
root@kmxbilr2 ~ #

Upgade 2) apache and dnsmasq disabled (all links shown above removed) and not running

Both upgrade sessions completed sucessfully without any problems

So I'm afraid, without being able to reproduce the problem, I can't help you any more at the moment

Thanks for the effort, but it might actually help me.
I will do a test adding those symbolic links manually first (if missing) and then try to upgrade.
If that does not work I can still do a test completely purging apache2 and dnsmasq.
If both tests still result in same error situation, then there is something else wrong I guess.
Will post any outcome later...

[Nachteule]

Ok.

If this fails - what I expect - you could do a workaround for continuing upgrade process:

After upgrade has failed, you could edit file /var/lib/dpkg/info/util-linux.postinst and comment out the set -e line, so the script does not exit on error and after doing this, continue with

Terminal

sudo apt-get install -f

and point 3 and 4 described in this post

from the command line

[Roy Terhaar]

(6th Nov, 2019 10:08 PM)Nachteule Wrote:  Ok.

If this fails - what I expect - you could do a workaround for continuing upgrade process:

After upgrade has failed, you could edit file /var/lib/dpkg/info/util-linux.postinst and comment out the set -e line, so the script does not exit on error and after doing this, continue with

Terminal

sudo apt-get install -f

and point 3 and 4 described in this post

from the command line

I have manually added 4 symbolic links that were missing for dnsmasq. The links for apache2 were already in place. Running the upgrade still showed some warnings along the way, but everything seemed to run fine ending in no errors. After reboot everything showed as normal.
Anything that I need to check just to be sure that the upgrade worked as expected?

[Nachteule]

(7th Nov, 2019 06:00 AM)Roy Terhaar Wrote:  I have manually added 4 symbolic links that were missing for dnsmasq. The links for apache2 were already in place. Running the upgrade still showed some warnings along the way, but everything seemed to run fine ending in no errors. After reboot everything showed as normal.
Anything that I need to check just to be sure that the upgrade worked as expected?

Wow, first positive message today Thanks for your feedback!

I don't think that you have to do more checks. If system boots, everything should to be ok

[Roy Terhaar]

(7th Nov, 2019 07:32 AM)Nachteule Wrote:  

(7th Nov, 2019 06:00 AM)Roy Terhaar Wrote:  I have manually added 4 symbolic links that were missing for dnsmasq. The links for apache2 were already in place. Running the upgrade still showed some warnings along the way, but everything seemed to run fine ending in no errors. After reboot everything showed as normal.
Anything that I need to check just to be sure that the upgrade worked as expected?

Wow, first positive message today Thanks for your feedback!

I don't think that you have to do more checks. If system boots, everything should to be ok

Nice. Thanks for the support!
Still not sure why adding the links helped, but it works anyway...