Hardware List Refresh fails after SLES12->SLES15 upgrade

[kurzandras]

Problem description

I have successfully upgraded a system from SLES12 to SLES15, but I am experiencing issues:

• Hardware List Refresh task stays in "picked up" state and fails after a few hours with "No job return event was received."
• When I click on "Apply Highstate" the task will get stuck in picked-up state too
• Uyuni web states that "The system requires a reboot", which does not disappear after reboot

Other things like package list update and patching are working fine. I could not see any issues in the server / client logs. How does Uyuni know which system needs to be rebooted? Could you please point me to the right direction regarding troubleshooting?

Steps to reproduce

1. Bootstrap SLES12SP5 host
2. Upgrade to SLES15SP3 according to the Uyuni documentation
3. Try to apply highstate and/or execute hardware list refresh
   ...

Uyuni version

Information for package Uyuni-Server-release:
---------------------------------------------
Repository     : uyuni-server-stable
Name           : Uyuni-Server-release
Version        : 2023.12-230900.210.2.uyuni3
Arch           : x86_64
Vendor         : obs://build.opensuse.org/systemsmanagement:Uyuni
Support Level  : Level 3
Installed Size : 1.4 KiB
Installed      : Yes (automatically)
Status         : out-of-date (version 2023.10-230900.209.1.uyuni3 installed)
Source package : Uyuni-Server-release-2023.12-230900.210.2.uyuni3.src
Summary        : Uyuni Server
Description    :
    Uyuni lets you efficiently manage physical, virtual,
    and cloud-based Linux systems. It provides automated and cost-effective
    configuration and software management, asset management, and system
    provisioning.

Uyuni proxy version (if used)

No response

Useful logs

No response

Additional information

No response

[kurzandras]

It seems that now there are more than usual 2 venv-salt-minion processes running, it might be related somehow to this issue:

root 5129 1 0 07:50 ? Ss 0:00 /usr/lib/venv-salt-minion/bin/python.original /usr/lib/venv-salt-minion/bin/salt-minion
root 5138 5129 0 07:50 ? S 0:02 /usr/lib/venv-salt-minion/bin/python.original /usr/lib/venv-salt-minion/bin/salt-minion
root 6880 5138 0 07:51 ? Sl 0:01 /usr/lib/venv-salt-minion/bin/python.original /usr/lib/venv-salt-minion/bin/salt-minion
root 9700 5138 0 08:07 ? S 0:01 /usr/lib/venv-salt-minion/bin/python.original /usr/lib/venv-salt-minion/bin/salt-minion