realtime: respect kernel-rt rpm #600

pirat89 · 2020-10-01T14:39:24Z

RHEL Real Time (RT) systems boot using kernel-rt rpm. But still,
even a kernel rpm can be installed. So now, detect whether we are
booted via kernel-rt or not and based on that:

do the comparison of the newest installed and booted kernel(-rt)
detect the target installed kernel(-rt) version

TODO

detect target kernel-rt version when original sys was RHEL RT
handle correctly default boot - boot into kernel-rt when RHEL RT

Comparison of kernel versions changes

Unfortunately, this make comparisons between releases more difficult
as we need to check the whole release, including alphanumeric
characters in release, e.g.:

   kernel-rt-3.10.0-1160.2.1.rt56.1133.el7.x86_64

Previously we worked just with numbers. To make the check valid and
more robust, let's use rpm.labelCompare(). We are sure this library
is on every RHEL 7 system as dnf requires it (and probably yum too).
Let's use it as it resolves all our current troubles right now and
makes whole comparison more reliable.

In this implementation I am skipping comparison with epoch as
there is no (and will not be) kernel with bumped epoch.

How postupgrade phases are affected

Actors working with bootloader and iniramfss needs to refer to specific kernel always. Currently, we are going to set the default boot and handle the target initramfs only for one specified kernel. If the original system was Real Time, work with kernel-rt related stuff. If the kernel-rt RHEL 8 rpm is not discovered (not installed), fallback to standard non-preemptive kernel. This is possible if user do not attach all expected RHEL 8 repositories (in this case, e.g. rhel-8-for-x86_64-rt-rpms).

If the original system is not real time (not booted with preemptive kernel) act as before. No change is visible in that case.

leapp-bot · 2020-10-01T14:39:39Z

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the
Leapp Guidelines and must pass all tests in order to be mergable.

If you want to re-run tests or request review, you can use following commands as a comment:

leapp-ci build to run unit tests, copr build and e2e tests in OAMG CI
e2e tests to run unit tests, copr build and end-to-end tests in Murphy CI (OAMG members only) [OLD PIPELINE]
review please to notify leapp developers of review request

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please consider rerunning the CI by commenting leapp-ci build (might require several comments). If the problem persists, contact leapp-infra.

pirat89 · 2020-10-07T15:53:52Z

@zhukovgreen @drehak : basically it should be done. you can run tests manually on your machine. not sure why the import fails when I see the function on centos and rhel as well. will look at it tomorrow.

pirat89 · 2020-10-08T11:04:20Z

@oamg/developers guys, the python3-rpm is not available on RHEL/CentOS 7. So I will keep it just on Python2 and skipp all related unite-tests for Python3 in the actor.

Update: So, let's skip all tests when module is not possible to load. So Mac-OS users will be happy too :)

zhukovgreen

Looks good to me in general, please give me your answers to the comments before merging.

...stem_upgrade/el7toel8/actors/kernel/checkinstalledkernels/libraries/checkinstalledkernels.py

pirat89 · 2020-11-04T11:35:37Z

I have realised we need to respect kernel-rt on more places...

pirat89 · 2020-11-04T21:58:47Z

Currently known issues have been fixed.

## Packaging - Add dependency on python2-leapp and leapp-framework - Add python-requests as dependency (oamg#407) - Drop leapp sos plugin (it's part of the sos rpm in RHEL 7.7+) - Remove dependency on Jinja2 related packages (oamg#407) ## Fixes - Do not mount pseudo and unsupposrted FS to overlayfs (e.g. proc) - Evaluate PES events transitively to create correct data for the upgrade transaction - Fix boot order on EFI systems - Fix checking of kernel drivers (oamg#400) - Fix failures caused by local rpms added into the upgrade transaction - Fix getting mount information with mountpoints with spaces in the path - Fix handling of XFS without ftype for every such mounted FS - Fix issue with random booting into old RHEL 7 kernel after the upgrade - Fix issues on systems with statically mapped IPs in /etc/hosts - Fix issues with device mapper and udev in a container - Fix issues with failing rpm transaction calculation because of duplicate instructions for dnf - Fix various issues related to RHSM (e.g. https://bugzilla.redhat.com/show_bug.cgi?id=1702691) - Fix yum repository scan in case of repositories with invalid URL - Inhibit the upgrade if multiple kernel-devel rpms are installed - Inhibit the upgrade when links on root dir '/' are not absolute to save the world - Parse correctly kernel cmdline inside the initrd (oamg#383) (fixes various issues on s390x) - Print warnings instead of a hard failure when expected rpms cannot be found (e.g. python3-nss inside an rpm module) (oamg#405) - Remove java11-openjdk-headless during the upgrade (https://bugzilla.redhat.com/show_bug.cgi?id=1820172) - Throw a nice error when invalid locale is set (oamg#430) ## Enhancements - Add initial multipath support (it doesn't handle all cases yet) - Changed upgrade paths: RHEL-ALT 7.6 -> 8.2; RHEL 7.8 -> 8.2 - Check if the latest installed kernel is booted before the upgrade - Check that the system satisfies minimum memory requirements for the upgrade (oamg#413) - Dump `grub2-editenv list` output to help with issues related to the default kernel for the boot - Improved report related to KDE/GNOME - Inhibit the upgrade for ipa-server (oamg#481) - Inhibit the upgrade on EFI systems when efibootmgr is not installed - Inhibit the upgrade on FIPS systems - Inhibit the upgrade when the raised dialogs are missing answers (oamg#589) - Introduce new ways of using custom repositories during the transaction - Make report messages more explicit about Dialogs (oamg#600) - Migrate SpamAssassin - Migrate cups-filters - Migrate sane-backend - Modify vim configuration to keep the original behaviour - Report changes in wireshark - Support the upgrade without the use of subscription-manager - The name and baseurl field in the CustomTargetRepository message are optional now - Use the new framework mechanism to inhibit the upgrade without reporting errors - Various texts are improved based on the feedback ## Additional changes interesting for devels - Add new functions in the config library to get envars related to leapp - Add support for testing with Beta and HTB systems - LEAPP_SKIP_CHECK_OS_RELEASE has been renamed to LEAPP_DEVEL_SKIP_CHECK_OS_RELEASE - Provide info about kernel cmdline via KernelCmdline message - The IPUConfig message contains information about booted kernel - The code is mostly Py2/Py3 compatible now and all PRs are tested on Py2 and Py3 compatibility (linters, unit-tests) - The config.version library contains is_rhel_alt() for detection of RHEL-ALT

mkluson · 2021-01-06T12:43:54Z

e2e tests

pirat89 · 2021-02-03T11:29:27Z

@mkluson rebasing against the master to ensure you will not hit recent issues. @oamg/developers re-review pls

pirat89 · 2021-02-03T11:37:57Z

@oamg/developers btw, it's possible I will have to rebase the branch again after the SAP-HANA PR is merged to fix possible conflicts.

The is_rhel_realtime function returns True only if the original system was booted using preemptive kernel (kernel-rt) and when the system is RHEL. The function is currently naive a little bit. The realtime kernel detection is based on the '.rt' substring in the release of kernel - which we know it will be there always only in case of the kernel-rt rpm. However, in future the test should be based rather on properties of kernel (preemptive-rt).

changed) RHEL Real Time (RT) systems boot using kernel-rt rpm. But still, even a kernel rpm can be installed. So now, detect whether we are booted via kernel-rt or not and do a comparison only in regarding the expected package name. Unfortunately, this make comparisons between releases more difficult as we need to check the whole release, including alphanumeric characters in release, e.g.: kernel-rt-3.10.0-1160.2.1.rt56.1133.el7.x86_64 Previously we worked just with numbers. To make the check valid and more robust, let's use rpm.labelCompare(). We are sure this library is on every RHEL 7 system as dnf requires it (and probably yum too). Let's use it as it resolves all our current troubles right now and makes whole comparison more reliable. In this implementation I am skipping comparison with epoch as there is no (and will not be) kernel with bumped epoch. Important: Regarding the use of the rpm module, someone could expect added dependency on rpm in python or spefile. Currently, nothing like that. In case case of python (pip) dependencies, there is not python module presented via pip as the module is written in CPython. Only way to install it is via packages. However, on RHEL 7 you can find just rpm module for Python2 and newer systems contain just rpm module for Python3. So there is no way to prepare virtual pyenv with specific rpm modules and the system one has to be used. Changes in the specfile are not needed as the related rpm is always installed and cannot be removed from systems based on RHEL and Fedora.

Previously just the kernel rpm has been detected. However, in case of real time systems, we need to detect the kernel-rt rpm. In case of real time kernels, the kernel rpm doesn't have to be installed at all. Currently it's not clear whether we should handle stuff realted to both kernels (kernel and kernel-rt) when both present. Right now, detect whether the original system was booted with preemptive kernel or not and based on that prefer the expected kernel. In case the target kernel-rt rpm is not present and it is expected, fallback to the kernel rpm to make possible finish of the upgrade, but log warning msg. TODO: raise report in such a case

Rezney · 2021-02-03T12:58:40Z

...stem_upgrade/el7toel8/actors/kernel/checkinstalledkernels/libraries/checkinstalledkernels.py

+        api.current_logger().info('The Real Time kernel boot detected.')
+        kernel_name = 'kernel-rt'
+
+    pkgs = get_pkgs(kernel_name)
    if not pkgs:
        # Hypothatical, user is not allowed to install any kernel that is not signed by RH


Suggested change

# Hypothatical, user is not allowed to install any kernel that is not signed by RH

# Hypothetically, user is not allowed to install any kernel that is not signed by RH

Rezney · 2021-02-03T13:00:31Z

...stem_upgrade/el7toel8/actors/kernel/checkinstalledkernels/libraries/checkinstalledkernels.py

+        api.current_logger().info('The Real Time kernel boot detected.')
+        kernel_name = 'kernel-rt'
+
+    pkgs = get_pkgs(kernel_name)
    if not pkgs:
        # Hypothatical, user is not allowed to install any kernel that is not signed by RH
        # In case we would like to be cautious, we could check whether there are no other


I vote to remove this comment or let's be cautious...

Rezney · 2021-02-03T13:03:50Z

repos/system_upgrade/el7toel8/actors/scaninstalledtargetkernelversion/libraries/scankernel.py

+    # pylint: disable=no-else-return  - false positive
+    # TODO: should we take care about stuff of kernel-rt and kernel in the same
+    # time when both are present? or just one? currently, handle only one
+    # of these during the upgrade. kernel-rt has higher prio when original sys


Suggested change

# of these during the upgrade. kernel-rt has higher prio when original sys

# of these during the upgrade. kernel-rt has higher prio when original system

Rezney · 2021-02-03T13:20:14Z

repos/system_upgrade/el7toel8/actors/scaninstalledtargetkernelversion/libraries/scankernel.py

+        # will crash without the created message. I am keeping kind of original
+        # behaviour in this case, but at least the let me log the error msg
+        #
+        api.current_logger().error('Cannot detect any kernel RPM')


Please be more specific and add "RHEL 8" string also here.

Rezney · 2021-02-03T13:21:39Z

repos/system_upgrade/el7toel8/actors/scaninstalledtargetkernelversion/libraries/scankernel.py

+        # behaviour in this case, but at least the let me log the error msg
+        #
+        api.current_logger().error('Cannot detect any kernel RPM')
+        # StopActorExecutionError('Cannot detect any RHEL 8 kernel RPM.')


Please use this or remove but I guess it is too late here anyway...

Rezney · 2021-02-03T13:22:42Z

repos/system_upgrade/el7toel8/libraries/config/version.py

+    """
+    Check whether the original system is RHEL Real Time.
+
+    Currently the check is based on the release of the original booted kernel.


Suggested change

Currently the check is based on the release of the original booted kernel.

Currently, the check is based on the release of the originally booted kernel.

Rezney · 2021-02-03T13:23:54Z

repos/system_upgrade/el7toel8/libraries/config/version.py

+    Currently the check is based on the release of the original booted kernel.
+    In case of RHEL, we are sure the release contains the ".rt" string and
+    non-realtime kernels don't. Let's use this minimalistic check for now.
+    In future, we could detect whether the system is preemptive or not based


Suggested change

In future, we could detect whether the system is preemptive or not based

In future, we could detect whether the system is preemptive or not, based

Rezney · 2021-02-03T13:26:42Z

...stem_upgrade/el7toel8/actors/kernel/checkinstalledkernels/libraries/checkinstalledkernels.py



-def get_kernel_rpm_release(rpm):
+def get_EVR(pkg):


Let's be consistent with uppercase and lowercase with the EVR in function names...

pirat89 · 2021-02-04T16:28:14Z

discussed with @Rezney, skipping now the review comments :)

Rezney

Ack

## Packaging - Bump required leapp-framework capability to 1.4 (oamg#642) ## Upgrade handling ### Fixes - Fix comparison of the newest installed and booted kernel (oamg#600) - Fix remediation command for ipa-server removal (oamg#617) - Fix crash due to missing network interfaces during upgrade phases (oamg#625) - Fix error with /boot/efi existing on non-EFI systems (oamg#627) - Fix false positive detection of issue in /etc/default/grub that led into GRUB prompt (oamg#587) - Fix syntax error in upgrade script (oamg#619) - Inhibit upgrade with mount options in fstab that break mounting on RHEL 8 (oamg#639) - Inhibit upgrade on s390x machines with /boot on a separate partition (oamg#641) - Inhibit upgrade if multiple kernel-debug pkgs are installed (oamg#599) - Remove the initial-setup package to avoid it asking for EULA acceptance during upgrade (oamg#626) - Remove the *leapp-resume* service after the *FirstBoot* phase to prevent kill of the leapp process on `systemctl daemon-reload` (oamg#611) ### Enhancements - Add upgrade support for SAP HANA (own upgrade path) (oamg#503) - Allow upgrade with SCA enabled manifest (oamg#615) - Add actors to migrate Quagga to FRR (oamg#467) - Add stable uniq Key id for every dialog (oamg#618) - Respect the *kernel-rt* package (oamg#600) ## Additional changes interesting for devels - Add a possibility to overwrite virtualenv name using `$VENVNAME` (oamg#613) - Update product certificates for RHEL 8.3 GA and 8.4 Beta/HTB (oamg#624) Related leapp release: https://github.com/oamg/leapp/releases/tag/v0.12.0

## Packaging - Bump required leapp-framework capability to 1.4 (#642) ## Upgrade handling ### Fixes - Fix comparison of the newest installed and booted kernel (#600) - Fix remediation command for ipa-server removal (#617) - Fix crash due to missing network interfaces during upgrade phases (#625) - Fix error with /boot/efi existing on non-EFI systems (#627) - Fix false positive detection of issue in /etc/default/grub that led into GRUB prompt (#587) - Fix syntax error in upgrade script (#619) - Inhibit upgrade with mount options in fstab that break mounting on RHEL 8 (#639) - Inhibit upgrade on s390x machines with /boot on a separate partition (#641) - Inhibit upgrade if multiple kernel-debug pkgs are installed (#599) - Remove the initial-setup package to avoid it asking for EULA acceptance during upgrade (#626) - Remove the *leapp-resume* service after the *FirstBoot* phase to prevent kill of the leapp process on `systemctl daemon-reload` (#611) ### Enhancements - Add upgrade support for SAP HANA (own upgrade path) (#503) - Allow upgrade with SCA enabled manifest (#615) - Add actors to migrate Quagga to FRR (#467) - Add stable uniq Key id for every dialog (#618) - Respect the *kernel-rt* package (#600) ## Additional changes interesting for devels - Add a possibility to overwrite virtualenv name using `$VENVNAME` (#613) - Update product certificates for RHEL 8.3 GA and 8.4 Beta/HTB (#624) Related leapp release: https://github.com/oamg/leapp/releases/tag/v0.12.0

pirat89 force-pushed the realtime_support branch 2 times, most recently from 5fcda96 to 08f1248 Compare October 7, 2020 13:23

pirat89 added the enhancement New feature or request label Oct 7, 2020

pirat89 force-pushed the realtime_support branch 2 times, most recently from c00fd40 to 3be1c2d Compare October 7, 2020 14:28

pirat89 marked this pull request as ready for review October 7, 2020 14:28

pirat89 requested review from drehak and zhukovgreen and removed request for drehak October 7, 2020 14:28

pirat89 force-pushed the realtime_support branch from 3be1c2d to 6eb17ce Compare October 7, 2020 14:33

pirat89 force-pushed the realtime_support branch 2 times, most recently from b2ce84d to 1234189 Compare October 8, 2020 13:32

pirat89 changed the title ~~[WIP] realtime: respect kernel-rt rpm~~ realtime: respect kernel-rt rpm Oct 8, 2020

pirat89 requested a review from a team October 15, 2020 10:05

zhukovgreen previously approved these changes Oct 19, 2020

View reviewed changes

pirat89 dismissed zhukovgreen’s stale review via f17a36e October 19, 2020 10:06

pirat89 force-pushed the realtime_support branch from 1234189 to f17a36e Compare October 19, 2020 10:06

pirat89 force-pushed the realtime_support branch from f17a36e to a2fff1e Compare November 4, 2020 09:41

pirat89 force-pushed the realtime_support branch 4 times, most recently from c4f0659 to 45a8eac Compare November 4, 2020 17:45

pirat89 force-pushed the realtime_support branch from 45a8eac to 6c27af2 Compare November 5, 2020 14:17

pirat89 force-pushed the realtime_support branch from 6c27af2 to 8c96037 Compare February 3, 2021 11:28

pirat89 added 3 commits February 3, 2021 14:08

pirat89 force-pushed the realtime_support branch from 8c96037 to 5fe6750 Compare February 3, 2021 13:10

Rezney reviewed Feb 3, 2021

View reviewed changes

drehak mentioned this pull request Feb 4, 2021

Release 0.13.0 #643

Merged

Rezney approved these changes Feb 4, 2021

View reviewed changes

pirat89 merged commit d362e96 into oamg:master Feb 4, 2021

pirat89 deleted the realtime_support branch May 5, 2021 12:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

realtime: respect kernel-rt rpm #600

realtime: respect kernel-rt rpm #600

pirat89 commented Oct 1, 2020 •

edited

Loading

leapp-bot commented Oct 1, 2020

pirat89 commented Oct 7, 2020

pirat89 commented Oct 8, 2020 •

edited

Loading

zhukovgreen left a comment

pirat89 commented Nov 4, 2020

pirat89 commented Nov 4, 2020

mkluson commented Jan 6, 2021

pirat89 commented Feb 3, 2021

pirat89 commented Feb 3, 2021

Rezney Feb 3, 2021

Rezney Feb 3, 2021

Rezney Feb 3, 2021

Rezney Feb 3, 2021

Rezney Feb 3, 2021

Rezney Feb 3, 2021

Rezney Feb 3, 2021

Rezney Feb 3, 2021

pirat89 commented Feb 4, 2021

Rezney left a comment

	# Hypothatical, user is not allowed to install any kernel that is not signed by RH
	# Hypothetically, user is not allowed to install any kernel that is not signed by RH

	# of these during the upgrade. kernel-rt has higher prio when original sys
	# of these during the upgrade. kernel-rt has higher prio when original system

	Currently the check is based on the release of the original booted kernel.
	Currently, the check is based on the release of the originally booted kernel.

	In future, we could detect whether the system is preemptive or not based
	In future, we could detect whether the system is preemptive or not, based



		def get_kernel_rpm_release(rpm):
		def get_EVR(pkg):

realtime: respect kernel-rt rpm #600

realtime: respect kernel-rt rpm #600

Conversation

pirat89 commented Oct 1, 2020 • edited Loading

TODO

Comparison of kernel versions changes

How postupgrade phases are affected

leapp-bot commented Oct 1, 2020

Thank you for contributing to the Leapp project!

pirat89 commented Oct 7, 2020

pirat89 commented Oct 8, 2020 • edited Loading

zhukovgreen left a comment

Choose a reason for hiding this comment

pirat89 commented Nov 4, 2020

pirat89 commented Nov 4, 2020

mkluson commented Jan 6, 2021

pirat89 commented Feb 3, 2021

pirat89 commented Feb 3, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pirat89 commented Feb 4, 2021

Rezney left a comment

Choose a reason for hiding this comment

pirat89 commented Oct 1, 2020 •

edited

Loading

pirat89 commented Oct 8, 2020 •

edited

Loading