-
Notifications
You must be signed in to change notification settings - Fork 144
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[inhibitwhenluks] Allow OS upgrade if luks volumes are only Ceph OSDs #735
Conversation
Thank you for contributing to the Leapp project!Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergable.
Please open ticket in case you experience technical problem with the CI. (RH internal only) Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please consider rerunning the CI by commenting leapp-ci build (might require several comments). If the problem persists, contact leapp-infra. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess we need more changes to properly process the Ceph Storage Info, especially in a containerized environment.
|
||
|
||
def run_ceph_lvm_list(): | ||
stream = subprocess.check_output(['ceph-volume','lvm','list','--format','json']) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly of the Ceph deployments are containerized since luminous, which means:
- no ceph-common can be found on the OSD node
- no ceph-volume command can be run from the hosts, but you need to run it using the following approach:
a. sudo podman ps | grep -i ceph
b. sudo podman exec -it ceph-osdX ceph-volume lvm list --format json
I guess some not containerized Ceph deployments still exist, but it's not the OpenStack use case
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, I didn't know about that OSP detail. Usually ceph-ansible installs ceph-common even with containerized setup but as you said it is not always the case. I will also check if it run with docker (RHEL7) or podman (RHEL8)
Thx
Hi @asm0deuz, just looking into the PR we will require more changes as well - e.g. the subprocess library is prohibited to be used in the project unless there is real use-case that is not covered by the |
@pirat89 removed subprocess and using run() instead |
9cbf303
to
17bdbd7
Compare
repos/system_upgrade/common/actors/cephvolumescan/libraries/cephvolumescan.py
Outdated
Show resolved
Hide resolved
repos/system_upgrade/common/actors/cephvolumescan/libraries/cephvolumescan.py
Outdated
Show resolved
Hide resolved
repos/system_upgrade/common/actors/cephvolumescan/libraries/cephvolumescan.py
Outdated
Show resolved
Hide resolved
repos/system_upgrade/common/actors/cephvolumescan/libraries/cephvolumescan.py
Show resolved
Hide resolved
repos/system_upgrade/common/actors/cephvolumescan/libraries/cephvolumescan.py
Outdated
Show resolved
Hide resolved
class Cephvolumescan(Actor): | ||
""" | ||
Retrieves the list of encrypted Ceph OSD | ||
""" | ||
|
||
name = 'cephvolumescan' | ||
consumes = (InstalledRPM,) | ||
produces = (CephInfo,) | ||
tags = (ChecksPhaseTag, IPUWorkflowTag) | ||
|
||
def process(self): | ||
output = cephvolumescan.encrypted_osds_list() | ||
self.produce(CephInfo(encrypted_volumes=output)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note that ceph-volume supports raw encrypted devices [1]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@guits I just compared the output of both commands. While the output layout is different, the content is somewhat the same. Should there be a encrypted raw device, would ceph-volume raw show one additional entry?
# ceph-volume raw list
{
"0": {
"ceph_fsid": "aef0f409-a4ab-4e60-98c4-dae89ea6e6fe",
"device": "/dev/mapper/qzFifc-B8JV-2TeL-VCo7-LmRl-BDLU-Dxb54h",
"osd_id": 0,
"osd_uuid": "37f0e5f1-80a8-4f42-bb4b-569582417d41",
"type": "bluestore"
},
"2": {
"ceph_fsid": "aef0f409-a4ab-4e60-98c4-dae89ea6e6fe",
"device": "/dev/mapper/oSZa3x-70Uw-5ill-DdiW-w2eU-0QNv-zA8ffu",
"osd_id": 2,
"osd_uuid": "4f5d6ba8-f23b-47eb-8518-8ceef3e2327d",
"type": "bluestore"
}
}
# ceph-volume lvm list
====== osd.0 =======
[block] /dev/ceph-5bb030e3-4a14-4963-924d-f285caa882cf/osd-data-2c2ef176-48b2-4a09-854e-a33c7e8b1e88
block device /dev/ceph-5bb030e3-4a14-4963-924d-f285caa882cf/osd-data-2c2ef176-48b2-4a09-854e-a33c7e8b1e88
block uuid qzFifc-B8JV-2TeL-VCo7-LmRl-BDLU-Dxb54h
cephx lockbox secret AQCs8SBhmHVyLRAAVlHxHpLcn15JNO6hC51T5Q==
cluster fsid aef0f409-a4ab-4e60-98c4-dae89ea6e6fe
cluster name ceph
crush device class None
encrypted 1
osd fsid 37f0e5f1-80a8-4f42-bb4b-569582417d41
osd id 0
osdspec affinity
type block
vdo 0
devices /dev/sdb
====== osd.2 =======
[block] /dev/ceph-b5aae797-7c65-4235-8e5f-c8f150db7bce/osd-data-fa384e0d-b4b6-41d1-b69c-da93f15cae54
block device /dev/ceph-b5aae797-7c65-4235-8e5f-c8f150db7bce/osd-data-fa384e0d-b4b6-41d1-b69c-da93f15cae54
block uuid oSZa3x-70Uw-5ill-DdiW-w2eU-0QNv-zA8ffu
cephx lockbox secret AQC88SBh47PoIBAAN+oZxXDHEkLSuYUlnbkjSQ==
cluster fsid aef0f409-a4ab-4e60-98c4-dae89ea6e6fe
cluster name ceph
crush device class None
encrypted 1
osd fsid 4f5d6ba8-f23b-47eb-8518-8ceef3e2327d
osd id 2
osdspec affinity
type block
vdo 0
devices /dev/sdc
When you mean raw device, it is a crypted partition like this?
sde 8:64 0 10G 0 disk
└─sde1 8:65 0 1G 0 part
└─manualluks 253:6 0 1022M 0 crypt
17bdbd7
to
8ab8fc1
Compare
2021-10-23 03:26:59.74 INFO PID: 22389 leapp.workflow.Checks: Executing actor cephvolumescan |
repos/system_upgrade/common/actors/cephvolumescan/libraries/cephvolumescan.py
Show resolved
Hide resolved
#If encrypted Ceph volumes present, check if there are more encrypted disk in lsblk than Ceph vol | ||
ceph_vol = [] | ||
try: | ||
ceph_info = next(self.consume(CephInfo)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To avoid the exception you can also use
ceph_info = next(self.consume(CephInfo)) | |
ceph_info = next(self.consume(CephInfo), None) |
Then ceph_info
will be None and you can conditionalize it over this - ceph_info cannot be None otherwise, since None is not a valid CephInfo
model instance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR has been linked in issue tracker (#OAMG-5854). |
8ab8fc1
to
68acef5
Compare
@asm0deuz Could you please do a rebase and fix the linter failures? Thanks... |
68acef5
to
7048269
Compare
@Rezney changes done. Sorry for the late reply, I didn't notice your update |
@asm0deuz no worries, thanks for the rebase. Could you please also fix the linter issues in the unit tests? |
7048269
to
7f64f73
Compare
@asm0deuz still some flake issues. Please take a look... |
7f64f73
to
ed5c073
Compare
ed5c073
to
c6bd09d
Compare
@asm0deuz As discussed, please add also unit tests. Thanks... |
|
||
def run_ceph_lvm_list(): | ||
command = container_or_rpm() | ||
output = run(command) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@asm0deuz are you sure the command will never end with non-zero exit code? if not, it should be handled nicely so customers cannot see just traceback. (except CalledProcessError
). See the StopActorExecutionError
exception to raise a nice error for leapp in such a case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 to @pirat89 , please wrap into try\except
@asm0deuz Hi, do you have any update regarding the ticket? Just asking... |
Still need to write the unit tests. I will do it this weekend. Sorry for the delay. Pirat89 comment fixed |
eb92aae
to
701ec20
Compare
339fa5b
to
d57f4ab
Compare
@asm0deuz
|
13eb204
to
c1a27ec
Compare
@Rezney Done! |
Please run also |
c1a27ec
to
072f434
Compare
Done, I hope it is good now. |
I tested the following functionalities on my lab with success:
Hope you can merge this PR. Thx |
The presence of even one encryted disks prevents the OS update. This addition permits the upgrade if all the disks are Ceph encrypted OSDs as it shouldn't impact the OS update Signed-off-by: Teoman ONAY <tonay@redhat.com>
072f434
to
db49e21
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you very much for your work @asm0deuz
## Packaging - Provide and require leapp-repository-dependencies 7 (oamg#952) - Provide `leapp-command(<CMD>)` for each CLI command provided by leapp-repository (oamg#947) - Require dracut, kmod, procps-ng on RHEL 8+ (oamg#952) - Require leapp-framework >= 3.1 (oamg#905, oamg#927) ## Upgrade handling ### Fixes - Do not create the upgrade bootloader entry when the dnf dry-run actor fails (oamg#912) - Do not inhibit in-place upgrades in case LUKS volumes are Ceph OSDs (oamg#735) - Fix & improve application of custom selinux rules to be less error prone and do not override changes done by RPM scriptlets (oamg#925) - Fix detection of deprecated devices (and drivers) regarding the PCI address (oamg#881) - Fix detection of deprecated kernel modules (oamg#874) - Fix the false positive NFS storage detection on NFS servers (oamg#888) - Fix the issues on systems with the LANGUAGE environment variable (oamg#887) - Fix the root directory scan to deal with non-utf8 filenames (oamg#927) - Skip comment lines when parsing the GRUB configuration file (oamg#883) - Stop propagating the “debug” and ”enforcing=0” kernel cmdline options into the target kernel cmdline options (oamg#938, oamg#950) - [IPU 7 -> 8] Fix the upgrade of the Satellite server (oamg#875, oamg#878, oamg#879 oamg#890, oamg#899, oamg#916, 934) - [IPU 7 -> 8] Fix SSSD: Prune old cache files (the format of data is incompatible) (oamg#922) - [IPU 8 -> 9] Enable the CRB repository for the upgrade only if enabled on the source system (oamg#942) - [IPU 8 -> 9] Drop obsoleted actor blocking upgrade on z16 (oamg#892) - [IPU 8 -> 9] Fix cloud provider detection on AWS (oamg#920) - [IPU 8 -> 9] Fix detention of the latest kernel on RHEL 8+ systems (oamg#909) - [IPU 8 -> 9] Fix issues caused by leapp artifacts from previous in-place upgrades (oamg#889) - [IPU 8 -> 9] Fix issues with false positive switch to emergency console during the upgrade (oamg#906) - [IPU 8 -> 9] Fix swap page size on aarch64 (oamg#937, oamg#948) - [IPU 8 -> 9] Fix the VDO scanner to skip partitions unrelated to VDO and adjust error messages (oamg#919) ### Enhancements - Add 8.7 & 9.1 Beta & GA product certificates (oamg#891) - Detect /var/lib/leapp being mounted in a non-persistent fashion (oamg#921) - Detect /var/lib/leapp mounted with the noexec option (oamg#908) - Improve the report msg when NFS partitions are discovered providing info about concrete mountpoints (oamg#806) - Inform about necessary migrations related to bacula-director (oamg#896) - [IPU 7 -> 8] The default upgrade path for RHEL SAP is 7.9 -> 8.6 (oamg#939) - [IPU 7 -> 8] Detect and fix missing newline at the end of /etc/default/grub (oamg#945) - [IPU 7 -> 8] Handle upgrades of SAP Apps systems on Azure (oamg#926) - [IPU 7 -> 8] Handle upgrades on RHUI Google Cloud (oamg#897, oamg#946) - [IPU 8 -> 9] Support upgrade path RHEL 8.7 -> 9.0 and RHEL SAP 8.6 -> 9.0 (oamg#903, oamg#894) - [IPU 8 -> 9] Add actors covering removal of NIS components on RHEL 9 (oamg#851) - [IPU 8 -> 9] Add checks for obsolete .NET versions (oamg#867) - [IPU 8 -> 9] Allow specifying the report schema v1.2.0 (oamg#872) - [IPU 8 -> 9] Check and handle upgrades with custom crypto policies (oamg#898) - [IPU 8 -> 9] Check and migrate OpenSSH configuration (oamg#864, oamg#860) - [IPU 8 -> 9] Check and migrate multipath configuration the upgrade (oamg#886) - [IPU 8 -> 9] Check minimum memory requirements (oamg#935) - [IPU 8 -> 9] Enable Base and SAP In-place upgrades on Azure (oamg#943) - [IPU 8 -> 9] Enable in-place upgrades in Azure RHEL 8 base images using RHUI (oamg#918) - [IPU 8 -> 9] Handle upgrades of SAP systems on AWS (oamg#924) - [IPU 8 -> 9] Inhibit upgrade when NVIDIA driver is detected (oamg#880) - [IPU 8 -> 9] Migrate blocklisted CAs (oamg#882) - [IPU 8 -> 9] Migrate the OpenSSL configuration (oamg#900) - [IPU 8 -> 9] Report changes around SCP and SFTP (oamg#863, oamg#893) ## Additional changes interesting for devels - Extend LsblkEntry model in StorageInfo by kernel name and size of partition in bytes (oamg#919) - Mass refactoring: Fix imports in actors and libraries to follow project guidelines (oamg#932) - Mass refactoring: Replace use of deprecated `reporting.(Tags|Flags)` by `reporting.Groups` (oamg#932) - PESEventScanner actor has been fully refactored (oamg#856, oamg#941) - Use library function is_inhibitor to check for failures (oamg#905) Signed-off-by: Petr Stodulka <pstodulk@redhat.com>
## Packaging - Provide and require leapp-repository-dependencies 7 (#952) - Provide `leapp-command(<CMD>)` for each CLI command provided by leapp-repository (#947) - Require dracut, kmod, procps-ng on RHEL 8+ (#952) - Require leapp-framework >= 3.1 (#905, #927) ## Upgrade handling ### Fixes - Do not create the upgrade bootloader entry when the dnf dry-run actor fails (#912) - Do not inhibit in-place upgrades in case LUKS volumes are Ceph OSDs (#735) - Fix & improve application of custom selinux rules to be less error prone and do not override changes done by RPM scriptlets (#925) - Fix detection of deprecated devices (and drivers) regarding the PCI address (#881) - Fix detection of deprecated kernel modules (#874) - Fix the false positive NFS storage detection on NFS servers (#888) - Fix the issues on systems with the LANGUAGE environment variable (#887) - Fix the root directory scan to deal with non-utf8 filenames (#927) - Skip comment lines when parsing the GRUB configuration file (#883) - Stop propagating the “debug” and ”enforcing=0” kernel cmdline options into the target kernel cmdline options (#938, #950) - [IPU 7 -> 8] Fix the upgrade of the Satellite server (#875, #878, #879 #890, #899, #916, 934) - [IPU 7 -> 8] Fix SSSD: Prune old cache files (the format of data is incompatible) (#922) - [IPU 8 -> 9] Enable the CRB repository for the upgrade only if enabled on the source system (#942) - [IPU 8 -> 9] Drop obsoleted actor blocking upgrade on z16 (#892) - [IPU 8 -> 9] Fix cloud provider detection on AWS (#920) - [IPU 8 -> 9] Fix detention of the latest kernel on RHEL 8+ systems (#909) - [IPU 8 -> 9] Fix issues caused by leapp artifacts from previous in-place upgrades (#889) - [IPU 8 -> 9] Fix issues with false positive switch to emergency console during the upgrade (#906) - [IPU 8 -> 9] Fix swap page size on aarch64 (#937, #948) - [IPU 8 -> 9] Fix the VDO scanner to skip partitions unrelated to VDO and adjust error messages (#919) ### Enhancements - Add 8.7 & 9.1 Beta & GA product certificates (#891) - Detect /var/lib/leapp being mounted in a non-persistent fashion (#921) - Detect /var/lib/leapp mounted with the noexec option (#908) - Improve the report msg when NFS partitions are discovered providing info about concrete mountpoints (#806) - Inform about necessary migrations related to bacula-director (#896) - [IPU 7 -> 8] The default upgrade path for RHEL SAP is 7.9 -> 8.6 (#939) - [IPU 7 -> 8] Detect and fix missing newline at the end of /etc/default/grub (#945) - [IPU 7 -> 8] Handle upgrades of SAP Apps systems on Azure (#926) - [IPU 7 -> 8] Handle upgrades on RHUI Google Cloud (#897, #946) - [IPU 8 -> 9] Support upgrade path RHEL 8.7 -> 9.0 and RHEL SAP 8.6 -> 9.0 (#903, #894) - [IPU 8 -> 9] Add actors covering removal of NIS components on RHEL 9 (#851) - [IPU 8 -> 9] Add checks for obsolete .NET versions (#867) - [IPU 8 -> 9] Allow specifying the report schema v1.2.0 (#872) - [IPU 8 -> 9] Check and handle upgrades with custom crypto policies (#898) - [IPU 8 -> 9] Check and migrate OpenSSH configuration (#864, #860) - [IPU 8 -> 9] Check and migrate multipath configuration the upgrade (#886) - [IPU 8 -> 9] Check minimum memory requirements (#935) - [IPU 8 -> 9] Enable Base and SAP In-place upgrades on Azure (#943) - [IPU 8 -> 9] Enable in-place upgrades in Azure RHEL 8 base images using RHUI (#918) - [IPU 8 -> 9] Handle upgrades of SAP systems on AWS (#924) - [IPU 8 -> 9] Inhibit upgrade when NVIDIA driver is detected (#880) - [IPU 8 -> 9] Migrate blocklisted CAs (#882) - [IPU 8 -> 9] Migrate the OpenSSL configuration (#900) - [IPU 8 -> 9] Report changes around SCP and SFTP (#863, #893) ## Additional changes interesting for devels - Extend LsblkEntry model in StorageInfo by kernel name and size of partition in bytes (#919) - Mass refactoring: Fix imports in actors and libraries to follow project guidelines (#932) - Mass refactoring: Replace use of deprecated `reporting.(Tags|Flags)` by `reporting.Groups` (#932) - PESEventScanner actor has been fully refactored (#856, #941) - Use library function is_inhibitor to check for failures (#905) Signed-off-by: Petr Stodulka <pstodulk@redhat.com>
The presence of even one encryted disks prevents
the OS update. This addition permits the upgrade
if all the disks are Ceph encrypted OSDs as it
shouldn't impact the OS update
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1885335
Signed-off-by: Teoman ONAY tonay@redhat.com