Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change systemd_postun to systemd_postun_restart #937

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

tiggi
Copy link

@tiggi tiggi commented Apr 3, 2024

Change the spec template to call systemd_postun_restart instead of %systemd_postun %{name}.service.

Currently the behavior is that a "yum update" updates the software but never restarts the daemons if they are running,
leaving the old binary to serve out data.
The _restart method changes this to restart the service if it's started.

@tiggi
Copy link
Author

tiggi commented Apr 4, 2024

showcaseing this from this morning (4.4.2024):
`Total download size: 44 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): smokeping_prober-0.8.1-1.el9.x86_64.rpm 1.8 MB/s | 3.6 MB 00:02
(2/2): prometheus2-2.51.1-1.el9.x86_64.rpm 7.6 MB/s | 40 MB 00:05

Total 8.2 MB/s | 44 MB 00:05
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Running scriptlet: smokeping_prober-0.8.1-1.el9.x86_64 1/4
Upgrading : smokeping_prober-0.8.1-1.el9.x86_64 1/4
Running scriptlet: smokeping_prober-0.8.1-1.el9.x86_64 1/4
Running scriptlet: prometheus2-2.51.1-1.el9.x86_64 2/4
Upgrading : prometheus2-2.51.1-1.el9.x86_64 2/4
Running scriptlet: prometheus2-2.51.1-1.el9.x86_64 2/4
Running scriptlet: smokeping_prober-0.8.0-1.el9.x86_64 3/4
Cleanup : smokeping_prober-0.8.0-1.el9.x86_64 3/4
Running scriptlet: smokeping_prober-0.8.0-1.el9.x86_64 3/4
Running scriptlet: prometheus2-2.51.0-1.el9.x86_64 4/4
Cleanup : prometheus2-2.51.0-1.el9.x86_64 4/4
Running scriptlet: prometheus2-2.51.0-1.el9.x86_64 4/4
Verifying : prometheus2-2.51.1-1.el9.x86_64 1/4
Verifying : prometheus2-2.51.0-1.el9.x86_64 2/4
Verifying : smokeping_prober-0.8.1-1.el9.x86_64 3/4
Verifying : smokeping_prober-0.8.0-1.el9.x86_64 4/4

Upgraded:
prometheus2-2.51.1-1.el9.x86_64 smokeping_prober-0.8.1-1.el9.x86_64

Complete!
[tiggi@fire ~]$ ps axuw|grep promet
prometh+ 901 0.1 4.0 2117728 152100 ? Ssl Mar28 15:09 /usr/bin/prometheus --config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/var/lib/prometheus/data --web.console.libraries=/usr/share/prometheus/console_libraries --web.console.templates=/usr/share/prometheus/consoles
prometh+ 915 0.7 0.7 1239296 29096 ? Ssl Mar28 74:08 /usr/bin/snmp_exporter --config.file=/etc/prometheus/snmp.yml
prometh+ 38408 0.7 0.5 1238044 19704 ? Ssl Apr03 9:05 /usr/bin/smokeping_prober --config.file=/etc/prometheus/smokeping_prober.yml
tiggi 39550 0.0 0.0 6408 2176 pts/0 S+ 07:54 0:00 grep --color=auto promet
`
Prometheus process is still the old version (started Mar28), not the upgraded version. With the patch applied prometheus and smokeping_prober would have been restarted to the new binary.

@karlism
Copy link
Collaborator

karlism commented Apr 4, 2024

@tiggi, thanks for your help, but I'm not entirely sure we really want this. I kinda expect that I will need to trigger service restart either manually or by using some automation tools like Ansible when I do software upgrades. It is also sometimes necessary to adjust configuration files or flags after the upgrade and before service restart.

@lest, what's your take on this?

@tiggi
Copy link
Author

tiggi commented Apr 4, 2024

The point is that for daemons you are supposed to use the _restart version so that a yum upgrade tells systemd to restart the daemon.
Else (like in my case from this morning) the old binary keeps running.

Upgrading a package and manually having to restart is not how it's supposed to be done.

Case study: there is a security incident, a new version is pushed out.... but the old exploitable version continues to run even when a check with rpm -q shows that the newest version is installed.

@karlism
Copy link
Collaborator

karlism commented Apr 4, 2024

Upgrading a package and manually having to restart is not how it's supposed to be done.

I'm not entirely sure I agree on this, would you expect package manager to automatically restart PostgreSQL if there is an update installed?

Case study: there is a security incident, a new version is pushed out.... but the old exploitable version continues to run even when a check with rpm -q shows that the newest version is installed.

Same with kernel, generally you need to restart the system manually despite the fact that rpm -q will show that you've got newer version installed.

For what it's worth, I've just upgraded firewalld, docker-ce and openssh-server packages on one Oracle Linux 8 VM, and in all of these cases related services were restarted automatically.

@yashumitsu
Copy link

Just a little note, the correct macro name seems to be:
%systemd_postun_with_restart
https://github.com/systemd/systemd/blob/v255/src/rpm/macros.systemd.in#L88-L94
not:
%systemd_postun_restart

like:

@yashumitsu
Copy link

yashumitsu commented Apr 4, 2024

I'm not entirely sure I agree on this, would you expect package manager to automatically restart PostgreSQL if there is an update installed?

Yes, in critical systems, package updates are not accidental. Package versions are locked using yum versionlock, or repository snapshots are used. Administrators can stop a service if they need to change the configuration before updating. Finally, the system being updated is usually put into maintenance mode, and the services can be restarted any number of times.

However, consider the opposite scenario, let's call it "spontaneous updating" (yum -y update). Here, ensuring system consistency and immediately reloading all updated files is preferable. It's unlikely that anyone would think to check all the updated packages for the need to restart.

Same with kernel, generally you need to restart the system manually despite the fact that rpm -q will show that you've got newer version installed.

I understand your concerns, but the live patching mechanism is becoming more popular in the Linux Kernel. Automatic updates can also trigger a reboot:
reboot: when-needed
https://dnf.readthedocs.io/en/latest/automatic.html#configuration-file-format

So, not everyone is satisfied with the current state of things.

@tiggi
Copy link
Author

tiggi commented Apr 5, 2024

Just a little note, the correct macro name seems to be: %systemd_postun_with_restart https://github.com/systemd/systemd/blob/v255/src/rpm/macros.systemd.in#L88-L94 not: %systemd_postun_restart

like:

Oops. Will fix immediately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants