Allow unlimited restart attempts for autostart VMs

[pshirshov]

Currently when the VMs initialize during system startup, they may fail because some bridges created by systemd-networkd may be not available yet:

Jun 29 13:44:30 vm systemd[1]: Starting MicroVM 'microvm-test'...
Jun 29 13:44:30 vm systemd[1]: Started MicroVM 'microvm-test'.
Jun 29 13:44:30 vm microvm@microvm-test[11347]: /run/wrappers/bin/qemu-bridge-helper: cannot run `/run/libvirt/nix-helpers/qemu-bridge-helper': No such file or directory
Jun 29 13:44:30 vm microvm@microvm-test[11339]: qemu-system-x86_64: -netdev bridge,id=eth0,br=br-untrusted,helper=/run/wrappers/bin/qemu-bridge-helper: bridge helper failed
Jun 29 13:44:30 vm systemd[1]: microvm@microvm-test.service: Main process exited, code=exited, status=1/FAILURE
Jun 29 13:44:30 vm systemd[1]: microvm@microvm-test.service: Failed with result 'exit-code'.
Jun 29 13:44:31 vm systemd[1]: microvm@microvm-test.service: Scheduled restart job, restart counter is at 5.
Jun 29 13:44:32 vm systemd[1]: Stopped MicroVM 'microvm-test'.
Jun 29 13:44:32 vm systemd[1]: Dependency failed for MicroVM 'microvm-test'.
Jun 29 13:44:32 vm systemd[1]: microvm@microvm-test.service: Job microvm@microvm-test.service/start failed with result 'dependency'.
Jun 29 16:25:23 vm systemd[1]: Starting MicroVM 'microvm-test'...
Jun 29 16:25:23 vm systemd[1]: Started MicroVM 'microvm-test'.

When I retry to launch the service manually in about 30s it launches perfectly.

It would be good to have an option for limitless boot retry attempts.

[astro]

The microvm@.service already includes Restart=always. Is that not sufficient?

You may also extend systemd.services."microvm@".after to solve the actual cause here though automatic restarts should work.

[pshirshov]

The microvm@.service already includes Restart=always. Is that not sufficient?

Seems like it's not, for some reason it makes 3 attempts then stops.

You may also extend systemd.services."microvm@".after to solve the actual cause here though automatic restarts should work.

Could you please give me an idea how? I'm not a nix expert yet :)

[pshirshov]

I believe that's the explanation why just "restart=always" is not enough: https://unix.stackexchange.com/questions/289629/systemd-restart-always-is-not-honored

[pshirshov]

Maybe it's a good idea to allow custom properties in the generated unit files?

[astro]

The second answer in that link explains that it is just rate limit what you are seeing. systemd is going to continue starting/restarting all over again after a while.

[pshirshov]

For some reason that "after a while" never happens. I've set up 3 vms, one failed to start and never re-attempted to start in a couple of weeks. Okay, sometimes systemd is a bit weird.