Appliance systemd overhaul #1300

ghost · 2018-01-09T21:45:14Z

Fixes #1215, fixes #1265, fixes #1267, and improves the reliability of our systemd configuration.

This change introduces a new systemd target, psc-ready.target, that component services relying on psc tokens may use as a Requires dependency.

A new service, vic-appliance-wait-psc-config.service, has be been introduced that mimics the functionality of systemd's wait-network-online, where the waiting unit is triggered as active when a psc configuration is present. This replaces the use of path files in the vic 1.3 ova.

Changes also introduce another target, vic-appliance.target that is used as the boot-level target as a systemd best practice.

The final change is the introduction of reconfigure_token.service, a script that restarts admiral and harbor if there are any changes to the psc configuration file. This is most useful during re-initialization, where altered config options need to be propagated to admiral and harbor.

Minor unit name fixes Revert bad changes to component services

Fix typo

Update test markdown Fix bad test variable name Fix broken keyword Add extra time for selenium tests Bump wait time some more Sleep before adding default users

Dont block on harbor/admiral restart Final fixes for startup units

Decrease harbor and admiral test timeouts

ghost · 2018-01-09T21:51:18Z

Opening this after I've closed #1266 to force drone to refresh merge refs, and to provide a clean slate for final review.

mdharamadas1

Couple of suggestions on tests.

mdharamadas1 · 2018-01-09T21:57:27Z

tests/test-cases/Group3-Admiral/3-01-Admiral-UI.robot

@@ -27,6 +27,15 @@ ${cp-card-status-stopped}  STOPPED

 *** Keywords ***
 Setup Base State
+    Log To Console  \nWaiting for Admiral to come up...


Here is the suggestion so that we don't have to repeat this in every suite file. Convert this new code block to a new keyword in OVA-Util.robot and call it inOVA-Setup.robot after line 29.

Done, good idea, thanks!

mdharamadas1 · 2018-01-09T21:59:06Z

tests/resources/Util.robot

@@ -91,8 +91,7 @@ Global Setup With Complete OVA Installation
    # UI tests variables
    Set Global Variable  ${FIREFOX_BROWSER}  firefox
    Set Global Variable  ${GRID_URL}  http://127.0.0.1:4444/wd/hub
-    Set Global Variable  ${EXPLICIT_WAIT}  30
-    Set Global Variable  ${EXTRA_EXPLICIT_WAIT}  50
+    Set Global Variable  ${EXPLICIT_WAIT}  60


Lets keep EXPLICIT_WAIT to 30 as its used commonly, instead update existing EXTRA_EXPLICIT_WAIT to 60 and use it.

anchal-agrawal

PSC changes look good - could you verify that Admiral and Harbor start successfully (w.r.t. PSC) in the following situations:

The OVA is restarted
The appliance re-initialize button is used

We should ensure that the get_token service isn't affected, the Admiral page shows any older data if present, and service startup dependencies w.r.t. PSC are honored.

Also, is this change intended for 1.3.1 too?

ghost · 2018-01-16T19:24:55Z

@anchal-agrawal Thanks, I'll take a look at a restarted ova. I've verified the other scenarios.
Also, to my knowledge, not intended for 1.3.1.

andrewtchin · 2018-01-16T19:27:15Z

definitely not in 1.3.1

ghost · 2018-01-16T19:56:53Z

@anchal-agrawal I've verified Harbor and Admiral start properly after a reboot. Get_token starts when the system boots, because it's needed by the psc-ready target. Token_reconfigure does not run on boot, as expected, so the token is valid. Looks good 👍

anchal-agrawal

PSC changes LGTM - nice work!

andrewtchin · 2018-01-17T01:13:47Z

Looks good after reinit

root@Photon [ ~ ]# systemctl list-jobs
 JOB UNIT                      TYPE    STATE
1400 reconfigure_token.service start   running
1679 harbor.service            restart running
1669 admiral.service           restart waiting
1578 get_token.service         restart waiting
1670 psc-ready.target          restart waiting

root@Photon [ ~ ]# systemctl list-jobs
 JOB UNIT                      TYPE    STATE
1400 reconfigure_token.service start   running
1679 harbor.service            start   waiting
1669 admiral.service           restart running
1578 get_token.service         restart waiting
1670 psc-ready.target          restart waiting

root@Photon [ ~ ]# systemctl list-jobs
 JOB UNIT                      TYPE  STATE
1400 reconfigure_token.service start running
1679 harbor.service            start waiting
1669 admiral.service           start waiting
1578 get_token.service         start running
1670 psc-ready.target          start waiting

root@Photon [ ~ ]# systemctl list-jobs
 JOB UNIT            TYPE    STATE
1679 harbor.service  start   waiting
1669 admiral.service restart running

root@Photon [ ~ ]# systemctl list-jobs
 JOB UNIT           TYPE  STATE
1679 harbor.service start running

andrewtchin · 2018-01-17T01:15:48Z

Looks good after reboot

root@Photon [ ~ ]# systemctl list-jobs
JOB UNIT                 TYPE  STATE
117 get_token.service    start running
  1 vic-appliance.target start waiting
114 harbor.service       start waiting
113 fileserver.service   start running
115 psc-ready.target     start waiting
118 admiral.service      start waiting

root@Photon [ ~ ]# systemctl list-jobs
JOB UNIT                 TYPE  STATE
  1 vic-appliance.target start waiting
114 harbor.service       start waiting
113 fileserver.service   start running
118 admiral.service      start running

root@Photon [ ~ ]# systemctl list-jobs
JOB UNIT                 TYPE  STATE
  1 vic-appliance.target start waiting
114 harbor.service       start waiting
118 admiral.service      start running

root@Photon [ ~ ]# systemctl list-jobs
JOB UNIT                 TYPE  STATE
  1 vic-appliance.target start waiting
114 harbor.service       start running

andrewtchin · 2018-01-17T01:52:19Z

installer/build/scripts/appliance/psc/reconfigure_token.path

@@ -0,0 +1,9 @@
+[Unit]
+Description=Reload PSC configuration.


nit but maybe this can be something like "Watch Admiral psc-config.properties for PSC token refresh"

andrewtchin

awesome :-)

Jason Morris and others added 13 commits January 9, 2018 11:55

Use a new default target

96589dd

Split appliance target into appliance-ready and psc-ready targets

6e6996c

Fix vic environment service

3d51f28

Minor unit name fixes Revert bad changes to component services

Fix missing net-tools package

e1ce3fc

Cherry pick fileserver changes from old branch

ff1a5d0

Fix typo

Ensure network connectivity before get_token

76e26af

Test changes to wait for systemd services

2e95eef

Update test markdown Fix bad test variable name Fix broken keyword Add extra time for selenium tests Bump wait time some more Sleep before adding default users

Bump timeout to 5 minutes

3778601

Restart admiral/harbor after psc changes

c09fd2e

Dont block on harbor/admiral restart Final fixes for startup units

Verify admiral is up before running selenium tests

61db16f

Intorduce psc reconfigure path

5a4181d

Add sso wait to harbor tests

a36be1f

Decrease harbor and admiral test timeouts

Use OVA_IP variable in tests instead of BASE-URL

6787898

ghost requested review from andrewtchin, anchal-agrawal and mdharamadas1 January 9, 2018 21:45

vmwclabot added the cla-not-required label Jan 9, 2018

mdharamadas1 suggested changes Jan 9, 2018

View reviewed changes

ghost mentioned this pull request Jan 10, 2018

OVA longevity issues #1290

Closed

mdubya66 mentioned this pull request Jan 10, 2018

VIC appliance webserver slow to start #1179

Closed

1 task

Add pr comments for test updates

311413b

mdharamadas1 approved these changes Jan 10, 2018

View reviewed changes

Jason Morris added 3 commits January 10, 2018 16:42

Run getting started registration during ova setup

21341f9

Add wait for online services in ova setup

e8a3972

Add final log messages

6af5d48

anchal-agrawal reviewed Jan 16, 2018

View reviewed changes

anchal-agrawal approved these changes Jan 16, 2018

View reviewed changes

Add get_token.service restart upon appliance reinit

56bca58

andrewtchin reviewed Jan 17, 2018

View reviewed changes

andrewtchin approved these changes Jan 17, 2018

View reviewed changes

ghost merged commit c646368 into vmware:master Jan 17, 2018

ghost deleted the fix/systemd-overhaul branch January 17, 2018 02:52

andrewtchin mentioned this pull request Jan 26, 2018

OVA: systemd service admiral_startup runs twice #442

Closed

anchal-agrawal mentioned this pull request Jan 31, 2018

Document how to register VIC Appliance with a new PSC and install/upgrade plug-ins after deployment #644

Closed

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Appliance systemd overhaul #1300

Appliance systemd overhaul #1300

ghost commented Jan 9, 2018 •

edited by ghost

Loading

ghost commented Jan 9, 2018

mdharamadas1 left a comment

mdharamadas1 Jan 9, 2018

ghost Jan 10, 2018

mdharamadas1 Jan 9, 2018

ghost Jan 10, 2018

anchal-agrawal left a comment •

edited

Loading

ghost commented Jan 16, 2018

andrewtchin commented Jan 16, 2018

ghost commented Jan 16, 2018 •

edited by ghost

Loading

anchal-agrawal left a comment

andrewtchin commented Jan 17, 2018

andrewtchin commented Jan 17, 2018

andrewtchin Jan 17, 2018

andrewtchin left a comment

Appliance systemd overhaul #1300

Appliance systemd overhaul #1300

Conversation

ghost commented Jan 9, 2018 • edited by ghost Loading

ghost commented Jan 9, 2018

mdharamadas1 left a comment

Choose a reason for hiding this comment

mdharamadas1 Jan 9, 2018

Choose a reason for hiding this comment

ghost Jan 10, 2018

Choose a reason for hiding this comment

mdharamadas1 Jan 9, 2018

Choose a reason for hiding this comment

ghost Jan 10, 2018

Choose a reason for hiding this comment

anchal-agrawal left a comment • edited Loading

Choose a reason for hiding this comment

ghost commented Jan 16, 2018

andrewtchin commented Jan 16, 2018

ghost commented Jan 16, 2018 • edited by ghost Loading

anchal-agrawal left a comment

Choose a reason for hiding this comment

andrewtchin commented Jan 17, 2018

andrewtchin commented Jan 17, 2018

andrewtchin Jan 17, 2018

Choose a reason for hiding this comment

andrewtchin left a comment

Choose a reason for hiding this comment

ghost commented Jan 9, 2018 •

edited by ghost

Loading

anchal-agrawal left a comment •

edited

Loading

ghost commented Jan 16, 2018 •

edited by ghost

Loading