Aquilon documentation improvements #244

jouvin · 2018-05-24T20:51:39Z

Configuration: add a section on how to do the actual configuration of a host with the template library
- Includes typical content for the main site templates needed
Configuration tutorial reviewed and fixed after an actual execution!
Spelling mistakes fixed

Final content must be adjusted after #87 has been finalized

ned21 · 2018-05-26T18:17:25Z

_aquilon/configuration.md

+
+### Enabling the Template Library
+
+A template library version is enabled in the context of an archetype, using `archetype/loadpath.pan`


This doesn't currently work in any released version of aq code. I think it would be better to document one of the workarounds I describe, e.g. creating a plenary/pan/units.pan or template-king/pan/units.pan that does include archetype/loadpath rather than documenting something that cannot work without applying a patch we are not keen on merging.

@ned21 I discussed this issue with @urbonegi. There is no workaround possible without hacking things... The easiest is to merge the PR solving this properly (quattor/aquilon#87 (comment)). This PR is currently under review and I really hope that MS will agree the last version as it doesn't change anything for MS... The idea is that this doc will be announced once all the pending PRs are merged... In the meantime it is not worst than what existed before (and was basically unusable without the help of experts!).

You could describe using pan/units.pan from either template-king or plenary as a hack but it's a simple and straight-forward one that's easy to follow and can be explained with a one line comment. So I think it's a workaround rather than a hack. The complexity of the broker changes you propose is much much higher and will carry a considerable code maintenance cost.

@ned21 honnestly, I cannot buy that. This is not more complex that what MS implemented with the include_pan option as an attempt to solve the template library problem. Just a few additional lines and one option that could be used for everything specific needed by the template library. I don't want to describe a hack, I want the broker to do as many things as possible for me!

I think we managed to converge... see quattor/aquilon#86

ned21 · 2018-05-26T18:21:06Z

_aquilon/configuration.md

+
+### Defining the site global variables
+
+It is a good practice to declare the main site variables in one template so that it is easy


Why not put these directly into archetype/base.pan rather than a separate file? That gives archetype/base more purpose.

Because these variable are typically things that you want to share between archetypes! I think this is explained, I'll check.

oh, I see - site is a top level directory. That makes sense, thanks for clarifying.

ned21 · 2018-05-26T18:23:56Z

_aquilon/configuration.md

+(the example is for the a repository called `el7x_x86_64`:
+
+    ```pan
+    structure template site/repository/snapshot/el7x_x86_64;


FYI, we keep these inside os/$osname/$osversion rather than using a site/ directory as it keeps them with the other OS config. Not sure if its helpful for us to try and converge practices or not?

In the template library, we tend to use a separate namespace for all the repositories, OS or not... At some point, this is a detail, just need to be consistent. I think this way is more inline with the template library/SCDB way and may be more familiar....

ned21 · 2018-05-26T18:25:42Z

_aquilon/configuration.md

+include 'config/core/base';
+```
+
+This template is calling a template independant of the particular OS name and version to


independant => independent

jouvin · 2018-05-26T20:44:43Z

I think I'm done with this PR and that it is ready for review. I tried to address most of the spellchecker remarks but there are a few ones where I don't know if I need to add the word or if the word I use is inappropriate... And please, review my additions to the dictionary!

jouvin · 2018-05-27T14:58:02Z

Now really fully tested 😄

- Rework the hardware-related sections of the configuration doc to rely on the template library as much as possible - Add a section on configuring the base OS - Fix spelling mistakes in previous mods

- Based on a real execution!

- Include a dictionary update

- Makes easier to copy/paste suggested contents

ned21 · 2018-05-27T20:13:58Z

_aquilon/configuration.md

+* Create the `/var/quattor/cfg/plenary/web_servers/archetype/loadpath.pan` template with the following content:
+
+```pan
+# It is VERY IMPORTANT not to use this template for anything else than the initial


I think it would be good to say why here so people actually understand the consequences of breaking the directive. e.g. I wrote in the example arch:

# First template loaded by an object, before / is populated. This must be a # declaration template because any change to / will be removed by a subsequent # line in the object template.

ned21 · 2018-05-27T20:15:51Z

_aquilon/configuration.md

 they can be reviewed by others and then deploy them into a domain.

 To publish the changes so that it is possible to deploy them later:

 ```bash
-aq publish --sandbox site-init
+aq publish --sandbox tutorial


This goes against what you said at line 506. It's better to s/--sandbox/--branch/ to minimise confusion.

@ned21 in fact, reading the help, I was wondering what is the difference between --sandbox and --branch. I had the feeling there was none... in which case you are right that it would be more consistent to use --sandbox when the value is of the form user/sandbox. Did I understand properly your point? Do you confirm that --sandbox and --branch are synonym for aq publish as the sandbox branch is not prefixed by the user (something I found surprising as I see a risk of conflict)?

- Installation: remove mention of PR now merged - Advanced configuration: remove mention about features bound to archetypes (implementation not really tested, use discouraged)

- Document stage 'previous'

jouvin · 2018-06-02T10:38:34Z

As for me, this PR is ready for merging once quattor/aquilon#87 is merged (should happen soon!). I still have plenty of ideas for useful additions but probably better to do in future PRs as this one is already quite big and good enough to allow somebody to start playing with Aquilon in a useful way...
Review welcome !

- New dictionary entries added

jouvin · 2018-06-03T14:37:51Z

@ned21 please, could you update your review which is currently blocking the PR but based on an earlier version....

ned21 · 2018-06-04T21:17:18Z

Sorry very busy until Thursday, will look then.

ned21

I found a couple of typos.

ned21 · 2018-06-06T17:52:55Z

_aquilon/configuration.md

-Aquilon requires for each archetype, in addition to its `base.pan` template created when the template library
-was imported, a `final.pan` template which will be the very last executed in the host profile. It will be
-created empty at this stage, in the `archetype` directory like `base.pan`:
+Aquilon requires 2 templates for each archetype, located in the `archetype`


Actually we need 3 now we have added declarations.pan so this should be:
"In addition to the declarations.pan template described above, each archetype requires 2 further templates located in a directory called <<name-of-archetype>>/archetype"

ned21 · 2018-06-06T18:01:03Z

_aquilon/configuration.md

+The `pakiti` feature is easily added to the Aquilon database with:
+
+```bash
+aq add feature --feature pakiti --type host --actation dispatch --deactivation reboot --grn test


actation => activation. You might want to consider adding a --comments "This feature deploys the pakiti software" to show how you can annotate your features with a helpful description

ned21 · 2018-06-06T18:01:20Z

_aquilon/configuration.md

+Once added it must be bound to the `web-servers` personality:
+
+```bash
+aq bind_feature --feature pakity --personality test --archetype web_servers


pakity => pakiti

ned21 · 2018-06-06T18:08:37Z

_aquilon/configuration.md

+This feature requires one variable to be added to our site configuration to define the name of the
+Pakiti server. Based on whether the current host is the pakiti server or not, the server or the client will
+be configured. Presently, we will configure the Pakiti client as part of the `test` personality. To achieve
+this, add the following variable definition to `site/config/global_variables.pan`:


Why wouldn't you define this server in the feature template itself? Otherwise you have two files managing a feature which is harder to use.

[Optional/future improvement] You could also use this as an example of how to use service bindings. Instead of binding a feature to a personality, you could define a service called pakiti and then use aq bind_server to set the desired PAKITI_SERVER.

@ned21 thanks for the advice. I have not played with services yet but I will and add a section on them in a future revision. The server cannot be defined in the template because this feature is coming from the template library so is site-agnostic...

ah, OK sorry I missed that bit. But it's still not really the right way to do it: you should have a file called features/pakiti/confg.pan that does:

variable PAKITI_SERVER = 'pakiti.dailyplanet.com'; include "template-library-standard/features/pakiti/config";

so the config is self-contained and not dependent on a single huge file where it's easy to get merge conflicts.

Looking a bit deeper, I see the way the standard library is included means you can't do this. There can be only one file called features/pakiti/config and that's in the TL; if you define another one in template-king it will take precedence and you won't be able to include the TL one.

You can do this with a service template though:

template service/pakiti/client/config; # included automatically by broker when a client is bound to a service variable PAKITI_SERVER = value("/system/services/pakiti/servers/0"); # load feature config from template-library-standard include "features/pakiti/config";

ned21 · 2018-06-06T18:09:34Z

_aquilon/configuration.md

+The new host configuration can now be compiled, published an deployed with the usual commands:
+
+```bash
+aq promote --personality test --archetype web_servers


The change was already promoted at line 934

Good catch! In fact I was not seeing it a couple of days ago thus I added in line 934!

ned21 · 2018-06-06T18:11:38Z

_aquilon/management.md

+use the following command:
+
+```bash
+aq promote --personality test --archetype web_servers


Do an aq show_diff --archetype web_servers --personality test --personality_stage current --other_stage next first to demonstrate how to see what changes are about to be promoted?

I have still a lot to learn about Aquilon advanced features! Adding it!

ned21 · 2018-06-06T18:14:26Z

_aquilon/configuration.md

+
+A template library version is enabled in the context of an archetype, using `archetype/declarations.pan`
+This template is typically placed in the archetype directory for which you want to enable this version
+of the template library. Currently, we'll add it to the plenary templates area, under


s/Currently/Temporarily, for the purposes of this tutorial,/

ned21 · 2018-06-06T19:46:22Z

_aquilon/configuration.md

+
+In our example, we will add the feature `pakiti` to the personality `test` used with the host
+`testsrv.dailyplanet.com`. [Pakiti](https://github.com/CESNET/pakiti-server) is a service to collect and
+display the patch status of hosts against known CVEs. Its configuration is part of the template library,


Can we link to the right bit of the template library here? i.e. https://github.com/quattor/template-library-standard/tree/master/features/pakiti

ned21 · 2018-06-06T19:48:50Z

_aquilon/configuration.md

+This feature requires one variable to be added to our site configuration to define the name of the
+Pakiti server. Based on whether the current host is the pakiti server or not, the server or the client will
+be configured. Presently, we will configure the Pakiti client as part of the `test` personality. To achieve
+this, add the following variable definition to `site/config/global_variables.pan`:


ah, OK sorry I missed that bit. But it's still not really the right way to do it: you should have a file called features/pakiti/confg.pan that does:

variable PAKITI_SERVER = 'pakiti.dailyplanet.com'; include "template-library-standard/features/pakiti/config";

so the config is self-contained and not dependent on a single huge file where it's easy to get merge conflicts.

ned21 · 2018-06-06T19:57:41Z

_aquilon/configuration.md

+This feature requires one variable to be added to our site configuration to define the name of the
+Pakiti server. Based on whether the current host is the pakiti server or not, the server or the client will
+be configured. Presently, we will configure the Pakiti client as part of the `test` personality. To achieve
+this, add the following variable definition to `site/config/global_variables.pan`:


Looking a bit deeper, I see the way the standard library is included means you can't do this. There can be only one file called features/pakiti/config and that's in the TL; if you define another one in template-king it will take precedence and you won't be able to include the TL one.

You can do this with a service template though:

template service/pakiti/client/config; # included automatically by broker when a client is bound to a service variable PAKITI_SERVER = value("/system/services/pakiti/servers/0"); # load feature config from template-library-standard include "features/pakiti/config";

ned21 · 2018-06-06T19:58:56Z

Test failure is not relevant - shall we merge this?

jouvin · 2018-06-06T20:09:37Z

As for me this is ok! Once I progress with the configuration of real production nodes, I'll improve it but will probably not happen before mid-June...

jouvin force-pushed the aquilon_doc branch 4 times, most recently from e01ae35 to 6564688 Compare May 25, 2018 22:58

jouvin mentioned this pull request May 25, 2018

1.12.58: profile prolog template needed to define the LOADPATH quattor/aquilon#86

Closed

jouvin force-pushed the aquilon_doc branch 2 times, most recently from b285056 to a80c7c6 Compare May 26, 2018 17:52

ned21 requested changes May 26, 2018

View reviewed changes

jouvin force-pushed the aquilon_doc branch 2 times, most recently from c72ba1d to 623abd6 Compare May 26, 2018 20:00

jouvin changed the title ~~[WIP] Aquilon documentation improvements~~ Aquilon documentation improvements May 26, 2018

jouvin force-pushed the aquilon_doc branch from b3a3f89 to 2d8276e Compare May 26, 2018 20:43

jouvin force-pushed the aquilon_doc branch 6 times, most recently from 035d8a6 to 4eeb7ad Compare May 27, 2018 14:57

jouvin added 5 commits May 27, 2018 17:03

Aquilon documentation improvements

611984a

- Rework the hardware-related sections of the configuration doc to rely on the template library as much as possible - Add a section on configuring the base OS - Fix spelling mistakes in previous mods

Aquilon configuration: command examples fixed

65ee87a

- Based on a real execution!

Aquilon configuration: initial section on personalities and features

f33f21f

Aquilon contribution clarifications

cfbe4cf

Aquilon Documentation: fix spelling mistakes

0259c4e

- Include a dictionary update

jouvin force-pushed the aquilon_doc branch from 4eeb7ad to 0259c4e Compare May 27, 2018 15:04

jouvin referenced this pull request in ned21/template-library-examples May 27, 2018

Example archetype definition for use with Aquilon

f38982c

Aquilon configuration: turn pan example into bash code

b3c0420

- Makes easier to copy/paste suggested contents

ned21 reviewed May 27, 2018

View reviewed changes

Aquilon configuration: address remarks by @ned21 on May 27

81afcc0

jouvin force-pushed the aquilon_doc branch from e7972a4 to 81afcc0 Compare May 28, 2018 16:05

Aquilon documentation miscellaneous changes

5c59acd

- Installation: remove mention of PR now merged - Advanced configuration: remove mention about features bound to archetypes (implementation not really tested, use discouraged)

jouvin force-pushed the aquilon_doc branch from 307fec5 to 5c59acd Compare May 30, 2018 19:48

jouvin added 5 commits May 30, 2018 22:15

Aquilon doc: loadpath.pan changed to declarations.pan

f6d1737

Aquilon doc: fix description of timezone configuration

858233f

Aquilon doc: misc. change in personality section

ea4f3c1

- Document stage 'previous'

Aquilon doc: misc. change

a7e5a67

Aquilon doc: personality management clarifications

3df85cb

jouvin force-pushed the aquilon_doc branch from 927bcf2 to 3df85cb Compare June 2, 2018 10:34

jouvin added 4 commits June 2, 2018 14:49

article layout: allow the author to be a list

34d42cc

Aquilon doc: update menu entries to better refect contents

2ba664d

Aquilon doc: miscellaneous change in global_variables.pan example

cf48d1d

Aquilon doc: fix spelling mistakes

d11d552

- New dictionary entries added

jouvin force-pushed the aquilon_doc branch from 93b936d to d11d552 Compare June 3, 2018 13:54

Aquilon doc: add a section on template library modification

b5bce86

jouvin force-pushed the aquilon_doc branch from a64e2d5 to b5bce86 Compare June 3, 2018 19:49

Aquilon doc: simpler OS template example

29fc25f

jouvin force-pushed the aquilon_doc branch from ec1dcab to 29fc25f Compare June 4, 2018 11:57

ned21 reviewed Jun 6, 2018

View reviewed changes

ned21 approved these changes Jun 6, 2018

View reviewed changes

Aquilon configuration: address @ned21 remarks on June 6

16fea23

jouvin force-pushed the aquilon_doc branch from a0d35f0 to 16fea23 Compare June 6, 2018 20:08

ned21 merged commit 35fa44b into quattor:master Jun 6, 2018


		### Enabling the Template Library

		A template library version is enabled in the context of an archetype, using `archetype/loadpath.pan`


		### Defining the site global variables

		It is a good practice to declare the main site variables in one template so that it is easy

Aquilon documentation improvements #244

Aquilon documentation improvements #244

Conversation

jouvin commented May 24, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ned21 May 26, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jouvin commented May 26, 2018

jouvin commented May 27, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jouvin commented Jun 2, 2018

jouvin commented Jun 3, 2018

ned21 commented Jun 4, 2018

ned21 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ned21 commented Jun 6, 2018

jouvin commented Jun 6, 2018 • edited Loading

jouvin commented May 24, 2018 •

edited

Loading

ned21 May 26, 2018 •

edited

Loading

jouvin commented Jun 6, 2018 •

edited

Loading