[docs] Manually migrating Kibana index #13696

tylersmalley · 2017-08-24T21:20:36Z

Documents the process for manually migrating the Kibana index in preparation for a migration to 6.x.

jbudz · 2017-08-24T21:29:25Z

docs/migration/migrate_6_0.asciidoc

-
-[float]
-=== Replace markdown parser `marked` with `markdown-it`
-*Details:* Starting in 6.0.0, Kibana will use `markdown-it` to parse markdown text. Kibana switched to `markdown-it` because `marked` is no longer actively maintained. Markdown-it supports CommonMark and GFM (GitHub Flavored Markdown) Tables and Strikethrough.


we probably want this still?

Yes, not sure what happened there

jbudz · 2017-08-24T21:34:52Z

docs/migration/migrate_6_0_index.asciidoc

+
+A UI is available in X-Pack 5.6 to assist with the migration. The following is provided should you be required to run the migration manually. The commands assume your using the default `.kibana` index. If `kibana.index` is set to something different in your `kibana.yml` you will need to modify the commands.
+
+To preform this migration without downtime, you will need to be running Kibana 5.6. This is the only version which supports both of the index types.


preform = perform

not saying we want to go down this path but a no downtime upgrade without 5.6 could be to stand up two kibana's, one to the reindexed .kibana and update A records to the new IP after

Yeah, that would be pretty tough to time correctly.

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

jbudz · 2017-08-24T21:45:21Z

docs/migration/migrate_6_0.asciidoc

+=== Types removed from Kibana index
+*Details:* With the imminent {es-ref}/removal-of-types.html[removal of types in Elasticsearch], we have updated our index to not rely on types. Kibana 5.6.x can support both of these mappings to minimize downtime. A UI is provided with X-Pack to assist with the migration of the Elastic stack.
+
+*Impact:* You will need migrate your Kibana index either {kibana-ref}/migrating-6.0-index.html[manually], or through the Migration UI in X-Pack.


is there a link a more info link to the migration ui available?

I believe @archanid is still working on that. We should link when it becomes available.

I think it shouldn't be called "Migration UI" here--we're calling it "Upgrade Assistant"

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

jbudz · 2017-08-25T14:47:17Z

docs/migration/migrate_6_0_index.asciidoc

+  "settings" : {
+    "number_of_shards" : 1,
+    "index.format": 6,
+    "index.refresh_interval": -1


we probably want to turn this back on after, right?

Actually, I am going to remove it. @archanid, is the upgrade UI turning this back on after the migration?

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

tylersmalley · 2017-08-25T14:56:14Z

@debadair, did you want to take a pass over this?

rhoboat

I left a few comments, including a small typo fix.

rhoboat · 2017-08-29T22:06:28Z

docs/migration/migrate_6_0.asciidoc

+=== Types removed from Kibana index
+*Details:* With the imminent {es-ref}/removal-of-types.html[removal of types in Elasticsearch], we have updated our index to not rely on types. Kibana 5.6.x can support both of these mappings to minimize downtime. A UI is provided with X-Pack to assist with the migration of the Elastic stack.
+
+*Impact:* You will need migrate your Kibana index either {kibana-ref}/migrating-6.0-index.html[manually], or through the Migration UI in X-Pack.


I think it shouldn't be called "Migration UI" here--we're calling it "Upgrade Assistant"

rhoboat · 2017-08-29T22:07:36Z

docs/migration/migrate_6_0_index.asciidoc

+
+[IMPORTANT]
+==============================================
+The following will make transformations to your Kibana index. It is recommended you backup your Kibana index prior to running any of the following commands.


Maybe a link to the snapshot/restore docs where you say "backup your Kibana index" ?

https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html

rhoboat · 2017-08-29T22:08:20Z

docs/migration/migrate_6_0_index.asciidoc

+The following will make transformations to your Kibana index. It is recommended you backup your Kibana index prior to running any of the following commands.
+==============================================
+
+In order to migrate to Kibana 6, your Kibana index needs to be re-indexed. This will make transformations to the data model, removing types which are being {es-ref}/removal-of-types.html[removed in Elasticsearch].


I think in other docs, we aren't hyphenating "reindex", so maybe being consistent here would be good.

rhoboat · 2017-08-29T22:08:58Z

docs/migration/migrate_6_0_index.asciidoc

+
+In order to migrate to Kibana 6, your Kibana index needs to be re-indexed. This will make transformations to the data model, removing types which are being {es-ref}/removal-of-types.html[removed in Elasticsearch].
+
+A UI is available in X-Pack 5.6 to assist with the migration. The following is provided should you be required to run the migration manually. The commands assume your using the default `.kibana` index. If `kibana.index` is set to something different in your `kibana.yml` you will need to modify the commands.


The commands assume your using => The commands assume you're using

rhoboat · 2017-08-29T22:09:47Z

docs/migration/migrate_6_0_index.asciidoc

+// CONSOLE
+--
+
+. Re-index `.kibana` into `.kibana-6`:


Re-index => Reindex perhaps

rhoboat · 2017-08-29T22:11:05Z

docs/migration/migrate_6_0_index.asciidoc

+  "actions" : [
+    { "add":  { "index": ".kibana-6", "alias": ".kibana" } },
+    { "remove_index": { "index": ".kibana" } }
+  ]


It looks like this command will wipe out any existing aliases on the old kibana index. Right?

Here's where we're adding the old aliases to the new index: https://github.com/elastic/x-pack-kibana/blob/master/plugins/upgrade/public/lib/util/index.js#L77-L87

I 38cc25f I have mentioned not using an alias. I would rather keep the commands simple for majority of users and call out the cases where folks have complex setups.

rhoboat · 2017-08-30T02:43:31Z

docs/migration/migrate_6_0_index.asciidoc

+      }
+    }
+  }
+}


This part kind of bothers me. Putting the whole mappings here feels like there could be a typo, or some change to it later (maybe not realistic), but it means it's in more than one place. Is it worth it to link to a mappings object as a central source of truth?

I believe we could include it, but we would need the whole mapping which I believe this is the first in the OS repo.

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

tylersmalley · 2017-08-30T22:31:19Z

@archanid, thanks for the feedback. I have addressed it - mind giving another look?

rhoboat · 2017-08-31T02:53:28Z

Other than that one comma thingy, it LGTM!

rhoboat · 2017-08-31T03:31:15Z

Also @tylersmalley to help you link to the Upgrade Assistant docs:

The file is docs/en/upgrade-assistant/index.asciidoc

But there are these tags at the top of it:

[role="xpack"]
[[xpack-upgrade-assistant]]

I don't know what that means--maybe the link is actually xpack-upgrade-assistant rather than upgrade-assistant/index.asciidoc. You can see what I'm talking about on @debadair's PR https://github.com/elastic/x-pack-kibana/pull/2452

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

tylersmalley · 2017-08-31T03:52:53Z

6.1/6.x: 220b39f

tylersmalley · 2017-08-31T03:53:29Z

I will follow-up with adding the link once the other PR is merged. I also need to change the xpack-ref to allow defining the branch, like the other refs.

epixa · 2017-09-01T13:17:55Z

@tylersmalley I think we'll want this content in master as well, even if it is slightly modified. These steps do not disappear with 7.0 as users will still need to do this reindex if they are beginning in 5.x and trying to go to 7. I know we're a long ways out from dealing directly with upgradability to 7.0, but we should try to keep the docs as current as possible in the meantime.

tylersmalley · 2017-09-05T16:13:36Z

@epixa, is that something we really want to support? With Elasticsearch, don't users need to go through each of the major versions when upgrading?

epixa · 2017-09-05T18:26:25Z

@tylersmalley That's a good point

tylersmalley added Team:Docs review v6.0.0 v6.1.0 labels Aug 24, 2017

tylersmalley requested review from jbudz and rhoboat August 24, 2017 21:20

tylersmalley force-pushed the 6-zero-migration-docs branch from fcc4c31 to 1fafcf4 Compare August 24, 2017 21:23

tylersmalley mentioned this pull request Aug 24, 2017

Document manual reindexing of .kibana for users without X-Pack elastic/stack-docs#19

Closed

jbudz reviewed Aug 24, 2017

View reviewed changes

tylersmalley mentioned this pull request Aug 24, 2017

6.0 Upgrade Project elastic/stack-docs#15

Closed

24 tasks

tylersmalley force-pushed the 6-zero-migration-docs branch from 1fafcf4 to 7deb526 Compare August 24, 2017 21:31

jbudz reviewed Aug 24, 2017

View reviewed changes

[docs] Manually migrating Kibana index

ef0857d

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

tylersmalley force-pushed the 6-zero-migration-docs branch from 7deb526 to ef0857d Compare August 24, 2017 21:43

jbudz reviewed Aug 24, 2017

View reviewed changes

Added refresh_interval

d0c59b8

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

jbudz reviewed Aug 25, 2017

View reviewed changes

Reverts refresh_interval

5d990c7

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

jbudz approved these changes Aug 25, 2017

View reviewed changes

tylersmalley added the Team:Operations Team label for Operations Team label Aug 29, 2017

rhoboat suggested changes Aug 29, 2017

View reviewed changes

rhoboat reviewed Aug 30, 2017

View reviewed changes

Tyler Smalley added 2 commits August 30, 2017 15:10

Feedback

2be3db7

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

Mention not using an alias

38cc25f

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

rhoboat approved these changes Aug 31, 2017

View reviewed changes

Adds a comma

e4aee8d

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

tylersmalley merged commit 39cab76 into elastic:6.0 Aug 31, 2017

tylersmalley added a commit that referenced this pull request Aug 31, 2017

[docs] Manually migrating Kibana index (#13696)

220b39f

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

jimgoodwin added the v6.0.0-rc1 label Sep 28, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[docs] Manually migrating Kibana index #13696

[docs] Manually migrating Kibana index #13696

tylersmalley commented Aug 24, 2017 •

edited

Loading

jbudz Aug 24, 2017

tylersmalley Aug 24, 2017

jbudz Aug 24, 2017

jbudz Aug 24, 2017

tylersmalley Aug 24, 2017

jbudz Aug 24, 2017

tylersmalley Aug 24, 2017

rhoboat Aug 29, 2017

jbudz Aug 25, 2017

tylersmalley Aug 25, 2017

tylersmalley commented Aug 25, 2017

rhoboat left a comment

rhoboat Aug 29, 2017

rhoboat Aug 29, 2017 •

edited

Loading

rhoboat Aug 29, 2017

rhoboat Aug 29, 2017

rhoboat Aug 29, 2017

rhoboat Aug 29, 2017

rhoboat Aug 29, 2017

tylersmalley Aug 30, 2017 •

edited

Loading

rhoboat Aug 30, 2017

tylersmalley Aug 30, 2017

tylersmalley commented Aug 30, 2017

rhoboat commented Aug 31, 2017

rhoboat commented Aug 31, 2017

tylersmalley commented Aug 31, 2017

tylersmalley commented Aug 31, 2017

epixa commented Sep 1, 2017

tylersmalley commented Sep 5, 2017

epixa commented Sep 5, 2017


		A UI is available in X-Pack 5.6 to assist with the migration. The following is provided should you be required to run the migration manually. The commands assume your using the default `.kibana` index. If `kibana.index` is set to something different in your `kibana.yml` you will need to modify the commands.

		To preform this migration without downtime, you will need to be running Kibana 5.6. This is the only version which supports both of the index types.


		In order to migrate to Kibana 6, your Kibana index needs to be re-indexed. This will make transformations to the data model, removing types which are being {es-ref}/removal-of-types.html[removed in Elasticsearch].

		A UI is available in X-Pack 5.6 to assist with the migration. The following is provided should you be required to run the migration manually. The commands assume your using the default `.kibana` index. If `kibana.index` is set to something different in your `kibana.yml` you will need to modify the commands.

[docs] Manually migrating Kibana index #13696

[docs] Manually migrating Kibana index #13696

Conversation

tylersmalley commented Aug 24, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tylersmalley commented Aug 25, 2017

rhoboat left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rhoboat Aug 29, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tylersmalley Aug 30, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tylersmalley commented Aug 30, 2017

rhoboat commented Aug 31, 2017

rhoboat commented Aug 31, 2017

tylersmalley commented Aug 31, 2017

tylersmalley commented Aug 31, 2017

epixa commented Sep 1, 2017

tylersmalley commented Sep 5, 2017

epixa commented Sep 5, 2017

tylersmalley commented Aug 24, 2017 •

edited

Loading

rhoboat Aug 29, 2017 •

edited

Loading

tylersmalley Aug 30, 2017 •

edited

Loading