[docs] Manually migrating Kibana index #13696
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,337 @@ | ||
| [[migrating-6.0-index]] | ||
| == Migrating Kibana index to 6.0 | ||
|
|
||
| [IMPORTANT] | ||
| ============================================== | ||
| The following will make transformations to your Kibana index. It is recommended you backup your Kibana index prior to running any of the following commands. | ||
|
There was a problem hiding this comment. Maybe a link to the snapshot/restore docs where you say "backup your Kibana index" ? https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html |
||
| ============================================== | ||
|
|
||
| In order to migrate to Kibana 6, your Kibana index needs to be re-indexed. This will make transformations to the data model, removing types which are being {es-ref}/removal-of-types.html[removed in Elasticsearch]. | ||
|
There was a problem hiding this comment. I think in other docs, we aren't hyphenating "reindex", so maybe being consistent here would be good. |
||
|
|
||
| A UI is available in X-Pack 5.6 to assist with the migration. The following is provided should you be required to run the migration manually. The commands assume your using the default `.kibana` index. If `kibana.index` is set to something different in your `kibana.yml` you will need to modify the commands. | ||
|
There was a problem hiding this comment.
|
||
|
|
||
| To perform this migration without downtime, you will need to be running Kibana 5.6. This is the only version which supports both of the index types. | ||
|
|
||
| . Set `.kibana` index to read-only: | ||
| + | ||
| -- | ||
| [source,js] | ||
| -------------------------------------------------- | ||
| PUT .kibana/_settings | ||
| { | ||
| "index.blocks.write": true | ||
| } | ||
| -------------------------------------------------- | ||
| // CONSOLE | ||
| -- | ||
|
|
||
| . Create `.kibana-6` index | ||
| + | ||
| -- | ||
| [source,js] | ||
| -------------------------------------------------- | ||
| PUT .kibana-6 | ||
| { | ||
| "settings" : { | ||
| "number_of_shards" : 1, | ||
| "index.format": 6 | ||
| }, | ||
| "mappings" : { | ||
| "doc": { | ||
| "properties": { | ||
| "type": { | ||
| "type": "keyword" | ||
| }, | ||
| "updated_at": { | ||
| "type": "date" | ||
| }, | ||
| "config": { | ||
| "properties": { | ||
| "buildNum": { | ||
| "type": "keyword" | ||
| } | ||
| } | ||
| }, | ||
| "index-pattern": { | ||
| "properties": { | ||
| "fieldFormatMap": { | ||
| "type": "text" | ||
| }, | ||
| "fields": { | ||
| "type": "text" | ||
| }, | ||
| "intervalName": { | ||
| "type": "keyword" | ||
| }, | ||
| "notExpandable": { | ||
| "type": "boolean" | ||
| }, | ||
| "sourceFilters": { | ||
| "type": "text" | ||
| }, | ||
| "timeFieldName": { | ||
| "type": "keyword" | ||
| }, | ||
| "title": { | ||
| "type": "text" | ||
| } | ||
| } | ||
| }, | ||
| "visualization": { | ||
| "properties": { | ||
| "description": { | ||
| "type": "text" | ||
| }, | ||
| "kibanaSavedObjectMeta": { | ||
| "properties": { | ||
| "searchSourceJSON": { | ||
| "type": "text" | ||
| } | ||
| } | ||
| }, | ||
| "savedSearchId": { | ||
| "type": "keyword" | ||
| }, | ||
| "title": { | ||
| "type": "text" | ||
| }, | ||
| "uiStateJSON": { | ||
| "type": "text" | ||
| }, | ||
| "version": { | ||
| "type": "integer" | ||
| }, | ||
| "visState": { | ||
| "type": "text" | ||
| } | ||
| } | ||
| }, | ||
| "search": { | ||
| "properties": { | ||
| "columns": { | ||
| "type": "keyword" | ||
| }, | ||
| "description": { | ||
| "type": "text" | ||
| }, | ||
| "hits": { | ||
| "type": "integer" | ||
| }, | ||
| "kibanaSavedObjectMeta": { | ||
| "properties": { | ||
| "searchSourceJSON": { | ||
| "type": "text" | ||
| } | ||
| } | ||
| }, | ||
| "sort": { | ||
| "type": "keyword" | ||
| }, | ||
| "title": { | ||
| "type": "text" | ||
| }, | ||
| "version": { | ||
| "type": "integer" | ||
| } | ||
| } | ||
| }, | ||
| "dashboard": { | ||
| "properties": { | ||
| "description": { | ||
| "type": "text" | ||
| }, | ||
| "hits": { | ||
| "type": "integer" | ||
| }, | ||
| "kibanaSavedObjectMeta": { | ||
| "properties": { | ||
| "searchSourceJSON": { | ||
| "type": "text" | ||
| } | ||
| } | ||
| }, | ||
| "optionsJSON": { | ||
| "type": "text" | ||
| }, | ||
| "panelsJSON": { | ||
| "type": "text" | ||
| }, | ||
| "refreshInterval": { | ||
| "properties": { | ||
| "display": { | ||
| "type": "keyword" | ||
| }, | ||
| "pause": { | ||
| "type": "boolean" | ||
| }, | ||
| "section": { | ||
| "type": "integer" | ||
| }, | ||
| "value": { | ||
| "type": "integer" | ||
| } | ||
| } | ||
| }, | ||
| "timeFrom": { | ||
| "type": "keyword" | ||
| }, | ||
| "timeRestore": { | ||
| "type": "boolean" | ||
| }, | ||
| "timeTo": { | ||
| "type": "keyword" | ||
| }, | ||
| "title": { | ||
| "type": "text" | ||
| }, | ||
| "uiStateJSON": { | ||
| "type": "text" | ||
| }, | ||
| "version": { | ||
| "type": "integer" | ||
| } | ||
| } | ||
| }, | ||
| "url": { | ||
| "properties": { | ||
| "accessCount": { | ||
| "type": "long" | ||
| }, | ||
| "accessDate": { | ||
| "type": "date" | ||
| }, | ||
| "createDate": { | ||
| "type": "date" | ||
| }, | ||
| "url": { | ||
| "type": "text", | ||
| "fields": { | ||
| "keyword": { | ||
| "type": "keyword", | ||
| "ignore_above": 2048 | ||
| } | ||
| } | ||
| } | ||
| } | ||
| }, | ||
| "server": { | ||
| "properties": { | ||
| "uuid": { | ||
| "type": "keyword" | ||
| } | ||
| } | ||
| }, | ||
| "timelion-sheet": { | ||
| "properties": { | ||
| "description": { | ||
| "type": "text" | ||
| }, | ||
| "hits": { | ||
| "type": "integer" | ||
| }, | ||
| "kibanaSavedObjectMeta": { | ||
| "properties": { | ||
| "searchSourceJSON": { | ||
| "type": "text" | ||
| } | ||
| } | ||
| }, | ||
| "timelion_chart_height": { | ||
| "type": "integer" | ||
| }, | ||
| "timelion_columns": { | ||
| "type": "integer" | ||
| }, | ||
| "timelion_interval": { | ||
| "type": "keyword" | ||
| }, | ||
| "timelion_other_interval": { | ||
| "type": "keyword" | ||
| }, | ||
| "timelion_rows": { | ||
| "type": "integer" | ||
| }, | ||
| "timelion_sheet": { | ||
| "type": "text" | ||
| }, | ||
| "title": { | ||
| "type": "text" | ||
| }, | ||
| "version": { | ||
| "type": "integer" | ||
| } | ||
| } | ||
| }, | ||
| "graph-workspace": { | ||
| "properties": { | ||
| "description": { | ||
| "type": "text" | ||
| }, | ||
| "kibanaSavedObjectMeta": { | ||
| "properties": { | ||
| "searchSourceJSON": { | ||
| "type": "text" | ||
| } | ||
| } | ||
| }, | ||
| "numLinks": { | ||
| "type": "integer" | ||
| }, | ||
| "numVertices": { | ||
| "type": "integer" | ||
| }, | ||
| "title": { | ||
| "type": "text" | ||
| }, | ||
| "version": { | ||
| "type": "integer" | ||
| }, | ||
| "wsState": { | ||
| "type": "text" | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
There was a problem hiding this comment. This part kind of bothers me. Putting the whole mappings here feels like there could be a typo, or some change to it later (maybe not realistic), but it means it's in more than one place. Is it worth it to link to a mappings object as a central source of truth? There was a problem hiding this comment. I believe we could include it, but we would need the whole mapping which I believe this is the first in the OS repo. |
||
| -------------------------------------------------- | ||
| // CONSOLE | ||
| -- | ||
|
|
||
| . Re-index `.kibana` into `.kibana-6`: | ||
|
|
||
| + | ||
| -- | ||
| [source,js] | ||
| -------------------------------------------------- | ||
| POST _reindex | ||
| { | ||
| "source": { | ||
| "index": ".kibana" | ||
| }, | ||
| "dest": { | ||
| "index": ".kibana-6" | ||
| }, | ||
| "script": { | ||
| "source": "ctx._source = [ ctx._type : ctx._source ]; ctx._source.type = ctx._type; ctx._id = ctx._type + \":\" + ctx._id; ctx._type = \"doc\"; " | ||
| } | ||
| } | ||
| -------------------------------------------------- | ||
| // CONSOLE | ||
| -- | ||
|
|
||
| . Alias `.kibana-6` to `.kibana` and remove legacy `.kibana` index: | ||
| + | ||
| -- | ||
| [source,js] | ||
| -------------------------------------------------- | ||
| POST /_aliases | ||
| { | ||
| "actions" : [ | ||
| { "add": { "index": ".kibana-6", "alias": ".kibana" } }, | ||
| { "remove_index": { "index": ".kibana" } } | ||
| ] | ||
|
There was a problem hiding this comment. It looks like this command will wipe out any existing aliases on the old kibana index. Right? There was a problem hiding this comment. Here's where we're adding the old aliases to the new index: https://github.com/elastic/x-pack-kibana/blob/master/plugins/upgrade/public/lib/util/index.js#L77-L87 There was a problem hiding this comment. I 38cc25f I have mentioned not using an alias. I would rather keep the commands simple for majority of users and call out the cases where folks have complex setups. |
||
| } | ||
| -------------------------------------------------- | ||
| // CONSOLE | ||
| -- | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is there a link a more info link to the migration ui available?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe @archanid is still working on that. We should link when it becomes available.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it shouldn't be called "Migration UI" here--we're calling it "Upgrade Assistant"