Skip to content

Commit

Permalink
[docs] Manually migrating Kibana index
Browse files Browse the repository at this point in the history
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
  • Loading branch information
Tyler Smalley committed Aug 24, 2017
1 parent 364bb0f commit ef0857d
Show file tree
Hide file tree
Showing 3 changed files with 344 additions and 2 deletions.
1 change: 1 addition & 0 deletions docs/migration.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@ your application from one version of Kibana to another.
--

include::migration/migrate_6_0.asciidoc[]
include::migration/migrate_6_0_index.asciidoc[]
8 changes: 6 additions & 2 deletions docs/migration/migrate_6_0.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,18 @@
This section discusses the changes that you need to be aware of when migrating
your application to Kibana 6.0.

[float]
=== Types removed from Kibana index
*Details:* With the imminent {es-ref}/removal-of-types.html[removal of types in Elasticsearch], we have updated our index to not rely on types. Kibana 5.6.x can support both of these mappings to minimize downtime. A UI is provided with X-Pack to assist with the migration of the Elastic stack.

*Impact:* You will need migrate your Kibana index either {kibana-ref}/migrating-6.0-index.html[manually], or through the Migration UI in X-Pack.


[float]
=== Removed option to use unsupported scripting languages
*Details:* Kibana 5.x allowed users to create scripted fields using any scripting language enabled in Elasticsearch.
Kibana 6.0 will only support Painless and Lucene expression based scripts.


*Impact:* You will need to migrate your groovy, python, javascript, etc. scripted fields to Painless or Lucene expressions.


Expand Down Expand Up @@ -73,7 +78,6 @@ This is no longer the case. Now, only commas are a valid query separator: e.g. `

*Impact:* No change is required for existing Kibana index patterns. Those previously configured with this option will gracefully use the new Elasticsearch optimizations instead, as will all new index patterns.


[float]
=== Replace markdown parser `marked` with `markdown-it`
*Details:* Starting in 6.0.0, Kibana will use `markdown-it` to parse markdown text. Kibana switched to `markdown-it` because `marked` is no longer actively maintained. Markdown-it supports CommonMark and GFM (GitHub Flavored Markdown) Tables and Strikethrough.
Expand Down
337 changes: 337 additions & 0 deletions docs/migration/migrate_6_0_index.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,337 @@
[[migrating-6.0-index]]
== Migrating Kibana index to 6.0

[IMPORTANT]
==============================================
The following will make transformations to your Kibana index. It is recommended you backup your Kibana index prior to running any of the following commands.
==============================================

In order to migrate to Kibana 6, your Kibana index needs to be re-indexed. This will make transformations to the data model, removing types which are being {es-ref}/removal-of-types.html[removed in Elasticsearch].

A UI is available in X-Pack 5.6 to assist with the migration. The following is provided should you be required to run the migration manually. The commands assume your using the default `.kibana` index. If `kibana.index` is set to something different in your `kibana.yml` you will need to modify the commands.

To perform this migration without downtime, you will need to be running Kibana 5.6. This is the only version which supports both of the index types.

. Set `.kibana` index to read-only:
+
--
[source,js]
--------------------------------------------------
PUT .kibana/_settings
{
"index.blocks.write": true
}
--------------------------------------------------
// CONSOLE
--

. Create `.kibana-6` index
+
--
[source,js]
--------------------------------------------------
PUT .kibana-6
{
"settings" : {
"number_of_shards" : 1,
"index.format": 6
},
"mappings" : {
"doc": {
"properties": {
"type": {
"type": "keyword"
},
"updated_at": {
"type": "date"
},
"config": {
"properties": {
"buildNum": {
"type": "keyword"
}
}
},
"index-pattern": {
"properties": {
"fieldFormatMap": {
"type": "text"
},
"fields": {
"type": "text"
},
"intervalName": {
"type": "keyword"
},
"notExpandable": {
"type": "boolean"
},
"sourceFilters": {
"type": "text"
},
"timeFieldName": {
"type": "keyword"
},
"title": {
"type": "text"
}
}
},
"visualization": {
"properties": {
"description": {
"type": "text"
},
"kibanaSavedObjectMeta": {
"properties": {
"searchSourceJSON": {
"type": "text"
}
}
},
"savedSearchId": {
"type": "keyword"
},
"title": {
"type": "text"
},
"uiStateJSON": {
"type": "text"
},
"version": {
"type": "integer"
},
"visState": {
"type": "text"
}
}
},
"search": {
"properties": {
"columns": {
"type": "keyword"
},
"description": {
"type": "text"
},
"hits": {
"type": "integer"
},
"kibanaSavedObjectMeta": {
"properties": {
"searchSourceJSON": {
"type": "text"
}
}
},
"sort": {
"type": "keyword"
},
"title": {
"type": "text"
},
"version": {
"type": "integer"
}
}
},
"dashboard": {
"properties": {
"description": {
"type": "text"
},
"hits": {
"type": "integer"
},
"kibanaSavedObjectMeta": {
"properties": {
"searchSourceJSON": {
"type": "text"
}
}
},
"optionsJSON": {
"type": "text"
},
"panelsJSON": {
"type": "text"
},
"refreshInterval": {
"properties": {
"display": {
"type": "keyword"
},
"pause": {
"type": "boolean"
},
"section": {
"type": "integer"
},
"value": {
"type": "integer"
}
}
},
"timeFrom": {
"type": "keyword"
},
"timeRestore": {
"type": "boolean"
},
"timeTo": {
"type": "keyword"
},
"title": {
"type": "text"
},
"uiStateJSON": {
"type": "text"
},
"version": {
"type": "integer"
}
}
},
"url": {
"properties": {
"accessCount": {
"type": "long"
},
"accessDate": {
"type": "date"
},
"createDate": {
"type": "date"
},
"url": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 2048
}
}
}
}
},
"server": {
"properties": {
"uuid": {
"type": "keyword"
}
}
},
"timelion-sheet": {
"properties": {
"description": {
"type": "text"
},
"hits": {
"type": "integer"
},
"kibanaSavedObjectMeta": {
"properties": {
"searchSourceJSON": {
"type": "text"
}
}
},
"timelion_chart_height": {
"type": "integer"
},
"timelion_columns": {
"type": "integer"
},
"timelion_interval": {
"type": "keyword"
},
"timelion_other_interval": {
"type": "keyword"
},
"timelion_rows": {
"type": "integer"
},
"timelion_sheet": {
"type": "text"
},
"title": {
"type": "text"
},
"version": {
"type": "integer"
}
}
},
"graph-workspace": {
"properties": {
"description": {
"type": "text"
},
"kibanaSavedObjectMeta": {
"properties": {
"searchSourceJSON": {
"type": "text"
}
}
},
"numLinks": {
"type": "integer"
},
"numVertices": {
"type": "integer"
},
"title": {
"type": "text"
},
"version": {
"type": "integer"
},
"wsState": {
"type": "text"
}
}
}
}
}
}
}
--------------------------------------------------
// CONSOLE
--

. Re-index `.kibana` into `.kibana-6`:
+
--
[source,js]
--------------------------------------------------
POST _reindex
{
"source": {
"index": ".kibana"
},
"dest": {
"index": ".kibana-6"
},
"script": {
"source": "ctx._source = [ ctx._type : ctx._source ]; ctx._source.type = ctx._type; ctx._id = ctx._type + \":\" + ctx._id; ctx._type = \"doc\"; "
}
}
--------------------------------------------------
// CONSOLE
--

. Alias `.kibana-6` to `.kibana` and remove legacy `.kibana` index:
+
--
[source,js]
--------------------------------------------------
POST /_aliases
{
"actions" : [
{ "add": { "index": ".kibana-6", "alias": ".kibana" } },
{ "remove_index": { "index": ".kibana" } }
]
}
--------------------------------------------------
// CONSOLE
--

0 comments on commit ef0857d

Please sign in to comment.