must switch kibana index from string to keyword

[LeeDr]

Kibana version: master commit 8d64270

Elasticsearch version: master commit c5b9c98b990d7661bc5796f00535bc1e3ed297a5

Server OS version:

Browser version:

Browser OS version:

Original install method (e.g. download page, yum, from source, etc.):

Description of the problem including expected versus actual behavior:
Kibana master fails to create .kibana index on Elasticsearch master. It seems to be caused by Elasticsearch removing support for string. I made a change and got past the issue.

UPGRADE is really the hard part of fixing this?

Here's the change I made to get past the problem:

src/core_plugins/elasticsearch/lib/kibana_index_mappings.js

diff --git a/src/core_plugins/elasticsearch/lib/kibana_index_mappings.js b/src/core_plugins/elasticsearch/lib/kibana_index_mappings.js
index b9a8021..959cf2c 100644
--- a/src/core_plugins/elasticsearch/lib/kibana_index_mappings.js
+++ b/src/core_plugins/elasticsearch/lib/kibana_index_mappings.js
@@ -2,8 +2,7 @@ export const mappings = {
   config: {
     properties: {
       buildNum: {
-        type: 'string',
-        index: 'not_analyzed'
+        type: 'keyword'
       }
     }
   },

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

server    log   [15:47:50.857] [info][status][plugin:elasticsearch@6.0.0-alpha1] Status changed from yellow to yellow - No existing Kibana index found
{ [mapper_parsing_exception] No handler for type [string] declared on field [buildNum] :: {"path":"/.kibana","query":{},"body":"{\"settings\":{\"number_of_shards\":1},\"mappings\":{\"config\":{\"properties\":{\"buildNum\":{\"type\":\"string\",\"index\":\"not_analyzed\"}}},\"server\":{\"properties\":{\"uuid\":{\"type\":\"keyword\"}}}}}","statusCode":400,"response":"{\"error\":{\"root_cause\":[{\"type\":\"mapper_parsing_exception\",\"reason\":\"No handler for type [string] declared on field [buildNum]\"}],\"type\":\"mapper_parsing_exception\",\"reason\":\"Failed to parse mapping [config]: No handler for type [string] declared on field [buildNum]\",\"caused_by\":{\"type\":\"mapper_parsing_exception\",\"reason\":\"No handler for type [string] declared on field [buildNum]\"}},\"status\":400}"}
    at respond (/home/leedr/git/kibana/node_modules/elasticsearch/src/lib/transport.js:289:15)
    at checkRespForFailure (/home/leedr/git/kibana/node_modules/elasticsearch/src/lib/transport.js:248:7)
    at HttpConnector.<anonymous> (/home/leedr/git/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:164:7)
    at IncomingMessage.wrapper (/home/leedr/git/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4962:19)
    at emitNone (events.js:91:20)
    at IncomingMessage.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:975:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)
  status: 400,
  displayName: 'BadRequest',
  message: '[mapper_parsing_exception] No handler for type [string] declared on field [buildNum]',
  path: '/.kibana',
  query: {},
  body:
   { error:
      { root_cause: [Object],
        type: 'mapper_parsing_exception',
        reason: 'Failed to parse mapping [config]: No handler for type [string] declared on field [buildNum]',
        caused_by: [Object] },
     status: 400 },
  statusCode: 400,
  response: '{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"No handler for type [string] declared on field [buildNum]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [config]: No handler for type [string] declared on field [buildNum]","caused_by":{"type":"mapper_parsing_exception","reason":"No handler for type [string] declared on field [buildNum]"}},"status":400}',
  toString: [Function],
  toJSON: [Function] }

[LeeDr]

Thanks @ycombinator for the debugging help!

[ycombinator]

As @LeeDr noted, the fix from string/not_analyzed -> keyword is easy, which is what the PR for this issue will address.

The harder problem is migration of the mapping for users who upgrade from kibana 5 -> 6. That is being tracked in a separate issue, #6409.

[LeeDr]

It looks like this one fix wasn't enough for Kibana to work. When I log in I just get an empty main pane. And the console shows;

angular.js:12477 Error: [mapper_parsing_exception] No handler for type [string] declared on field [title]
    at respond (elasticsearch.angular.js?6e3d:23653)
    at checkRespForFailure (elasticsearch.angular.js?6e3d:23612)
    at elasticsearch.angular.js?6e3d:299
    at processQueue (angular.js:14745)
    at angular.js:14761
    at Scope.$eval (angular.js:15989)
    at Scope.$digest (angular.js:15800)
    at Scope.$apply (angular.js:16097)
    at done (angular.js:10546)
    at completeRequest (angular.js:10744)

[ycombinator]

Thanks @LeeDr. I'll look into the 2nd error as well.

[tylersmalley]

The associated Elasticsearch PR: elastic/elasticsearch#21670

[ycombinator]

@LeeDr I'm unable to reproduce the 2nd error. Any more details on where that's coming from? I can't tell much from that stack trace, unfortunately.

[lukasolson]

I'm getting that second error too. I've made the change suggested above to kibana_index_mappings.js and deleted my Kibana index. Then, when I start up Kibana and try to load the app, I get the error. (If you have X-Pack installed, you'll have to login first to see the error.)

[ycombinator]

Ah, thanks @lukasolson. I didn't delete the kibana index (doh!).

[ycombinator]

Hmmm... even after deleting the .kibana I can't reproduce the 2nd error :( @lukasolson any ideas which part of the code that error is coming from?

[lukasolson]

Yeah, looks like there's a request that gets sent when you load the application:

PUT https://localhost:5601/nqz/elasticsearch/.kibana/_mapping/index-pattern
{
  "index-pattern": {
    "properties": {
      "title": {
        "type": "string"
      },
      "timeFieldName": {
        "type": "string"
      },
      "notExpandable": {
        "type": "boolean"
      },
      "intervalName": {
        "type": "string"
      },
      "fields": {
        "type": "string"
      },
      "sourceFilters": {
        "type": "string"
      },
      "fieldFormatMap": {
        "type": "string"
      }
    }
  }
}

And here's the response:

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "No handler for type [string] declared on field [title]"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "No handler for type [string] declared on field [title]"
  },
  "status": 400
}

So it looks like it might be related to this piece of code.

[lukasolson]

Also, here's where the request is actually made.

[ycombinator]

Thanks @lukasolson for those details. I'm still unable to reproduce the issue but I've made the changes based on what you provided and updated the PR: #9280.