export dashboard API fails to complete JSON output

[LeeDr]

Kibana version: 6.3.0-BC4

Elasticsearch version: 6.3.0-BC4

Server OS version: Windows 2012 Server

Browser version: N/A

Browser OS version: N/A

Original install method (e.g. download page, yum, from source, etc.): zip files

Describe the bug: I used the export dashboard api and although it exported dashboard, all the visualizations, saved search, and index pattern, it didn't complete the last closing } ] }. So it won't import now.

Steps to reproduce:

1. use esArchiver to load this set of logstash* and .kibana index (unzip it); https://github.com/elastic/kibana/files/2002475/data_dashboard.zip
   node scripts/es_archiver load ./data_dashboard --es-url http://localhost:9200
2. export the dashboard and all dependencies;

curl -k -XGET 'http://localhost:5601/api/kibana/dashboards/export?dashboard=3a0da240-54a1-11e8-951c-136187190a21' > export_dashboard_3a0da240-54a1-11e8-951c-136187190a21.json

3. Inspect the json output file to see if it has the closing } ] } at the end
4. and trying to import like this;

curl -k -XPOST 'http://localhost:5601/api/kibana/dashboards/import -H 'Content-Type: application/json' -d @export_dashboard_3a0da240-54a1-11e8-951c-136187190a21.json

output is:
{"statusCode":400,"error":"Bad Request","message":"Invalid request payload JSON format"}

Expected behavior: The JSON from export should be valid and should import back into Kibana

Provide logs and/or server output (if relevant):

Any additional context:
I should note that someone else used the dashboard API to export and it worked correctly, so there's something about this data set that breaks the export functionality. It could be the number of visualizations (there are 30)?

leedr@mediacenter:~/git/kibana$ grep "\"type\"" export_dashboard_3a0da240-54a1-11e8-951c-136187190a21.json       "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "visualization",
      "type": "search",
      "type": "index-pattern",
      "type": "dashboard",

[adrienrenaud]

I'm seeing the same behaviour: missing } ] } at the end of the json file when exporting a dashboard using the kibana api. Using the ELK 6.2.3 dockers.

[philhagen]

Seeing this in 6.3.2 rpm-based version, but only on some dashboards.

[AurelienGttn]

I have this issue too in 6.3.1. The missing part isn't even always the same: sometimes the file lacks "] }", sometimes it's only "}". It's always the same dashboards that have that issue but they don't seem to have any particular feature, it seems to be quite random. Do you confirm this or do you see a pattern?

[philhagen]

I see exactly the same behavior you describe. Consistently wrong on a specific dashboard.

[kjrace]

I have the same issue on 6.3.1 - I'd noticed that the response is always complete in the browser.

Adding an Accept-Encoding header and compressed param to the curl command works for me though. e.g.

curl --compressed -H "Accept-Encoding: gzip, deflate, br" "http://localhost:5601/api/kibana/dashboard/export?dashboard=0c5781a0-94ab-11e8-8260-e5e9895a1cf3" > ./dashboard.json

[azasypkin]

Okay, we managed to track down the issue, the problem is that we incorrectly set Content-Length: xxx header here. We should not rely on String.length since stringified JSON can include multi-byte characters that will lead to incorrect Content-Length header and partially downloaded file.

The fix seems to be simple and require just using Buffer.byteLength(json, 'utf8') instead of json.length. I'll take care of it on Monday.

[philhagen]

I can confirm that adding the request headers per @kjrace's comment works great! thank you!
may also be fixed with the UTF8 issues, though I don't believe there was any UNICODE in the dashboard that didn't export correctly. There were five ® characters, though. The only dashboard that failed contained those characters - no others failed or contained an ®.

[azasypkin]

There were five ® characters, though. The only dashboard that failed contained those characters - no others failed or contained an ®.

Yeah, ® can definitely be a culprit, it consists of 2 bytes: http://www.ltg.ed.ac.uk/~richard/utf-8.cgi?input=%C2%AE&mode=char

[philhagen]

oh wow! the more you know....