Make CSV upload limit configurable #8228
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Can you add this configuration value to the docs? |
Kibana's Upload CSV feature isn't intended for gigantic import jobs, so I originally set a sane default of 1GB. Some users exprssed a desire to import slightly larger files, they should be able to import something that's 1.1GB without being blocked by an arbitrary limit. So I've made the limit configurable via kibana.yml. This change includes a few pieces: * Added optional `kibana.addDataMaxBytes` key to kibana.yml with 1GB default * Set upload data route payload limit based on new config value * Updated help text in UI to use the dynamic config value on the parse csv page * Updated parse csv page to check file size and fail early if the selected file is too big Resolves: elastic#7671
|
@epixa done |
|
LGTM |
1 similar comment
|
LGTM |
| @@ -21,6 +22,8 @@ modules.get('apps/management') | |||
| const maxSampleRows = 10; | |||
| const maxSampleColumns = 20; | |||
|
|
|||
| this.maxBytesFormatted = numeral(addDataMaxBytes).format('0 b'); | |||
There was a problem hiding this comment.
Doesn't seem this is used anywhere. Did you mean to use it in the formatted error message on line 63?
There was a problem hiding this comment.
It's used in the view
|
Thanks for the speedy review guys! |
|
@Bargs Please make sure this is backported to |
This was referenced Sep 12, 2016
elastic-jasper
added a commit
that referenced
this pull request
Sep 12, 2016
--------- **Commit 1:** Make CSV upload limit configurable Kibana's Upload CSV feature isn't intended for gigantic import jobs, so I originally set a sane default of 1GB. Some users exprssed a desire to import slightly larger files, they should be able to import something that's 1.1GB without being blocked by an arbitrary limit. So I've made the limit configurable via kibana.yml. This change includes a few pieces: * Added optional `kibana.addDataMaxBytes` key to kibana.yml with 1GB default * Set upload data route payload limit based on new config value * Updated help text in UI to use the dynamic config value on the parse csv page * Updated parse csv page to check file size and fail early if the selected file is too big Resolves: #7671 * Original sha: 0503aa8 * Authored by Matthew Bargar <mbargar@gmail.com> on 2016-09-12T18:13:44Z
elastic-jasper
added a commit
that referenced
this pull request
Sep 12, 2016
--------- **Commit 1:** Make CSV upload limit configurable Kibana's Upload CSV feature isn't intended for gigantic import jobs, so I originally set a sane default of 1GB. Some users exprssed a desire to import slightly larger files, they should be able to import something that's 1.1GB without being blocked by an arbitrary limit. So I've made the limit configurable via kibana.yml. This change includes a few pieces: * Added optional `kibana.addDataMaxBytes` key to kibana.yml with 1GB default * Set upload data route payload limit based on new config value * Updated help text in UI to use the dynamic config value on the parse csv page * Updated parse csv page to check file size and fail early if the selected file is too big Resolves: #7671 * Original sha: 0503aa8 * Authored by Matthew Bargar <mbargar@gmail.com> on 2016-09-12T18:13:44Z
Bargs
pushed a commit
that referenced
this pull request
Sep 12, 2016
[backport] PR #8228 to 5.x - Make CSV upload limit configurable
Bargs
pushed a commit
that referenced
this pull request
Sep 12, 2016
[backport] PR #8228 to 5.0 - Make CSV upload limit configurable
airow
pushed a commit
to airow/kibana
that referenced
this pull request
Feb 16, 2017
--------- **Commit 1:** Make CSV upload limit configurable Kibana's Upload CSV feature isn't intended for gigantic import jobs, so I originally set a sane default of 1GB. Some users exprssed a desire to import slightly larger files, they should be able to import something that's 1.1GB without being blocked by an arbitrary limit. So I've made the limit configurable via kibana.yml. This change includes a few pieces: * Added optional `kibana.addDataMaxBytes` key to kibana.yml with 1GB default * Set upload data route payload limit based on new config value * Updated help text in UI to use the dynamic config value on the parse csv page * Updated parse csv page to check file size and fail early if the selected file is too big Resolves: elastic#7671 * Original sha: bdc2f855000d3d153e85c16deef81a17f853d69c [formerly 0503aa8] * Authored by Matthew Bargar <mbargar@gmail.com> on 2016-09-12T18:13:44Z Former-commit-id: 7479654
airow
pushed a commit
to airow/kibana
that referenced
this pull request
Feb 16, 2017
[backport] PR elastic#8228 to 5.x - Make CSV upload limit configurable Former-commit-id: c417cac
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Kibana's Upload CSV feature isn't intended for gigantic import jobs, so
I originally set a sane default of 1GB. Some users exprssed a desire to
import slightly larger files, they should be able to import something
that's 1.1GB without being blocked by an arbitrary limit. So I've made
the limit configurable via kibana.yml.
This change includes a few pieces:
kibana.addDataMaxByteskey to kibana.yml with 1GBdefault
csv page
selected file is too big
Resolves: #7671