export statements #778
export statements #778
Conversation
pkg/apiserver/statement/statement.go
Outdated
| csvData := [][]string{fields} | ||
| for _, overview := range overviews { | ||
| row := []string{} | ||
| v := reflect.ValueOf(overview) | ||
| for _, field := range fields { | ||
| filedName := fieldsMap[field] | ||
| s := v.FieldByName(filedName) | ||
| var val string | ||
| switch s.Interface().(type) { | ||
| case int: | ||
| val = strconv.FormatInt(s.Int(), 10) | ||
| default: | ||
| val = s.String() | ||
| } | ||
| row = append(row, val) | ||
| } | ||
| csvData = append(csvData, row) |
There was a problem hiding this comment.
Do we have some 3rd-party library for easier use? This logic may be the same when exporting slow queries.
There was a problem hiding this comment.
Make sense, let me have a check.
There was a problem hiding this comment.
Seems this lib can help us: https://github.com/oleiade/reflections , let me try it.
But it doesn't reduce much code, we need to extract it to a util method when we need to reuse it.
Co-authored-by: Wenxuan <hi@breeswish.org>
pkg/apiserver/statement/statement.go
Outdated
| secretKey := cryptopasta.NewEncryptionKey() | ||
|
|
||
| rowBuf := bytes.NewBuffer(nil) | ||
| csvwriter := csv.NewWriter(rowBuf) |
There was a problem hiding this comment.
This library implements an encrypted io.Reader / io.Writer, which could be a better choice. Maybe you can have a try: https://github.com/Xeoncross/go-aesctr-with-hmac
The encapsulation of the stream is file stream -> encrypted stream -> csv stream.
There was a problem hiding this comment.
Ok, thank you! let me have a look. Since I found zip the encrypted file won't reduce the size much, so I think it doesn't need to zip file in local, right? but to still zip it when sending data back to the client after being decrypted.
There was a problem hiding this comment.
IIRC, we use gzip already. Is there any further benefit of packaging as a zip for transmission?
There was a problem hiding this comment.
Make sense, let me focus on the encryption first.
There was a problem hiding this comment.
It works fine! and the final flow likes this:
|
Encrypt and decrypt flow:
|
|
New effect: (Mocked time delays) |
|
|
There is a spinner message: https://codesandbox.io/s/3yq1h |
Cool! |
|
New effect:
|
|
TODO:
|
|
@baurine Is this PR ready to merge? |
Yep，I think so. |
* misc: Increase ulimit to 65535 for test env (#756) * test: Fix frontend CI (#752) * ui: fix dayjs i18n (#755) * ui: handle error globally (#757) * statement, slow_query: support all fields in list page (#749) * ui: memorize expand/collapse full text in detail pages (#775) * ui: break loop dependencies (#771) * ui: fix browser compatibility check (#776) * ui: Refine store location, add zoom and pan (#772) * ui: show disk usage information for statement and slow query (#777) * ui: use qps instead of ops (#786) * statement: support export (#778) *: Fix slow query and start_ts not working in some cases (#793) * ui: fix errors doesn't display (#794) * ui: fix the error message doesn't show correct (#799) * slow_queries: support export (#792) * ui: add MySqlFormatter to customize the sql formatter (#805) *: fix query statement detail error cause by round (#806) * ui: copy original content instead of formatted content for CopyLink (#802) * add min height of topology canvas (#804) * metrics: Support customize Prometheus address (#808) * clusterinfo: Refine (#815) * ui: Open statement and slow log in new tab (#816) * ui: add more time field for slow query detail page (#810) * slowlog: Improve descriptions (#817) * build: add action to check release-version is changed for release branch * Release v2020.11.26.1
close #565
What did:
Screenshots:
(mocked time delays)
