-
Notifications
You must be signed in to change notification settings - Fork 5.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
lightning: refactor to reuse in load data, part 3 #42689
Conversation
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. Reviewer can indicate their review by submitting an approval review. |
/run-integration-br-test |
2 similar comments
/run-integration-br-test |
/run-integration-br-test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
9 / 10 files viewed
if e.recordCache != nil { | ||
return e.recordCache | ||
} | ||
return make([]types.Datum, 0, len(e.Columns)+1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we save it to e.recordCache
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's saved after recode as KV
, didn't change prev logic
} | ||
|
||
// Record2KV converts a row into a KV pair. | ||
func (e *BaseKVEncoder) Record2KV(record, originalRow []types.Datum, rowID int64) (*KvPairs, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not sure in future if we provide a batch version of Record2KV, which adds multiple records at once, the performance will be better
} | ||
|
||
// ProcessColDatum processes the datum of a column. | ||
func (e *BaseKVEncoder) ProcessColDatum(col *table.Column, rowID int64, inputDatum *types.Datum) (types.Datum, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this need to be public? Seems it only need to be called during Encode
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@@ -59,6 +60,8 @@ func init() { | |||
// Setup the Hooks to dynamic control global resource controller. | |||
variable.EnableGlobalResourceControlFunc = tikv.EnableResourceControl | |||
variable.DisableGlobalResourceControlFunc = tikv.DisableResourceControl | |||
// cannot use this package directly, it causes import cycle | |||
importer.GetKVStore = getKVStore |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just curious, how does store/driver
depend on the importer package?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for load data physical import, importer package will use it to create a kvStore, it's not used in this pr.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What I mean is how the import cycle is encountered. In other words, is there such a dependency chain: store/driver
-> A -> B -> .... -> importer?
/merge |
This pull request has been accepted and is ready to merge. Commit hash: ac59540
|
What problem does this PR solve?
Issue Number: ref #40499
Problem Summary:
What is changed and how it works?
Check List
Tests
Side effects
Documentation
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.