Skip to content
This repository has been archived by the owner on Aug 21, 2023. It is now read-only.

support specialized csv separator and delimiter #116

Merged
merged 3 commits into from
Jul 8, 2020

Conversation

lichunzhu
Copy link
Contributor

What problem does this PR solve?

Resolve #113

What is changed and how it works?

Add arguments "csv-separator" and "csv-delimiter" to support specialized csv separator and delimiter.

Check List

Tests

  • Unit test
  • Integration test

Side effects

  • Possible performance regression
  • Increased code complexity

Related changes

  • Need to update the documentation

Release note

@codecov
Copy link

codecov bot commented Jul 7, 2020

Codecov Report

Merging #116 into master will decrease coverage by 1.16%.
The diff coverage is 27.55%.

@@            Coverage Diff             @@
##           master     #116      +/-   ##
==========================================
- Coverage   51.92%   50.76%   -1.17%     
==========================================
  Files          17       17              
  Lines        1770     1775       +5     
==========================================
- Hits          919      901      -18     
- Misses        782      811      +29     
+ Partials       69       63       -6     

Copy link
Collaborator

@kennytm kennytm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest LGTM

@@ -153,11 +153,11 @@ func (r RowReceiverArr) WriteToBuffer(bf *bytes.Buffer, escapeBackslash bool) {
bf.WriteByte(')')
}

func (r RowReceiverArr) WriteToBufferInCsv(bf *bytes.Buffer, escapeBackslash bool, csvNullValue string) {
func (r RowReceiverArr) WriteToBufferInCsv(bf *bytes.Buffer, escapeBackslash bool, opt csvOption) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pass this by reference please (also all other occurrences)

Suggested change
func (r RowReceiverArr) WriteToBufferInCsv(bf *bytes.Buffer, escapeBackslash bool, opt csvOption) {
func (r RowReceiverArr) WriteToBufferInCsv(bf *bytes.Buffer, escapeBackslash bool, opt *csvOption) {

"2;'female';'sarah@mail.com';'020-1253';'healthy'\n" +
"3;'male';'john@mail.com';'020-1256';'healthy'\n" +
"4;'female';'sarah@mail.com';'020-1235';'healthy'\n"
c.Assert(bf.String(), Equals, expected)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please include a test case where separator has multiple characters and delimiter is "ma" (male should become mamamalema, femail should become mafemamalema, xxx@mail.com should become maxxx@mamail.comma).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in 7eb9742

Copy link
Collaborator

@kennytm kennytm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

v4/export/writer_util_test.go Outdated Show resolved Hide resolved
@kennytm kennytm added status/LGT1 One reviewer approved (LGTM1) and removed status/PTAL labels Jul 8, 2020
@kennytm kennytm merged commit 99dd904 into pingcap:master Jul 8, 2020
@lichunzhu lichunzhu deleted the supportCSVSeparatorDelimiter branch December 11, 2020 09:55
tisonkun pushed a commit to tisonkun/dumpling that referenced this pull request Oct 20, 2021
* support specialized csv separator and delimiter

* address comments

* address comment
tisonkun pushed a commit to tisonkun/dumpling that referenced this pull request Oct 20, 2021
* support specialized csv separator and delimiter

* address comments

* address comment
tisonkun pushed a commit to tisonkun/dumpling that referenced this pull request Oct 20, 2021
* support specialized csv separator and delimiter

* address comments

* address comment
tisonkun pushed a commit to tisonkun/dumpling that referenced this pull request Oct 20, 2021
* support specialized csv separator and delimiter

* address comments

* address comment
tisonkun pushed a commit to tisonkun/tidb that referenced this pull request Oct 20, 2021
* support specialized csv separator and delimiter

* address comments

* address comment
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
status/LGT1 One reviewer approved (LGTM1)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support customized csv separator/delimiter
2 participants