row count limit

[jibanes]

There seem to be a row count limitation, as shown below, is there a known workaround/fix?

python tdeBuilder.py -i extractDesc.json
Totalrows: 614166
Build start time: 2017-01-26 10:50:17
Percent: 0.0 %, row: 1
Percent: 2.0 %, row: 12284
Percent: 4.0 %, row: 24567
Percent: 6.0 %, row: 36850
Percent: 8.0 %, row: 49134
Percent: 10.0 %, row: 61417
Percent: 12.0 %, row: 73700
Percent: 14.0 %, row: 85984
Percent: 16.0 %, row: 98267
Percent: 18.0 %, row: 110550
Percent: 20.0 %, row: 122834
Percent: 22.0 %, row: 135117
Percent: 24.0 %, row: 147400
Percent: 26.0 %, row: 159684
Percent: 28.0 %, row: 171967
tde file built to: test_table.tde
End Time: 2017-01-26 10:51:15.753012
Build elapsed time: 0:00:58.627732
Uploaded to server: http://10.200.144.22

NOTE: This behavior is identical whereas the machine has 8GB of ram or 256GB (tested).

[jibanes]

I think this problem occurs when one of the field contains a "\n" character, as such the row count is the number of lines in the file, not the number of records, could you confirm this? In essence it's purely cosmetic.

[guzzijones]

I would need the input file. Could u post it on google docs?

[guzzijones]

I just saw your note about the cosmetic nature of the bug.

[jibanes]

yep, looks like just Totalraw's value is wrong with exotic CSVs, and, implicitely calculated %.

[guzzijones]

i am surprised this file even works and the error is only cosmetic. I certainly did not code for the edge case of having embedded line feeds inside quotes.

[guzzijones]

i have the file.