-
Notifications
You must be signed in to change notification settings - Fork 985
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DRILL-7948: Unable to query file with required fixed_len_byte_array decimal columns #2254
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The main fix is for BINARY
/FIXED_LEN_BYTE_ARRAY
Dictionary and non DictionaryColumnChunkMetaData
type, right?
And the missing case was PrimitiveTypeName.FIXED_LEN_BYTE_ARRAY
and ConvertedType.INTERVAL
right? Should we cover it with a test case?
@vdiravka, sorry, but I don't know how to generate a parquet file with the correct INTERVAL type and using dictionary encoding, so no test was added, but we have tests for INTERVAL with non-dictionary encoding. |
Yes, the main change is the fix for the |
Let me try to generate it within Drill's |
@vdiravka, thanks to referring to
Here is the example of output for dictionary encoding:
|
@vdiravka, did you have a chance to take a look at generating a parquet file? |
For me it is some sort of bug in Parquet lib. Anyway looks like there is a workaround: you can remove
where Initially there was a following meta for this file for me:
|
DRILL-7948: Enable testNullableIntervalDictionaryEncoding test
@vdiravka, I have added requested unit tests and fixed several issues, please take a look |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work! LGTM +1
DRILL-7948: Unable to query file with required fixed_len_byte_array decimal columns
Description
FIXED_LEN_BYTE_ARRAY
andBINARY
typesDocumentation
NA
Testing
Added UT.