Problem with array, UNNEST and JSON_PARSE

[fabricebaranski]

Please provide a detailed title (e.g. "Broker crashes when using TopN query with Bound filter" instead of just "Broker crashes").

Affected Version

Druid version 29.0.1

Description

Step 1: Create a datasource using this query (select array ingest mode: array)

INSERT INTO "jsondatasource"
SELECT ARRAY['{"key": "value1"}'] as "jsonarray"
PARTITIONED BY ALL

Step 2: Query your datasource
SELECT PARSE_JSON("json") FROM "jsondatasource" CROSS JOIN UNNEST("jsonarray") AS "json"

I get the following error:

Error: RUNTIME_FAILURE (OPERATOR)

Selector must have a dictionary

org.apache.druid.java.util.common.ISE

[clintropolis]

thanks for the report, this is a bug caused by an odd interaction between the underlying array column and unnest functionality when used with a string expression on the unnested value. The problem only occurs with string functions with a single input column, which internally tries to use a caching expression selector, which must be used with a dictionary encoded column.

The underlying array column is actually dictionary encoded, but after unnesting it isn't really directly usable anymore (currently at least), but the column metadata is still marked as such, so the expression planning thinks it can use the caching selector because it thinks its dealing with a regular string column, resulting in this error.

In the future i think unnest will be improved to take advantage of the underlying dictionary encoded nature of the array column, but for now I will probably make a simpler fix to just not take that code path (dictionary encoded selectors are for historical reasons pretty coupled with string types at the moment, so its a bit of effort to make that not be the case).

[fabricebaranski]

Thanks Clint