DO-NOT-MERGE: Jackson access issue

[pan3793]

While investigating #10447, I suspect there are something wrong within Gradle itself.

This simple change produces the following errors.

> Task :iceberg-mr:test FAILED
TestJackson > testAccessJsonReadFeatureEnum() FAILED
    java.lang.NoSuchFieldError: ALLOW_LEADING_PLUS_SIGN_FOR_NUMBERS
        at com.fasterxml.jackson.core.json.JsonReadFeature.<clinit>(JsonReadFeature.java:121)
        at org.apache.iceberg.mr.TestJackson.testAccessJsonReadFeatureEnum(TestJackson.java:30)

[pan3793]

@nastra thought?

[nastra]

The error indicates that ALLOW_LEADING_PLUS_SIGN_FOR_NUMBERS isn't available in the Jackson version that is being used at runtime when the test is executed. I checked and ALLOW_LEADING_PLUS_SIGN_FOR_NUMBERS was introduced with Jackson 2.14, meaning that the test is using an earlier Jackson version.

You could try and check which dependencies are being used exactly via ./gradlew :iceberg-mr:dependencies

[pan3793]

@nastra I don't think this is a simple "version not match" issue.

1. the code accesses ALLOW_LEADING_DECIMAL_POINT_FOR_NUMBERS(not the reported one). it should be available.
2. the code accesses an Enum value directly without reflection, there is no compile time error but a runtime error.

[Vampire]

org.apache.calcite.avatica:avatica:1.8.0 is misbehaving.
It shades an old version of jackson-core without relocating it.
You have that dependency in your test classpaths and it comes in the classpath before the jackson-core dependency.
It does not contain com.fasterxml.jackson.core.json.JsonReadFeature, so the one from jackson-core is used.
com.fasterxml.jackson.core.json.JsonReadFeature of the newer jackson-core uses com.fasterxml.jackson.core.JsonParser.Feature.ALLOW_LEADING_PLUS_SIGN_FOR_NUMBERS in its static initialization.
com.fasterxml.jackson.core.JsonParser.Feature is taken from avatica but is so old that it of course does not have that constant, so it fails.

Avatica 1.8.0 was released 2016, so you should probably update it to a newer version.
The concrete problem should indeed be fixed in 1.9.0 already with resolving this issue: https://issues.apache.org/jira/browse/CALCITE-1224

Before they relocated some shaded dependencies but missed Jackson.
After they released non-shaded version and a shaded version and fixed the relocation in the shaded version to include Jackson.

[pan3793]

@Vampire big thanks! I confirmed that test passed after removing org.apache.calcite.avatica:avatica, I'm going to evaluate if we can exclude this dep directly or upgrade to a newer version.

[pan3793]

Temporary conclusion:

Seems we must disable CBO and exclude calcite&avatica to address such a Jackson conflict issue.

The combination of Hive 2.3 strictly depends on calcite 1.10, and calcite 1.10 strictly depends on avatica 1.8. Simply bumping avatica or calcite causes runtime linkage issues.

@pvary can we disable CBO in Iceberg Hive test? otherwise, we can not upgrade Hive 2.3.10.

[pvary]

@pvary can we disable CBO in Iceberg Hive test? otherwise, we can not upgrade Hive 2.3.10.

Disabling the test would just hide the issue that Hive 2.3.10 is not supported with CBO enabled.

Do we have a clear understanding that why and when was the dependency updated? Are the owners of the change aware of this issue? Are there any alternatives to accepting that the Hive integration is not working anymore?