21 Pull requests merged by 10 people

• Spark: support read of partition metadata column when table is over 1k

  #10547 merged Jul 5, 2024

• Migrate source package in Flink

  #10632 merged Jul 5, 2024

• Core: Fix create v1 table on REST Catalog

  #10369 merged Jul 5, 2024

• Build: Use official revapi Gradle plugin

  #10631 merged Jul 5, 2024

• AWS: Retain Glue Catalog column comment after updating Iceberg table

  #10276 merged Jul 5, 2024

• Core: Handles possible heap data corruption of OAuth2Util.AuthSession#headers

  #10615 merged Jul 4, 2024

• Flink: Fix long casting issues

  #10629 merged Jul 4, 2024

• Core: Assume issued_token_type is access_token to fully comply with RFC 6749

  #10314 merged Jul 4, 2024

• Apply IntelliJ inspection findings to older Spark + Flink versions

  #10625 merged Jul 4, 2024

• REST: disallow overriding "credential" in table sessions

  #10345 merged Jul 4, 2024

• Fix some long casting issues

  #10580 merged Jul 3, 2024

• Enable the Gradle build cache

  #10602 merged Jul 3, 2024

• Address Intellij inspection findings

  #10583 merged Jul 3, 2024

• Backport HadoopCatalog related classes in Flink

  #10620 merged Jul 2, 2024

• Build: Bump io.github.goooler.shadow:shadow-gradle-plugin from 8.1.7 to 8.1.8

  #10612 merged Jul 2, 2024

• Core: Handles potential NPE in RESTSessionCatalog#newSessionCache

  #10607 merged Jul 2, 2024

• Core: Fix ParallelIterable memory leak where queue continues to be populated even after iterator close

  #9402 merged Jul 1, 2024

• Migrate HadoopCatalog related tests in Flink

  #10358 merged Jul 1, 2024

• Build: Bump software.amazon.awssdk:bom from 2.26.7 to 2.26.12

  #10611 merged Jun 30, 2024

• Build: Bump nessie from 0.91.2 to 0.91.3

  #10608 merged Jun 30, 2024

• Fix incorrect double-checked-locking around TestStreamScanSql#tEnv

  #10605 merged Jun 29, 2024

16 Pull requests opened by 13 people

• Ability to build for all Scala versions

  #10606 opened Jun 29, 2024

• Build: Bump junit from 5.10.1 to 5.10.3

  #10609 opened Jun 30, 2024

• Build: Bump com.aliyun.oss:aliyun-sdk-oss from 3.10.2 to 3.18.0

  #10610 opened Jun 30, 2024

• Core: Fix possible proken in `Tasks.Builder.runSingleThreaded`

  #10613 opened Jun 30, 2024

• REST: refactor OAuth logic into AuthManager Interface

  #10621 opened Jul 2, 2024

• WIP: Initial Support for Spark 4.0

  #10622 opened Jul 2, 2024

• core:Refactor the code of HadoopTableOptions

  #10623 opened Jul 3, 2024

• Core: Refactor ZOrderByteUtils

  #10624 opened Jul 3, 2024

• Fixed an incorrect example

  #10627 opened Jul 4, 2024

• Support create multiple element ns together for nessie

  #10630 opened Jul 4, 2024

• Build: Downgrade Gradle from 8.8 to 8.7 due to bug with older OSX versions

  #10637 opened Jul 5, 2024

• Core: improve DefaultErrorHandler message for unhandled codes

  #10640 opened Jul 5, 2024

• Spark 3.3/3.4: support read of partition metadata column when table is over 1k

  #10641 opened Jul 5, 2024

• Add interfaces for Action CheckSnapshotIntegrity

  #10642 opened Jul 5, 2024

• Add interfaces for Action RemoveExpiredFiles

  #10643 opened Jul 5, 2024

• [Docs] Add examples for DataFrame branch writes

  #10644 opened Jul 5, 2024

24 Issues closed by 6 people

• Unable to get field from serde: org.apache.iceberg.mr.hive.HiveIcebergSerDe

  #10633 closed Jul 5, 2024

• Read manifest file Permission denied

  #10634 closed Jul 5, 2024

• Missing Artifacts for Flink v1.19

  #10638 closed Jul 5, 2024

• AWS: Updating Glue catalog table removes column descriptions

  #10220 closed Jul 5, 2024

• Possible heap data corruption of `org.apache.iceberg.rest.auth.OAuth2Util.AuthSession#headers`

  #10591 closed Jul 4, 2024

• Avro inferred mapping issues with nested struct

  #2774 closed Jul 4, 2024

• The job of writing iceberg v2 table threw a validException when testing the merge of iceberg v2 table

  #2773 closed Jul 3, 2024

• Does MERGE INTO operations support hidden partition on timestamp columns?

  #2765 closed Jul 3, 2024

• Deduplication support in RewriteDataFilesAction

  #2764 closed Jul 3, 2024

• Support TIME_MILLIS in Arrow code

  #2755 closed Jul 3, 2024

• Token expiration issued when using WORKER_POOL with Spark Thrift Server

  #2753 closed Jul 3, 2024

• ORC support broken for Java 8

  #10598 closed Jul 2, 2024

• Question about `day` transform result type

  #10616 closed Jul 2, 2024

• Potential NPE in `o.a.i.rest.RESTSessionCatalog#newSessionCache`

  #10593 closed Jul 2, 2024

• Manifests reader queue in ParallelIterable is unlimited caused OOM

  #7843 closed Jul 1, 2024

• "ICEBERG" vs. "iceberg" for "table_type" parameter

  #2751 closed Jul 1, 2024

• Support hive metastore client retries

  #2741 closed Jul 1, 2024

• Getting - java.lang.IllegalArgumentException: Can't zip RDDs with unequal numbers of partitions: List(1, 0) on MERGE INTO

  #2734 closed Jul 1, 2024

• Refactoring table and format doc contents

  #2728 closed Jul 1, 2024

• How to realize Write Iceberg Tables via Hive? (Ideas share)

  #2685 closed Jul 1, 2024

• Does spark integrated iceberg support concurrent insert overwrite

  #2633 closed Jul 1, 2024

• spark3 can't query iceberg: failed to connect to Hive Metastore

  #2359 closed Jun 30, 2024

• Running iceberg with spark 3 in local mode

  #2176 closed Jun 30, 2024

• Incorrect double-checked-locking around `o.a.i.flink.source.TestStreamScanSql#tEnv`

  #10592 closed Jun 29, 2024

10 Issues opened by 9 people

• REST Spec: Server-side Metadata Tables

  #10645 opened Jul 5, 2024

• ugi not correct in WORKER_POOL

  #10639 opened Jul 5, 2024

• [Build] Gradle 8.8 crashed on MacOS 11.x and before

  #10636 opened Jul 5, 2024

• Spark: Add read/write support for UUIDs from bytes

  #10635 opened Jul 5, 2024

• [SUPPORT] Support setting the maximum number of partitions for a table

  #10628 opened Jul 4, 2024

• NullPointerException after deleting old partition column

  #10626 opened Jul 3, 2024

• Overwrite with condition from spark failed

  #10619 opened Jul 2, 2024

• Support server side data commit in REST

  #10618 opened Jul 1, 2024

• Add Multi-Table Transaction API

  #10617 opened Jul 1, 2024

• Allow static credentials for all AWS clients, not only for S3

  #10614 opened Jun 30, 2024

47 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Spark Action to Analyze table

  #10288 commented on Jul 4, 2024 • 29 new comments

• Deprecate `oauth/tokens` endpoint

  #10603 commented on Jul 4, 2024 • 25 new comments

• API: implement types timestamp_ns and timestamptz_ns

  #9008 commented on Jul 4, 2024 • 15 new comments

• Kafka Connect: Commit coordination

  #10351 commented on Jul 3, 2024 • 14 new comments

• Flink: Pre-create fieldGetters to avoid constructing them for each row

  #10565 commented on Jul 5, 2024 • 12 new comments

• OpenAPI: Express server capabilities via /config endpoint

  #9940 commented on Jul 4, 2024 • 11 new comments

• Flink 1.19: Run without Hadoop

  #7369 commented on Jul 2, 2024 • 10 new comments

• Support building with Java 21

  #10474 commented on Jul 2, 2024 • 8 new comments

• Iceberg table not able to read data from S3 after few hours using Athena .

  #9684 commented on Jul 1, 2024 • 4 new comments

• Parquet: Make row-group filters cooperate to filter

  #10090 commented on Jun 30, 2024 • 3 new comments

• Concurrency issues in `o.a.i.hive.MetastoreLock`

  #10595 commented on Jul 2, 2024 • 3 new comments

• Core: Exclude unexpected namespaces JdbcCatalog.listNamespaces

  #10498 commented on Jul 5, 2024 • 3 new comments

• Spec: Make NDV blob metadata property required

  #10549 commented on Jul 5, 2024 • 3 new comments

• Let `./gradlew clean` clean everything

  #10601 commented on Jul 2, 2024 • 3 new comments

• Flink: Fix duplicate data in Flink's upsert writer for format V2

  #10526 commented on Jul 1, 2024 • 2 new comments

• DO-NOT-MERGE: Jackson access issue

  #10460 commented on Jul 3, 2024 • 2 new comments

• Spark: Support alter partition in V2 Catalog SparkCatalog Class

  #3558 commented on Jul 3, 2024 • 2 new comments

• Flink-1.19: Fix the file offset mismatch when Flink reader first seek…

  #10567 commented on Jul 1, 2024 • 2 new comments

• AWS: Add Option to don't write non current columns in glue schema closes #7584

  #9420 commented on Jul 4, 2024 • 2 new comments

• Use SupportsPrefixOperations for Remove OrphanFile Procedure

  #7914 commented on Jul 4, 2024 • 2 new comments

• Support for Flink's SpeculativeExecution in batch execution mode

  #10548 commented on Jul 3, 2024 • 1 new comment

• Docs: Update defaults for distribution mode

  #10575 commented on Jul 3, 2024 • 1 new comment

• Core: Support appending files with different specs

  #9860 commented on Jul 3, 2024 • 1 new comment

• Speed up Spark + Flink unit test execution

  #10581 commented on Jun 29, 2024 • 1 new comment

• `TestS3RestSigner` is flaky

  #10599 commented on Jun 29, 2024 • 1 new comment

• Update Table Error: UPDATE TABLE is not supported temporarily.

  #9960 commented on Jul 1, 2024 • 1 new comment

• Old Avro file not found breaks Athena iceberg table

  #10560 commented on Jul 1, 2024 • 1 new comment

• Support client-side purge in REST catalog

  #10089 commented on Jul 1, 2024 • 1 new comment

• Spark DataFrame write fails if input dataframe has columns in different order than iceberg schema

  #741 commented on Jul 2, 2024 • 1 new comment

• insert to hive table with icberg table format is failing

  #7840 commented on Jul 2, 2024 • 1 new comment

• Allow snapshotting iceberg table (create new table based on certain Iceberg snapshot)

  #2481 commented on Jul 4, 2024 • 1 new comment

• Drop table doesn't clean up metadata and data files

  #2784 commented on Jul 4, 2024 • 1 new comment

• Spark3 DSv2: Handle Snapshots of Type OVERWRITE - while reading Stream from Iceberg table

  #2788 commented on Jul 4, 2024 • 1 new comment

• Running MERGE INTO with more than one WHEN condition fails if the number of columns in the target table is > 321

  #10294 commented on Jul 4, 2024 • 1 new comment

• Alter table to generic types

  #2791 commented on Jul 5, 2024 • 1 new comment

• Add more description to the Write-audit-publish feature

  #2802 commented on Jul 5, 2024 • 1 new comment

• Flink 1.17: Supports batch queries using time ranges

  #7362 commented on Jul 2, 2024 • 1 new comment

• Spark 3.4: Incremental scan specify branch

  #8384 commented on Jul 2, 2024 • 1 new comment

• Hive: Add View support for HIVE catalog

  #9852 commented on Jul 3, 2024 • 0 new comments

• Drop support for Java 8

  #10518 commented on Jul 3, 2024 • 0 new comments

• Iceberg REST Spec Extensions for Access Decision Exchange

  #10395 commented on Jul 1, 2024 • 0 new comments

• Support Securable Objects in Iceberg REST Catalog

  #10407 commented on Jul 1, 2024 • 0 new comments

• Add PrePlanTable and PlanTable Endpoints to open api spec

  #10391 commented on Jul 1, 2024 • 0 new comments

• New REST Catalog Spec/RFC

  #10075 commented on Jul 1, 2024 • 0 new comments

• Security improvements in the Iceberg REST specification

  #10537 commented on Jul 1, 2024 • 0 new comments

• Express REST server capabilities

  #10462 commented on Jul 1, 2024 • 0 new comments

• Relax client.credentials-provider requirements

  #10604 commented on Jul 2, 2024 • 0 new comments