Publishing website 2024/05/03 05:38:02 at commit 782b644

website/generated-content/blog/beam-2.56.0/index.html

@@ -38,8 +38,8 @@
 <script>function showSearch(){addPlaceholder();var e,t=document.querySelector(".searchBar");t.classList.remove("disappear"),e=document.querySelector("#iconsBar"),e.classList.add("disappear")}function addPlaceholder(){$("input:text").attr("placeholder","What are you looking for?")}function endSearch(){var e,t=document.querySelector(".searchBar");t.classList.add("disappear"),e=document.querySelector("#iconsBar"),e.classList.remove("disappear")}function blockScroll(){$("body").toggleClass("fixedPosition")}function openMenu(){addPlaceholder(),blockScroll()}</script><div class="body__contained center no__padding content-up"><article class=post itemscope itemtype=http://schema.org/BlogPosting><div class=post-content><div class=post-info><p>blog & release
        </p><p>2024/05/01</p></div><header class=post-header><h2 itemprop="name headline">Apache Beam 2.56.0</h1><div class=post-info><span>Danny McCormick</span></div></header><div class="arrow-list header-top-margin" itemprop=articleBody><p>We are happy to present the new 2.56.0 release of Beam.
 This release includes both improvements and new functionality.
-See the <a href=/get-started/downloads/#2550-2023-03-25>download page</a> for this release.</p><p>For more information on changes in 2.56.0, check out the <a href=https://github.com/apache/beam/milestone/20>detailed release notes</a>.</p><h2 id=highlights>Highlights</h2><ul><li>Added FlinkRunner for Flink 1.17, removed support for Flink 1.12 and 1.13. Previous version of Pipeline running on Flink 1.16 and below can be upgraded to 1.17, if the Pipeline is first updated to Beam 2.56.0 with the same Flink version. After Pipeline runs with Beam 2.56.0, it should be possible to upgrade to FlinkRunner with Flink 1.17. (<a href=https://github.com/apache/beam/issues/29939>#29939</a>)</li></ul><h2 id=ios>I/Os</h2><ul><li>Upgraded Avro version to 1.11.3, kafka-avro-serializer and kafka-schema-registry-client versions to 7.6.0 (Java) (<a href=https://github.com/apache/beam/pull/30638>#30638</a>).
-The newer Avro package is known to have breaking changes. If you are affected, you can keep pinned to older Avro versions which are also tested with Beam.</li></ul><h2 id=new-features--improvements>New Features / Improvements</h2><ul><li>Profiling of Cythonized code has been disabled by default. This might improve performance for some Python pipelines (<a href=https://github.com/apache/beam/pull/30938>#30938</a>).</li><li>Bigtable enrichment handler now accepts a custom function to build a composite row key. (Python) (<a href=https://github.com/apache/beam/issues/30975>#30974</a>).</li></ul><h2 id=breaking-changes>Breaking Changes</h2><ul><li>Default consumer polling timeout for KafkaIO.Read was increased from 1 second to 2 seconds. Use KafkaIO.read().withConsumerPollingTimeout(Duration duration) to configure this timeout value when necessary (<a href=https://github.com/apache/beam/issues/30870>#30870</a>).</li><li>Python Dataflow users no longer need to manually specify &ndash;streaming for pipelines using unbounded sources such as ReadFromPubSub.</li></ul><h2 id=bugfixes>Bugfixes</h2><ul><li>Fixed locking issue when shutting down inactive bundle processors. Symptoms of this issue include slowness or stuckness in long-running jobs (Python) (<a href=https://github.com/apache/beam/pull/30679>#30679</a>).</li><li>Fixed logging issue that caused silecing the pip output when installing of dependencies provided in <code>--requirements_file</code> (Python).</li></ul><h2 id=list-of-contributors>List of Contributors</h2><p>According to git shortlog, the following people contributed to the 2.56.0 release. Thank you to all contributors!</p><p>Abacn</p><p>Ahmed Abualsaud</p><p>Andrei Gurau</p><p>Andrey Devyatkin</p><p>Aravind Pedapudi</p><p>Arun Pandian</p><p>Arvind Ram</p><p>Bartosz Zablocki</p><p>Brachi Packter</p><p>Byron Ellis</p><p>Chamikara Jayalath</p><p>Clement DAL PALU</p><p>Damon</p><p>Danny McCormick</p><p>Daria Bezkorovaina</p><p>Dip Patel</p><p>Evan Burrell</p><p>Hai Joey Tran</p><p>Jack McCluskey</p><p>Jan Lukavský</p><p>JayajP</p><p>Jeff Kinard</p><p>Julien Tournay</p><p>Kenneth Knowles</p><p>Luís Bianchin</p><p>Maciej Szwaja</p><p>Melody Shen</p><p>Oleh Borysevych</p><p>Pablo Estrada</p><p>Rebecca Szper</p><p>Ritesh Ghorse</p><p>Robert Bradshaw</p><p>Sam Whittle</p><p>Sergei Lilichenko</p><p>Shahar Epstein</p><p>Shunping Huang</p><p>Svetak Sundhar</p><p>Timothy Itodo</p><p>Veronica Wasson</p><p>Vitaly Terentyev</p><p>Vlado Djerek</p><p>Yi Hu</p><p>akashorabek</p><p>bzablocki</p><p>clmccart</p><p>damccorm</p><p>dependabot[bot]</p><p>dmitryor</p><p>github-actions[bot]</p><p>liferoad</p><p>martin trieu</p><p>tvalentyn</p><p>xianhualiu</p></div></div><div class=blog-content><h2>Latest from the blog</h2></div><div class=posts-list><a class=post-card href=/blog/beam-yaml-release/ data-categories="blog "><div class="post-info post-category"><p>blog</p><p>2024/04/11</p></div><div class=post><p class=post-title>Introducing Beam YAML: Apache Beam's First No-code SDK</p><p class=post-info>Jeff Kinard</p></div></a><a class=post-card href=/blog/beam-2.55.0/ data-categories="blog release "><div class="post-info post-category"><p>blog & release
+See the <a href=/get-started/downloads/#2550-2023-03-25>download page</a> for this release.</p><p>For more information on changes in 2.56.0, check out the <a href=https://github.com/apache/beam/milestone/20>detailed release notes</a>.</p><h2 id=highlights>Highlights</h2><ul><li>Added FlinkRunner for Flink 1.17, removed support for Flink 1.12 and 1.13. Previous version of Pipeline running on Flink 1.16 and below can be upgraded to 1.17, if the Pipeline is first updated to Beam 2.56.0 with the same Flink version. After Pipeline runs with Beam 2.56.0, it should be possible to upgrade to FlinkRunner with Flink 1.17. (<a href=https://github.com/apache/beam/issues/29939>#29939</a>)</li><li>New Managed I/O Java API (<a href=https://github.com/apache/beam/pull/30830>#30830</a>).</li><li>New Ordered Processing PTransform added for processing order-sensitive stateful data (<a href=https://github.com/apache/beam/pull/30735>#30735</a>).</li></ul><h2 id=ios>I/Os</h2><ul><li>Upgraded Avro version to 1.11.3, kafka-avro-serializer and kafka-schema-registry-client versions to 7.6.0 (Java) (<a href=https://github.com/apache/beam/pull/30638>#30638</a>).
+The newer Avro package is known to have breaking changes. If you are affected, you can keep pinned to older Avro versions which are also tested with Beam.</li><li>Iceberg read/write support is available through the new Managed I/O Java API (<a href=https://github.com/apache/beam/pull/30830>#30830</a>).</li></ul><h2 id=new-features--improvements>New Features / Improvements</h2><ul><li>Profiling of Cythonized code has been disabled by default. This might improve performance for some Python pipelines (<a href=https://github.com/apache/beam/pull/30938>#30938</a>).</li><li>Bigtable enrichment handler now accepts a custom function to build a composite row key. (Python) (<a href=https://github.com/apache/beam/issues/30975>#30974</a>).</li></ul><h2 id=breaking-changes>Breaking Changes</h2><ul><li>Default consumer polling timeout for KafkaIO.Read was increased from 1 second to 2 seconds. Use KafkaIO.read().withConsumerPollingTimeout(Duration duration) to configure this timeout value when necessary (<a href=https://github.com/apache/beam/issues/30870>#30870</a>).</li><li>Python Dataflow users no longer need to manually specify &ndash;streaming for pipelines using unbounded sources such as ReadFromPubSub.</li></ul><h2 id=bugfixes>Bugfixes</h2><ul><li>Fixed locking issue when shutting down inactive bundle processors. Symptoms of this issue include slowness or stuckness in long-running jobs (Python) (<a href=https://github.com/apache/beam/pull/30679>#30679</a>).</li><li>Fixed logging issue that caused silecing the pip output when installing of dependencies provided in <code>--requirements_file</code> (Python).</li></ul><h2 id=list-of-contributors>List of Contributors</h2><p>According to git shortlog, the following people contributed to the 2.56.0 release. Thank you to all contributors!</p><p>Abacn</p><p>Ahmed Abualsaud</p><p>Andrei Gurau</p><p>Andrey Devyatkin</p><p>Aravind Pedapudi</p><p>Arun Pandian</p><p>Arvind Ram</p><p>Bartosz Zablocki</p><p>Brachi Packter</p><p>Byron Ellis</p><p>Chamikara Jayalath</p><p>Clement DAL PALU</p><p>Damon</p><p>Danny McCormick</p><p>Daria Bezkorovaina</p><p>Dip Patel</p><p>Evan Burrell</p><p>Hai Joey Tran</p><p>Jack McCluskey</p><p>Jan Lukavský</p><p>JayajP</p><p>Jeff Kinard</p><p>Julien Tournay</p><p>Kenneth Knowles</p><p>Luís Bianchin</p><p>Maciej Szwaja</p><p>Melody Shen</p><p>Oleh Borysevych</p><p>Pablo Estrada</p><p>Rebecca Szper</p><p>Ritesh Ghorse</p><p>Robert Bradshaw</p><p>Sam Whittle</p><p>Sergei Lilichenko</p><p>Shahar Epstein</p><p>Shunping Huang</p><p>Svetak Sundhar</p><p>Timothy Itodo</p><p>Veronica Wasson</p><p>Vitaly Terentyev</p><p>Vlado Djerek</p><p>Yi Hu</p><p>akashorabek</p><p>bzablocki</p><p>clmccart</p><p>damccorm</p><p>dependabot[bot]</p><p>dmitryor</p><p>github-actions[bot]</p><p>liferoad</p><p>martin trieu</p><p>tvalentyn</p><p>xianhualiu</p></div></div><div class=blog-content><h2>Latest from the blog</h2></div><div class=posts-list><a class=post-card href=/blog/beam-yaml-release/ data-categories="blog "><div class="post-info post-category"><p>blog</p><p>2024/04/11</p></div><div class=post><p class=post-title>Introducing Beam YAML: Apache Beam's First No-code SDK</p><p class=post-info>Jeff Kinard</p></div></a><a class=post-card href=/blog/beam-2.55.0/ data-categories="blog release "><div class="post-info post-category"><p>blog & release
 
    </p><p>2024/03/25</p></div><div class=post><p class=post-title>Apache Beam 2.55.0</p><p class=post-info>Yi Hu</p></div></a><a class=post-card href=/blog/beam-2.54.0/ data-categories="blog release "><div class="post-info post-category"><p>blog & release
 

website/generated-content/blog/index.xml

@@ -17,11 +17,14 @@ See the <a href="/get-started/downloads/#2550-2023-03-25">download page</a
 <h2 id="highlights">Highlights</h2>
 <ul>
 <li>Added FlinkRunner for Flink 1.17, removed support for Flink 1.12 and 1.13. Previous version of Pipeline running on Flink 1.16 and below can be upgraded to 1.17, if the Pipeline is first updated to Beam 2.56.0 with the same Flink version. After Pipeline runs with Beam 2.56.0, it should be possible to upgrade to FlinkRunner with Flink 1.17. (<a href="https://github.com/apache/beam/issues/29939">#29939</a>)</li>
+<li>New Managed I/O Java API (<a href="https://github.com/apache/beam/pull/30830">#30830</a>).</li>
+<li>New Ordered Processing PTransform added for processing order-sensitive stateful data (<a href="https://github.com/apache/beam/pull/30735">#30735</a>).</li>
 </ul>
 <h2 id="ios">I/Os</h2>
 <ul>
 <li>Upgraded Avro version to 1.11.3, kafka-avro-serializer and kafka-schema-registry-client versions to 7.6.0 (Java) (<a href="https://github.com/apache/beam/pull/30638">#30638</a>).
 The newer Avro package is known to have breaking changes. If you are affected, you can keep pinned to older Avro versions which are also tested with Beam.</li>
+<li>Iceberg read/write support is available through the new Managed I/O Java API (<a href="https://github.com/apache/beam/pull/30830">#30830</a>).</li>
 </ul>
 <h2 id="new-features--improvements">New Features / Improvements</h2>
 <ul>

website/generated-content/categories/blog/index.xml

@@ -17,11 +17,14 @@ See the <a href="/get-started/downloads/#2550-2023-03-25">download page</a
 <h2 id="highlights">Highlights</h2>
 <ul>
 <li>Added FlinkRunner for Flink 1.17, removed support for Flink 1.12 and 1.13. Previous version of Pipeline running on Flink 1.16 and below can be upgraded to 1.17, if the Pipeline is first updated to Beam 2.56.0 with the same Flink version. After Pipeline runs with Beam 2.56.0, it should be possible to upgrade to FlinkRunner with Flink 1.17. (<a href="https://github.com/apache/beam/issues/29939">#29939</a>)</li>
+<li>New Managed I/O Java API (<a href="https://github.com/apache/beam/pull/30830">#30830</a>).</li>
+<li>New Ordered Processing PTransform added for processing order-sensitive stateful data (<a href="https://github.com/apache/beam/pull/30735">#30735</a>).</li>
 </ul>
 <h2 id="ios">I/Os</h2>
 <ul>
 <li>Upgraded Avro version to 1.11.3, kafka-avro-serializer and kafka-schema-registry-client versions to 7.6.0 (Java) (<a href="https://github.com/apache/beam/pull/30638">#30638</a>).
 The newer Avro package is known to have breaking changes. If you are affected, you can keep pinned to older Avro versions which are also tested with Beam.</li>
+<li>Iceberg read/write support is available through the new Managed I/O Java API (<a href="https://github.com/apache/beam/pull/30830">#30830</a>).</li>
 </ul>
 <h2 id="new-features--improvements">New Features / Improvements</h2>
 <ul>

website/generated-content/categories/release/index.xml

@@ -17,11 +17,14 @@ See the <a href="/get-started/downloads/#2550-2023-03-25">download page</a
 <h2 id="highlights">Highlights</h2>
 <ul>
 <li>Added FlinkRunner for Flink 1.17, removed support for Flink 1.12 and 1.13. Previous version of Pipeline running on Flink 1.16 and below can be upgraded to 1.17, if the Pipeline is first updated to Beam 2.56.0 with the same Flink version. After Pipeline runs with Beam 2.56.0, it should be possible to upgrade to FlinkRunner with Flink 1.17. (<a href="https://github.com/apache/beam/issues/29939">#29939</a>)</li>
+<li>New Managed I/O Java API (<a href="https://github.com/apache/beam/pull/30830">#30830</a>).</li>
+<li>New Ordered Processing PTransform added for processing order-sensitive stateful data (<a href="https://github.com/apache/beam/pull/30735">#30735</a>).</li>
 </ul>
 <h2 id="ios">I/Os</h2>
 <ul>
 <li>Upgraded Avro version to 1.11.3, kafka-avro-serializer and kafka-schema-registry-client versions to 7.6.0 (Java) (<a href="https://github.com/apache/beam/pull/30638">#30638</a>).
 The newer Avro package is known to have breaking changes. If you are affected, you can keep pinned to older Avro versions which are also tested with Beam.</li>
+<li>Iceberg read/write support is available through the new Managed I/O Java API (<a href="https://github.com/apache/beam/pull/30830">#30830</a>).</li>
 </ul>
 <h2 id="new-features--improvements">New Features / Improvements</h2>
 <ul>

website/generated-content/documentation/io/connectors/index.html

@@ -139,7 +139,7 @@
 <a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/sparkreceiver/SparkReceiverIO.html>native</a></td><td>Not available</td><td>Not available</td><td>Not available</td><td class=absent>✘</td><td class=present>✔</td></tr><tr><td>CdapIO (<a href=/documentation/io/built-in/cdap/>guide</a>)</td><td class=present>✔</td><td class=present>✔</td><td class=present>✔
 <a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/cdap/CdapIO.html>native</a></td><td>Not available</td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=present>✔</td></tr><tr><td>SingleStoreDB (<a href=/documentation/io/built-in/singlestore/>guide</a>)</td><td class=present>✔</td><td class=present>✔</td><td class=present>✔
 <a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/singlestore/SingleStoreIO.html>native</a></td><td>Not available</td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=absent>✘</td></tr><tr><td>GoogleAdsIO</td><td class=present>✔</td><td class=present>✔</td><td class=present>✔
-<a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/googleads/GoogleAdsIO.html>native</a></td><td>Not available</td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=present>✔</td></tr><tr><td>Web APIs (<a href=/documentation/io/built-in/webapis/>guide</a>)</td><td class=present>✔</td><td class=present>✔</td><td class=present>✔ <a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/io/requestresponse/RequestResponseIO.html>native</a></td><td class=present>✔ <a href="https://beam.apache.org/releases/pydoc/current/apache_beam.io.requestresponse.html?highlight=requestresponse#apache_beam.io.requestresponse.RequestResponseIO">native</a></td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=present>✔</td></tr></table></div><h2 id=other-io-connectors-for-apache-beam>Other I/O Connectors for Apache Beam</h2><div class=table-container-wrapper><table class="table table-bordered table-connectors"><tr><th>Connector Name</th><th>Source Supported</th><th>Sink Supported</th><th>Java</th><th>Python</th><th>Go</th><th>Typescript</th><th>Batch Supported</th><th>Streaming Supported</th></tr><tr><td><a href=https://github.com/SolaceProducts/solace-apache-beam>Solace</a></td><td class=present>✔</td><td class=absent>✘</td><td class=present>✔
+<a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/googleads/GoogleAdsIO.html>native</a></td><td>Not available</td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=present>✔</td></tr><tr><td>Web APIs (<a href=/documentation/io/built-in/webapis/>guide</a>)</td><td class=present>✔</td><td class=present>✔</td><td class=present>✔ <a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/io/requestresponse/RequestResponseIO.html>native</a></td><td class=present>✔ <a href="https://beam.apache.org/releases/pydoc/current/apache_beam.io.requestresponse.html?highlight=requestresponse#apache_beam.io.requestresponse.RequestResponseIO">native</a></td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=present>✔</td></tr><tr><td>Iceberg (Managed I/O)</td><td class=present>✔</td><td class=present>✔</td><td class=present>✔ <a href=https://beam.apache.org/releases/javadoc/2.56.0/org/apache/beam/sdk/managed/Managed.html>native</a></td><td>Not available</td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=present>✔</td></tr></table></div><h2 id=other-io-connectors-for-apache-beam>Other I/O Connectors for Apache Beam</h2><div class=table-container-wrapper><table class="table table-bordered table-connectors"><tr><th>Connector Name</th><th>Source Supported</th><th>Sink Supported</th><th>Java</th><th>Python</th><th>Go</th><th>Typescript</th><th>Batch Supported</th><th>Streaming Supported</th></tr><tr><td><a href=https://github.com/SolaceProducts/solace-apache-beam>Solace</a></td><td class=present>✔</td><td class=absent>✘</td><td class=present>✔
 native</td><td>Not available</td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=present>✔</td></tr><tr><td><a href=https://github.com/google/hana-bq-beam-connector>SAP Hana to Google BigQuery</a></td><td class=present>✔</td><td class=present>✔</td><td class=present>✔
 native</td><td>Not available</td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=absent>✘</td></tr><tr><td><a href=https://github.com/esakik/beam-mysql-connector>MySQL</a></td><td class=present>✔</td><td class=present>✔</td><td>Not available</td><td class=present>✔
 native</td><td>Not available</td><td>Not available</td><td class=present>✔</td><td class=absent>✘</td></tr><tr><td><a href=https://github.com/Refinitiv/trep-websockets-beam-io>TrepWsIO</a></td><td class=present>✔</td><td class=absent>✘</td><td class=present>✔