Skip to content

Commit

Permalink
Bump to nvcomp 3.0.6. (rapidsai#15128)
Browse files Browse the repository at this point in the history
This PR bumps nvcomp to 3.0.6. This is needed as a hotfix for rapidsai#15096.

Depends on:
- conda-forge/nvcomp-feedstock#14
- rapidsai/rapids-cmake#542
- rapidsai/kvikio#346

Authors:
   - Bradley Dice (https://github.com/bdice)

Approvers:
   - Lawrence Mitchell (https://github.com/wence-)
   - Ray Douglass (https://github.com/raydouglass)
  • Loading branch information
bdice authored Feb 27, 2024
1 parent ac438c4 commit dc88dcb
Show file tree
Hide file tree
Showing 6 changed files with 15 additions and 4 deletions.
2 changes: 1 addition & 1 deletion conda/environments/all_cuda-118_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ dependencies:
- numpy>=1.21,<1.25
- numpydoc
- nvcc_linux-64=11.8
- nvcomp==3.0.5
- nvcomp==3.0.6
- nvtx>=0.2.1
- packaging
- pandas>=1.3,<1.6.0dev0
Expand Down
2 changes: 1 addition & 1 deletion conda/environments/all_cuda-120_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ dependencies:
- numba>=0.57
- numpy>=1.21,<1.25
- numpydoc
- nvcomp==3.0.5
- nvcomp==3.0.6
- nvtx>=0.2.1
- packaging
- pandas>=1.3,<1.6.0dev0
Expand Down
2 changes: 1 addition & 1 deletion conda/recipes/libcudf/conda_build_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ spdlog_version:
- ">=1.12.0,<1.13"

nvcomp_version:
- "=3.0.5"
- "=3.0.6"

zlib_version:
- ">=1.2.13"
Expand Down
2 changes: 1 addition & 1 deletion dependencies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ dependencies:
- libkvikio==24.2.*
- librdkafka>=1.9.0,<1.10.0a0
# Align nvcomp version with rapids-cmake
- nvcomp==3.0.5
- nvcomp==3.0.6
- spdlog>=1.12.0,<1.13
build_wheels:
common:
Expand Down
Binary file not shown.
11 changes: 11 additions & 0 deletions python/cudf/cudf/tests/test_parquet.py
Original file line number Diff line number Diff line change
Expand Up @@ -3040,3 +3040,14 @@ def test_parquet_reader_multiindex():
def test_parquet_reader_engine_error():
with pytest.raises(ValueError):
cudf.read_parquet(BytesIO(), engine="abc")


def test_parquet_reader_zstd_huff_tables(datadir):
# Ensure that this zstd-compressed file does not overrun buffers. The
# problem was fixed in nvcomp 3.0.6.
# See https://github.com/rapidsai/cudf/issues/15096
fname = datadir / "zstd_huff_tables_bug.parquet"

expected = pa.parquet.read_table(fname).to_pandas()
actual = cudf.read_parquet(fname)
assert_eq(actual, expected)

0 comments on commit dc88dcb

Please sign in to comment.