Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with opening using h5netcf #6

Open
SammyAgrawal opened this issue Jun 27, 2024 · 2 comments
Open

Issue with opening using h5netcf #6

SammyAgrawal opened this issue Jun 27, 2024 · 2 comments
Assignees

Comments

@SammyAgrawal
Copy link
Contributor

There seems to be an ingestion issue, not with pgf but with the raw file reading engine.

  • Works when opened with netcdf4 but not remotely h5netcdf
  • Try copy_to_local to see if that circumvents
@SammyAgrawal SammyAgrawal self-assigned this Jun 27, 2024
@SammyAgrawal
Copy link
Contributor Author

SammyAgrawal commented Jun 27, 2024

Reproducible Code Snippet:

import xarray as xr
import fsspec
url = 'https://zenodo.org/records/10513552/files/eNATL60-BLBT02_y2009m07d01.1d_TSWm_60m.nc'
with fsspec.open(url, mode='rb').open() as file: 
    with open("test_ds1.nc", 'wb') as f:
        f.write(file.read())
    ds_local_h5 = xr.open_dataset("test_ds1.nc", engine="h5netcdf", use_cftime=True, decode_cf=False, decode_times=False) 
    ds_local_cdf4 = xr.open_dataset("test_ds1.nc", engine="netcdf4", use_cftime=True, decode_cf=False, decode_times=False)
    ds_virtual = xr.open_dataset(file, engine='h5netcdf')
ds_virtual.time.shape, ds_local_h5.time.shape, ds_local_cdf4.time.shape

The shapes are 0, 0, and 1. Essentially, anything opened with h5netcdf fails to properly load the time dimension leaving copying to local and opening with netcdf4 as the only viable option. This must somehow be an issue with h5netcdf, not with ffspec or the raw bytes (since the downloaded local file works).

I'm not sure if h5netcdf's inability to read the file is a product of the file or the library, but it would seem the latter if other engines are able to succeed?

@SammyAgrawal
Copy link
Contributor Author

| OpenWithXarray(copy_to_local=True, xarray_open_kwargs = {'use_cftime':True, 'engine':"netcdf4"})

Confirm that this fixes the time dimension issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant