time array with mixture of types decoded from non-standard calendar

[thenaomig]

Reading a set of files with 360-day calendar with open_mf_dataset(), yields a time array with a mixture of types cftime._cftime.Datetime360Day and Timestamp. The files are CMIP5 output for HadGEM2-ES. I don't think there is a way to force it to use only cftime type for all times? All times should have the same type so slicing is possible (see below).

precip = xy.open_mfdataset(dataDir+'prc_day_HadGEM2-ES_rcp85_r1i1p1_*.nc')
precip['prc'].time 

outputs:

array([cftime._cftime.Datetime360Day(2005, 12, 1, 12, 0, 0, 0, -1, 331),
       cftime._cftime.Datetime360Day(2005, 12, 2, 12, 0, 0, 0, -1, 332),
       cftime._cftime.Datetime360Day(2005, 12, 3, 12, 0, 0, 0, -1, 333), ...,
       Timestamp('2099-12-28 12:00:00'), Timestamp('2099-12-29 12:00:00'),
       Timestamp('2099-12-30 12:00:00')], dtype=object)
Coordinates:
  * time     (time) object 2005-12-01 12:00:00 2005-12-02 12:00:00 ...
Attributes:
    standard_name:  time
    long_name:      time
    bounds:         time_bnds
    axis:           T

Problem description

One consistent type is needed for slicing. It would be fine if they were all type cftime.Datetime360Day, as one could then do the following:

startslice = cftime.Datetime360Day(startYear, 1, 1,0,0,0,0,-1,1)
endslice = cftime.Datetime360Day(endYear, 12, 30,23,0,0,0,-1,360)
precip.sel(time=slice(startslice,endslice))

but with the mixture of types, this returns a KeyError

Expected Output

For some reason with another set of 360-day calendar files (from HadGEM2-CC for what it's worth) it did decode for me with all one type, the time array looks as expected, and slicing as above works:

array([cftime._cftime.Datetime360Day(2005, 12, 1, 12, 0, 0, 0, -1, 331),
       cftime._cftime.Datetime360Day(2005, 12, 2, 12, 0, 0, 0, -1, 332),
       cftime._cftime.Datetime360Day(2005, 12, 3, 12, 0, 0, 0, -1, 333), ...,
       cftime._cftime.Datetime360Day(2100, 12, 28, 12, 0, 0, 0, -1, 358),
       cftime._cftime.Datetime360Day(2100, 12, 29, 12, 0, 0, 0, -1, 359),
       cftime._cftime.Datetime360Day(2100, 12, 30, 12, 0, 0, 0, -1, 360)],
      dtype=object)
Coordinates:
  * time     (time) object 2005-12-01 12:00:00 2005-12-02 12:00:00 ...
Attributes:
    standard_name:  time
    long_name:      time
    bounds:         time_bnds
    axis:           T

Output of xr.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.15.final.0
python-bits: 64
OS: Linux
OS-release: 3.10.0-693.21.1.el7.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: None.None

xarray: 0.10.3
pandas: 0.22.0
numpy: 1.14.3
scipy: 1.1.0
netCDF4: 1.4.0
h5netcdf: 0.5.1
h5py: 2.7.1
Nio: None
zarr: None
bottleneck: 1.2.1
cyordereddict: 1.0.0
dask: 0.17.4
distributed: 1.21.8
matplotlib: 2.2.2
cartopy: 0.16.0
seaborn: 0.8.1
setuptools: 39.1.0
pip: 9.0.3
conda: 4.5.3
pytest: None
IPython: 5.6.0
sphinx: None

[jhamman]

@thenaomig - thanks for the report. Can you confirm that all the files you are opening have the same time format (e.g. units)?

It may also be worth trying your example with the @spencerkclark's CFTimeIndex (#1252).

I think your example may work much better with the following approach:

with xr.set_options(enable_cftimeindex=True):
    precip = xy.open_mfdataset(dataDir+'prc_day_HadGEM2-ES_rcp85_r1i1p1_*.nc')

[thenaomig]

Double-checked all the files do have the same time units.
Trying with the new CFTimeIndex approach worked! Thanks @spencerkclark, and @jhamman for quick reply. Hope this gets merged into the release version soon.