test_from_dict_backends failure on gpuCI

[jrbourbeau]

See #9675 (review)

10:10:22 ________________________ test_from_dict_backends[cudf] _________________________
10:10:22 [gw2] linux -- Python 3.9.13 /opt/conda/envs/dask/bin/python3.9
10:10:22 
10:10:22 backend = 'cudf'
10:10:22 
10:10:22     @pytest.mark.gpu
10:10:22     @pytest.mark.parametrize("backend", ["pandas", "cudf"])
10:10:22     def test_from_dict_backends(backend):
10:10:22         _lib = pytest.importorskip(backend)
10:10:22         with config.set({"dataframe.backend": backend}):
10:10:22             data = {"a": [1, 2, 3, 4], "B": [10, 11, 12, 13]}
10:10:22             expected = _lib.DataFrame(data)
10:10:22     
10:10:22             # Check dd.from_dict API
10:10:22             got = dd.from_dict(data, npartitions=2)
10:10:22             assert_eq(expected, got)
10:10:22     
10:10:22             # Check from_dict classmethod
10:10:22 >           got_classmethod = got.from_dict(data, npartitions=2)
10:10:22 
10:10:22 dask/dataframe/io/tests/test_io.py:988: 
10:10:22 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
10:10:22 dask/dataframe/core.py:6054: in from_dict
10:10:22     return from_dict(
10:10:22 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
10:10:22 
10:10:22 args = ({'B': [10, 11, 12, 13], 'a': [1, 2, 3, 4]}, 2)
10:10:22 kwargs = {'columns': None, 'constructor': <class 'cudf.core.dataframe.DataFrame'>, 'dtype': None, 'orient': 'columns'}
10:10:22 
10:10:22     @wraps(fn)
10:10:22     def wrapper(*args, **kwargs):
10:10:22 >       return getattr(self, dispatch_name)(*args, **kwargs)
10:10:22 E       TypeError: from_dict() got an unexpected keyword argument 'constructor'
10:10:22 
10:10:22 dask/backends.py:122: TypeError

cc @rjzamora @galipremsagar @charlesbluca

[jrbourbeau]

From skimming the above traceback and some relevant parts of cudf, my guess is this error is because CudfBackendEntrypoint.from_dict doesn't have a constructor= keyword (though it's a bit convoluted figuring out what's being called where from the backend dispatching)

[galipremsagar]

I'm looking into it.

[rjzamora]

Thanks for raising this issue @jrbourbeau .

@galipremsagar - It looks like we are missing the constructor kwarg here (my fault for missing this earlier).

[rjzamora]

From skimming the above traceback and some relevant parts of cudf, my guess is this error is because CudfBackendEntrypoint.from_dict doesn't have a constructor= keyword (though it's a bit convoluted figuring out what's being called where from the backend dispatching)

Agree @jrbourbeau . I think we should add a try/except in the register_inplace wrapper to make it more obvious when there is an error in the entrypoint method.

[galipremsagar]

register_inplace

@rjzamora Would want to open a PR with dask? I'll be opening a PR in dask-cudf

[jrbourbeau]

I think we should add a try/except in the register_inplace wrapper to make it more obvious when there is an error in the entrypoint method.

+1 for more informative error messages

I'll be opening a PR in dask-cudf

Sweet, thanks @galipremsagar

FWIW I'm not viewing this as a blocker for the dask / distributed release today as the patch is needed in dask-cudf

[rjzamora]

FWIW I'm not viewing this as a blocker for the dask / distributed release today as the patch is needed in dask-cudf

Yeah, totally agree that this is not a dask blocker. Thanks for confirming!

[rjzamora]

This issue should now be resolved in cudf. Thanks @galipremsagar for fixing and @jrbourbeau for surfacing!