-
Notifications
You must be signed in to change notification settings - Fork 91
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tests failing after Distributed upstream changes #1224
Comments
@rjzamora if you have time can you also take a look here ? |
Let me have a go... |
wence-
added a commit
to wence-/dask-cuda
that referenced
this issue
Aug 25, 2023
Now that task keys are no longer strings when stored in the worker data dictionary, we should not stringify the set of keys when staging. - Closes rapidsai#1224
Should be fixed by #1225. |
rapids-bot bot
pushed a commit
that referenced
this issue
Aug 25, 2023
Now that keys are no longer strings there are two places we must adapt here. 1. Explicit comms must no longer manually stringify task keys before staging and intersection with the on-worker data (since that data mapping doesn't use the stringified version) 2. The `zict.File`-backed slow buffer in `DeviceHostFile` needs to translate non-string keys to string keys before writing to disk, to do this, use the same implementation that distributed uses for its own spilling buffer. - Closes #1224 Authors: - Lawrence Mitchell (https://github.com/wence-) Approvers: - Peter Andreas Entschev (https://github.com/pentschev) URL: #1225
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Changes in dask/distributed#8083 have broken all nightly tests. If I didn't overlook anything, only
test_explicit_comms.py
andtest_spill.py
tests were broken.@madsbk would you mind looking at the explicit comms tests? I have to still look in more details of what exactly changed w.r.t. spilling but I see the size of data that gets stored in
zict.Buffer
has changed, which indicates a change in how sizes are computed or that the stringify changes really changed the underlying stored data.The text was updated successfully, but these errors were encountered: