[QST] NumbaPerformanceWarning triggered by DataFrame.iloc

[gmarkall]

(Background: This originally was brought to my attention by @rnyak, who showed that the warning popped up surprisingly in an NVTabular workflow.)

Numba 0.54 added the emission of warnings when a kernel is launched with a configuration that will result in low occupancy - for examples, see https://nbviewer.jupyter.org/github/gmarkall/numba-examples/blob/cuda-054-release-notebook/notebooks/Numba_054_CUDA_Release_Demo.ipynb#Small-grid-warnings

Using df.iloc results in one of these warnings, because scalar_broadcast_to uses __setitem__ on a Numba device array, which in turn launches a kernel that has low occupancy. This is called from:

cudf/python/cudf/cudf/utils/utils.py

Line 77 in 11781e8

out_col.data_array_view[:] = scalar

To reproduce one of these warnings, running:

import cudf

df = cudf.DataFrame({'a': list(range(20)),
                     'b': list(reversed(range(20))),
                     'c': list(range(20))
                    })

df.iloc[0]

is sufficient, generating the output:

/home/gmarkall/rapidsdev/numba/numba/cuda/compiler.py:872:
    NumbaPerformanceWarning: Grid size (1) < 2 * SM count (144) 
                             will likely result in GPU under utilization due to low occupancy.

I think there are some different conclusions we could come to, but I don't have much feel for which ones are best:

• This doesn't matter, and we needn't do anything about it - perhaps there's no more efficient way to do something simple like this.
• This doesn't matter, but we should find a way to suppress the warning so that it doesn't surprise / confuse users.
• This matters, and we should do something about it - e.g. perhaps finding a way to use memset to set the correct values in the array, perhaps using a different array library's abstraction (for example, would treating the data as a CuPy array give a more efficient __setitem__)?
• This matters, and Numba is being wrong / unhelpful here, and we should convince a Numba maintainer (such as myself :-) ) that __setitem__ of device arrays should not raise warnings like this because it's up to Numba to implement it efficiently, and / or it shouldn't be expected that __setitem__ on a device array is an SOL implementation.

My guess would be that this will pop up in a lot of different workflows, if they use iloc.

Can others add their thoughts / help guide this issue in the right direction please?

[gmarkall]

I just remembered there's a config variable in Numba for this, which we could set, perhaps in https://github.com/rapidsai/cudf/blob/branch-21.10/python/cudf/cudf/__init__.py - if config.CUDA_LOW_OCCUPANCY_WARNINGS is set to 0 it will turn this warning off. E.g.:

import cudf

from numba import config

config.CUDA_LOW_OCCUPANCY_WARNINGS = 0

df = cudf.DataFrame({'a': list(range(20)),
                     'b': list(reversed(range(20))),
                     'c': list(range(20))
                    })

df.iloc[0]

runs with no warnings produced.

[gmarkall]

I've also been made aware of #9236 which seems to be related in that it moves towards using CuPy arrays rather than Numba device arrays, similar / relevant to:

This matters, and we should do something about it - e.g. perhaps finding a way to use memset to set the correct values in the array, perhaps using a different array library's abstraction (for example, would treating the data as a CuPy array give a more efficient setitem)?

[vyasr]

@gmarkall feel free to reach out and we can chat about this (and in the context of #9236) when you're free. In general, though, my inclination is that numba should keep this kind of warning but cuDF should catch and silence it. cuDF disables certain operations that are possible in pandas because they would be prohibitively slow, for example implicit conversion to numpy arrays via __array__ or iteration via __iter__. Indexers are realistically probably too core a feature to disable without just breaking too many user scripts, but I also think we should be discouraging their use as much as possible since I don't believe that even the best possible implementations can avoid significant overhead. The switch to cupy is more about usability (and pandas consistency) than performance; while libraries like cuML can easily ingest numba DeviceNDArrays, users looking to implemnt their own workflows will probably be much more comfortable with numpy array-like semantics.

[kkraus14]

I believe this code is for generating a column of a size given a scalar and predates having the libcudf function for this. In general, the whole scalar_broadcast_to function should be able to be replaced with the libcudf function of make_column_from_scalar anyway.