-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Threading backend using omp_set_nested
which is deprecated in OpenMP 5?
#5275
Comments
…ba.vectorize OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead. numba/numba#5275 (comment) Conclusion: 1. numba.vectorize is really fast 2. using Enum.mem.value in numba.vectorize function seems good 3. use parallel flag only when the number of points is extreamly large (>=100x100x100) 4. numba.njit is bad?! even slower than numpy when number of points gets larger 5. fastmath does nothing?
Am I correct to assume that this is only a warning? I am seeing this quite often but it looks like there is nothing that the end user can do to resolve this. Is that right? If so, is there a recommended way to suppress this warning? I've tried the typical (which ignores other warnings as well - 😢 ):
but it still persists. Any suggestions would be greatly appreciated! |
@seanlaw You're right, it is a warning only. I looked in the past at fixing this by only using the non-deprecated API on newer OpenMPs and using the old one on older OpenMP versions, but I found there are some implementations that aren't compliant with the spec around this area (I can't recall the details and might have discussed it outside of Github issues unfortunately) so it's very hard to detect the right thing to do whilst supporting various different implementations on different platforms. The way to suppress it (if it is possible) will depend on your OpenMP implementation (e.g. setting the environment variable |
@gmarkall So, this is coming up frequently with many of our STUMPY users who are new to Python and who may not realize that it is nothing to be too concerned with and nothing for them to do. This is why I'm trying to look for a way to hide this specific warning from all users who install STUMPY but without hiding other important warnings. Does that make sense? |
Ah, I found the PR I tried: #7511 |
This does make sense. I think one workaround could be not to use the OpenMP backend - you could set the backend using the options documented at https://numba.readthedocs.io/en/latest/reference/envvars.html#threading-control. |
Thank you for your input. In addition to the OpenMP warning, users are also getting this TBB warning so, in order to avoid both warnings, does that mean that I'm basically left with setting the I certainly want all users to be able to leverage the fastest backend available to them without needing to ask/force them to install additional software. This is why I'd rather catch these two specific warnings rather than turn off the backends for all users but I understand/ackowledge that this stuff is HARD! |
Workqueue is always available as it is part of Numba that only relies on the underlying OS's threading layer (POSIX threads or Windows threads). However, it's not threadsafe (you can't use multiple threads of your own safely with it) so it might be limiting for your use case. I don't know how well it performs compared to the other backends, but I'd be surprised if it was as good in all cases as OpenMP or TBB.
If you can figure out a way to reliably replace this call: numba/numba/np/ufunc/omppool.cpp Line 227 in 0994f97
with a call to @stuartarchibald Maybe we could make it a requirement to have an OpenMP 3.0 or later for the OMP backend, and use |
Ohh, does this mean that this warning is coming from the CPP layer and therefore we can't catch/filter it in the Python layer? At the end of the day, I'm not trying to stop the warning from happening. Instead, I just want to filter it so that the end user doesn't see it. |
That's correct, it's the underlying OpenMP implementation, whatever that is (it could be written in Fortran or anything else really). |
I think this is probably ok:
@gmarkall I suggest we go with patching up the OMP backend to use the OpenMP 3 or later API and seeing what (if anything) breaks on the farm. The use of true nested behaviour is quite rare and so it may well be that the impact of this change is relatively small anyway. Further, the threading layers "safe load" and will fall back to other implementations/suggest how to create a fall back if there's a problem, which should make it reasonably easy for users with 2.5 or incomplete 3.x+ to work out an alternative. I do recall one issue was that the binding between OpenMP version spec and conda package name for the library wasn't particularly clear (I'd really like to be able to constrain with something like |
omp_set_nested() is deprecated and often results in a warning on recent installations. Following the discussion in Issue numba#5275, this commit changes the call to omp_set_max_active_levels(). Ideally we'd set it to the maximum supported nesting level, but we can't determine what that is; the function `omp_get_supported_active_levels()` was only added in OpenMP 5.0, which is quite recent and was not supported by some implementations until very recently (e.g. it was added to GNU OpenMP in mid-2020). We arbitrarily set the nesting level to 32, which is far greater than any real requirement, and the implementation will limit the nesting levels to the supported number as per the spec if it is less than 32.
In case it matters, I found that I was able to suppress the warnings with:
Note that you must set
|
@seanlaw, |
Explicitly sets the threading layer for Numba to a simple implementation that should always be available. UMAP uses Numba internally and, on some systems, the Numba threading layer can get set to an OpenMP (OMP) backend. Since Numba internally calls OMP with a function that is deprecated, OMP emits a deprecation warning. This warning is especially useless since we force UMAP to run with a single CPU. The main effect of this warning was to break our functional tests. For more details, see this Numba issue comment on GitHub: numba/numba#5275 (comment)
Explicitly sets the threading layer for Numba to a simple implementation that should always be available. UMAP uses Numba internally and, on some systems, the Numba threading layer can get set to an OpenMP (OMP) backend. Since Numba internally calls OMP with a function that is deprecated, OMP emits a deprecation warning. This warning is especially useless since we force UMAP to run with a single CPU. The main effect of this warning was to break our functional tests. For more details, see this Numba issue comment on GitHub: numba/numba#5275 (comment)
Reporting a bug
the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
to write one see http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports).
This
run like:
does this:
Think this is because OpenMP 5.0 deprecated
omp_set_nested
and Anaconda MKL 2020 packages depends on in Intel-OpenMP which is 5.0 compliant.The text was updated successfully, but these errors were encountered: