Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_concurrent_cwt failure on macOS arm64 with free-threaded CPython #758

Closed
rgommers opened this issue Jun 27, 2024 · 6 comments
Closed

test_concurrent_cwt failure on macOS arm64 with free-threaded CPython #758

rgommers opened this issue Jun 27, 2024 · 6 comments
Labels
bug

Comments

@rgommers
Copy link
Member

We have a set of concurrent tests that use concurrent.futures.ThreadPoolExecutor. It has never failed before AFAIK; the tests also passed a number of times already with the new Linux free-threaded CI. It just now failed for the first time in the macOS arm64 free-threaded wheel build job (from the CI log):

_____________________________ test_concurrent_cwt ______________________________
  
      @uses_futures
      def test_concurrent_cwt():
          atol = rtol = 1e-14
          time, sst = pywt.data.nino()
          dt = time[1]-time[0]
          transform = partial(pywt.cwt, scales=np.arange(1, 4), wavelet='cmor1.5-1',
                              sampling_period=dt)
          for _ in range(10):
              arrs = [sst.copy() for _ in range(50)]
              with futures.ThreadPoolExecutor(max_workers=max_workers) as ex:
  >               results = list(ex.map(transform, arrs))

...

   ../venv-test-arm64/lib/python3.13/site-packages/pywt/tests/test_concurrent.py:101: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  /Library/Frameworks/PythonT.framework/Versions/3.13/lib/python3.13/concurrent/futures/_base.py:619: in result_iterator
      yield _result_or_cancel(fs.pop())
  /Library/Frameworks/PythonT.framework/Versions/3.13/lib/python3.13/concurrent/futures/_base.py:317: in _result_or_cancel
      return fut.result(timeout)
  /Library/Frameworks/PythonT.framework/Versions/3.13/lib/python3.13/concurrent/futures/_base.py:456: in result
      return self.__get_result()
  /Library/Frameworks/PythonT.framework/Versions/3.13/lib/python3.13/concurrent/futures/_base.py:401: in __get_result
      raise self._exception
  /Library/Frameworks/PythonT.framework/Versions/3.13/lib/python3.13/concurrent/futures/thread.py:58: in run
      result = self.fn(*self.args, **self.kwargs)
  ../venv-test-arm64/lib/python3.13/site-packages/pywt/_cwt.py:202: in cwt
      frequencies = scale2frequency(wavelet, scales, precision)
  ../venv-test-arm64/lib/python3.13/site-packages/pywt/_functions.py:183: in scale2frequency
      return central_frequency(wavelet, precision=precision) / scale
...
          index = np.argmax(abs(fft(psi)[1:])) + 2
  >       if index > len(psi) / 2:
  E       TypeError: A loop/promoter has already been registered with 'greater' for (<class 'numpy.dtypes.Float64DType'>, <class 'numpy.dtypes.Float64DType'>, <class 'numpy.dtypes.BoolDType'>)
  
  ../venv-test-arm64/lib/python3.13/site-packages/pywt/_functions.py:156: TypeError
  =========================== short test summary info ============================
  FAILED tests/test_concurrent.py::test_concurrent_cwt - TypeError: A loop/promoter has already been registered with 'greater' for (<class 'numpy.dtypes.Float64DType'>, <class 'numpy.dtypes.Float64DType'>, <class 'numpy.dtypes.BoolDType'>)
  ================ 1 failed, 812 passed, 225 deselected in 4.72s =================

That looks like a missing lock around a cache in NumPy, which is probably already fixed by one of @ngoldbaum's PRs.
@rgommers rgommers added the bug label Jun 27, 2024
@rgommers
Copy link
Member Author

Some of the NumPy wheels at https://anaconda.org/scientific-python-nightly-wheels/numpy/files are 11 days old, which is due to gh-26810. So if the fix landed already in numpy's main branch, it may still not have propagated.

@ngoldbaum
Copy link

ngoldbaum commented Jun 28, 2024 •
edited
Loading

I'm able to reproduce random failures and seg faults on this test using the wheel. I think rebuilding it will make this test more stable.

That said, I do see a failure running a different test in the same file on the latest numpy main:

================================= test session starts ==================================
platform darwin -- Python 3.13.0b2, pytest-8.2.2, pluggy-1.5.0 -- /Users/goldbaum/.pyenv/versions/3.13.0b2-nogil/bin/python
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/Users/goldbaum/Documents/pywt/.hypothesis/examples'))
rootdir: /Users/goldbaum/Documents/pywt
configfile: pytest.ini
plugins: hypothesis-6.103.1
collected 4 items

pywt/tests/test_concurrent.py::test_concurrent_swt PASSED                        [ 25%]
pywt/tests/test_concurrent.py::test_concurrent_wavedec PASSED                    [ 50%]
pywt/tests/test_concurrent.py::test_concurrent_dwt FAILED                        [ 75%]
pywt/tests/test_concurrent.py::test_concurrent_cwt PASSED                        [100%]

======================================= FAILURES =======================================
_________________________________ test_concurrent_dwt __________________________________

    @uses_futures
    def test_concurrent_dwt():
        # dwt on 1D data calls the Cython dwt_single
        # other cases call dwt_axis
        for dwt_func, x in zip([pywt.dwt, pywt.dwt2, pywt.dwtn],
                               [np.ones(8), np.eye(16), np.eye(16)]):
            transform = partial(dwt_func, wavelet='haar')
            for _ in range(10):
                arrs = [x.copy() for _ in range(100)]
                with futures.ThreadPoolExecutor(max_workers=max_workers) as ex:
>                   results = list(ex.map(transform, arrs))

_          = 1
arrs       = [array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.]),
 array([1., 1., 1., 1., 1., 1., 1., 1.])]
dwt_func   = <function dwt at 0x266bfcc05a0>
ex         = <concurrent.futures.thread.ThreadPoolExecutor object at 0x266bfb95f10>
results    = [(array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.])),
 (array([1.41421356, 1.41421356, 1.41421356, 1.41421356]),
  array([0., 0., 0., 0.]))]
transform  = functools.partial(<function dwt at 0x266bfcc05a0>, wavelet='haar')
x          = array([1., 1., 1., 1., 1., 1., 1., 1.])

pywt/tests/test_concurrent.py:84:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../.pyenv/versions/3.13.0b2-nogil/lib/python3.13/concurrent/futures/_base.py:608: in map
    fs = [self.submit(fn, *args) for args in zip(*iterables)]
        chunksize  = 1
        fn         = functools.partial(<function dwt at 0x266bfcc05a0>, wavelet='haar')
        iterables  = ([array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.]),
  array([1., 1., 1., 1., 1., 1., 1., 1.])],)
        self       = <concurrent.futures.thread.ThreadPoolExecutor object at 0x266bfb95f10>
        timeout    = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <concurrent.futures.thread.ThreadPoolExecutor object at 0x266bfb95f10>
fn = functools.partial(<function dwt at 0x266bfcc05a0>, wavelet='haar')
args = (array([1., 1., 1., 1., 1., 1., 1., 1.]),), kwargs = {}

    def submit(self, fn, /, *args, **kwargs):
>       with self._shutdown_lock, _global_shutdown_lock:
E       TypeError: descriptor '__enter__' for '_thread.RLock' objects doesn't apply to a '_thread.lock' object

args       = (array([1., 1., 1., 1., 1., 1., 1., 1.]),)
fn         = functools.partial(<function dwt at 0x266bfcc05a0>, wavelet='haar')
kwargs     = {}
self       = <concurrent.futures.thread.ThreadPoolExecutor object at 0x266bfb95f10>

../../.pyenv/versions/3.13.0b2-nogil/lib/python3.13/concurrent/futures/thread.py:165: TypeError
=============================== short test summary info ================================
FAILED pywt/tests/test_concurrent.py::test_concurrent_dwt - TypeError: descriptor '__enter__' for '_thread.RLock' objects doesn't apply to a '_thread.lock' object
============================= 1 failed, 3 passed in 0.39s ===================

No idea why this fails randomly (a few percent of the time on my M3 Mac) in this way.

@rgommers
Copy link
Member Author

Thanks for checking @ngoldbaum.

That second failure looks like a problem in either CPython or Cython:

self = <concurrent.futures.thread.ThreadPoolExecutor object at 0x266bfb95f10>
fn = functools.partial(<function dwt at 0x266bfcc05a0>, wavelet='haar')
args = (array([1., 1., 1., 1., 1., 1., 1., 1.]),), kwargs = {}

    def submit(self, fn, /, *args, **kwargs):
>       with self._shutdown_lock, _global_shutdown_lock:
E       TypeError: descriptor '__enter__' for '_thread.RLock' objects doesn't apply to a '_thread.lock' object

args       = (array([1., 1., 1., 1., 1., 1., 1., 1.]),)
fn         = functools.partial(<function dwt at 0x266bfcc05a0>, wavelet='haar')
kwargs     = {}
self       = <concurrent.futures.thread.ThreadPoolExecutor object at 0x266bfb95f10>

../../.pyenv/versions/3.13.0b2-nogil/lib/python3.13/concurrent/futures/thread.py:165: TypeError
=============================== short test summary info ================================
FAILED pywt/tests/test_concurrent.py::test_concurrent_dwt - TypeError: descriptor '__enter__' for '_thread.RLock' objects doesn't apply to a '_thread.lock' object
============================= 1 failed, 3 passed in 0.39s ===================

I can't find any existing bug report for this; _thread.RLock does have a ton of bug reports related to it not being pickle-able, but nothing is doing that in this test case.

@ambv ambv mentioned this issue Jul 4, 2024
Closed
@ngoldbaum
Copy link

I had a chat with @ambv over discord and he managed to find the upstream problem: python/cpython#121368. Closing in favor of that since this is unrelated to pywavelets.

@ngoldbaum
Copy link

Except I don't have triage rights in this repo, so maybe @rgommers can do that.

@rgommers
Copy link
Member Author

rgommers commented Jul 4, 2024

That's great, thanks @ngoldbaum and @ambv!

@rgommers rgommers closed this as completed Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rgommers @ngoldbaum