Test

[h-vetinari]

For conda-forge/conda-smithy#2060 & scipy/scipy#21553

[conda-forge-webservices]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

[h-vetinari]

Raised an issue for 5eb8df4 / 36a062e: conda-forge/conda-smithy#2061

[h-vetinari]

I cannot tell exactly what's going wrong here @ilayn, but note that we make it to the very last of the

import tests

scipy-feedstock/recipe/meta.yaml

Lines 130 to 172 in 07e1f9f

	imports:
	- scipy
	# reference for public API is effectively
	# https://github.com/scipy/scipy/blob/main/scipy/_lib/tests/test_public_api.py
	- scipy.cluster
	- scipy.cluster.hierarchy
	- scipy.cluster.vq
	- scipy.constants
	- scipy.datasets
	- scipy.fft
	- scipy.fftpack
	- scipy.integrate
	- scipy.interpolate
	- scipy.io
	- scipy.io.arff
	- scipy.io.matlab
	- scipy.io.wavfile
	- scipy.linalg
	- scipy.linalg.blas
	- scipy.linalg.cython_blas
	- scipy.linalg.cython_lapack
	- scipy.linalg.interpolative
	- scipy.linalg.lapack
	- scipy.misc
	- scipy.ndimage
	- scipy.odr
	- scipy.optimize
	- scipy.signal
	- scipy.signal.windows
	- scipy.sparse
	- scipy.sparse.csgraph
	- scipy.sparse.linalg
	- scipy.spatial
	- scipy.spatial.distance
	- scipy.spatial.transform
	- scipy.special
	- scipy.stats
	- scipy.stats.contingency
	- scipy.stats.distributions
	- scipy.stats.mstats
	- scipy.stats.qmc
	- scipy.stats.sampling
	commands:

In fact, we even do so twice (well, conda-build does a weird log duplication for the import tests; not 100% sure it's actually run twice). In any case, it seems like something is leaving an error code (or some other dirtied resource) behind so that conda-build sees some kind of failure at the end of the import tests. It's clear as mud to me what might be the cause.

[ilayn]

Whoa, that's really cryptic stuff. Regardless of the issue hats off to you for handling this chaos. I can only see one tiny sliver of hope that the failure in the azure pipelines stacktrace ends with

conda_build.exceptions.CondaBuildUserError: TESTS FAILED: scipy-1.14.1-py313hdf33807_0.conda

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=1028042&view=logs&j=00f5923e-fdef-5026-5091-0d5a0b3d5a2c&t=f42c1d49-d161-5453-1c0e-990f827a8536&l=10989

Also the scipy version a bit above in the package list says SciPy version 1.14.1 and local. So maybe a cache needs to be invalidated?

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=1028042&view=logs&j=00f5923e-fdef-5026-5091-0d5a0b3d5a2c&t=f42c1d49-d161-5453-1c0e-990f827a8536&l=10859

If it's not that, I'm with you with the snorkel view about this.

[h-vetinari]

Unfortunately you're fishing for red herrings here. 😅

That's a regular message for failing tests, and the local is important because it means we're testing the just-built scipy (rather than an already-published one). Finally, the version is reported as old because I haven't updated the conda-side metadata in this PR.

[h-vetinari]

But it is indeed unusual to fail already at the import stage without any other info. During the pytest stage we normally get better errors. I guess that's also an option here - disable the import tests and see what happens...

[ilayn]

I think this is not scipy related but something else is broken in the toolchain. Looks like the .bat file is returning an error.

[ilayn]

But nevermind, I already saw that it is building so that's good enough for me. I don't want to cause more work for you.

[h-vetinari]

I think this is not scipy related but something else is broken in the toolchain. Looks like the .bat file is returning an error.

Unfortunately, it's 100% guaranteed to be related to the changes that happened between 1.14 and your PR. If you want I can test SciPy main and see if it works as well, but you see a CI run further up where 1.14 passed CI today.

The fact that you see the failure in the bat script is just because that's the wrapper that executes the test suite. Unfortunately, it appears that the same kind of corruption also happens when we invoke pytest (probably during test collection, which already imports most of SciPy).

It's a frustrating kind of error to run into and to debug, but it's very real.

[ilayn]

Hmm then I have no idea what it might be and more perplexing that all passing in the SciPy CI. Frustrating indeed

[h-vetinari]

Good news (for your PR at least). SciPy main also fails in the same way...

[ilayn]

I commented there but here it is less noise I think, but the MSVC job of scipy is going through but that is using ifx. Is it a Flang problem possibility ?

[h-vetinari]

Is it a Flang problem possibility ?

Always a possibility, but not my first suspicion. Could also be a VS2019 vs. 2022 issue. We'll see what the bisection yields

[ilayn]

Do you mind adding the very last commit from scipy/scipy#21553 just to test if it will fail again? Then I would know how to recreate the failing test locally.

[h-vetinari]

CI retriggered, which should pick up the newest commits from that branch.

[ilayn]

Thank you, fingers crossed.

[ilayn]

Apologies Axel, I'm losing focus these days. I missed a few places that randomization was done in the old style. Could you retrigger this if you can please?

[h-vetinari]

I mentioned this in the SciPy PR already, but just to allow you to move at your own speed: you can retrigger CI here yourself with the following magic incantation (and because the sources point to your branch, that will pick up whatever new commits are in there):

@conda-forge-admin, please restart ci

[ilayn]

@conda-forge-admin, please restart ci

[ilayn]

@conda-forge-admin, please restart ci

[h-vetinari]

Congrats, bug squashed successfully! 😎

[ilayn]

For the life of me, I don't know why this made a difference but I'll take it. Thank you for the support Axel, I hope this will make your life easier down the line a bit when it hits the road.