Test failed on Debian armel

[yuzibo]

Hi,

I have packaged the module for Debian, but there is one test case failed that stopping the package migration to testing from sid.

The specilly log is here

 56s 4.11.3 from /usr/lib/python3/dist-packages/tox/__init__.py
 56s registered plugins:
 56s     tox-current-env-0.0.11 at /usr/lib/python3/dist-packages/tox_current_env/hooks.py
 56s 
 56s 
 56s 
 56s 
 56s =================================== FAILURES ===================================
 56s __________ test_allenvs_print_extras_to_file[--print-extras-to-file] ___________
 56s [gw6] linux -- Python 3.11.6 /usr/bin/python3.11
 56s 
 56s tmp_path = PosixPath('/tmp/pytest-of-debci/pytest-0/popen-gw6/test_allenvs_print_extras_to_f0')
 56s option = '--print-extras-to-file'
 56s 
 56s     @pytest.mark.parametrize("option", ("--print-extras-to", "--print-extras-to-file"))
 56s     def test_allenvs_print_extras_to_file(tmp_path, option):
 56s         extraspath = tmp_path / "extras"
 56s         result = tox(option, str(extraspath))
 56s         assert sorted(extraspath.read_text().splitlines()) == sorted(
 56s             ["dev", "full"] * len(envs_from_tox_ini())
 56s         )
 56s         expected = ""
 56s         for env in envs_from_tox_ini()[:-1]:
 56s             expected += f"{env}: OK\n"
 56s         expected += tox_footer(spaces=0) + "\n"
 56s >       assert prep_tox_output(result.stdout) == expected
 56s E       AssertionError: assert 'py36: OK\npy...ulations :)\n' == 'py36: OK\npy...ulations :)\n'
 56s E           py36: OK
 56s E           py37: OK
 56s E           py38: OK
 56s E         + py310: OK
 56s E           py39: OK
 56s E         - py310: OK
 56s E             py36: OK...
 56s E         
 56s E         ...Full output truncated (6 lines hidden), use '-vv' to show
 56s 
 56s /tmp/autopkgtest-lxc.4a3lx05w/downtmp/autopkgtest_tmp/tests/test_integration_tox4.py:231: AssertionError
 56s ----------------------------- Captured stdout call -----------------------------
 56s current /tmp/pytest-of-debci/pytest-0/popen-gw6/test_allenvs_print_extras_to_f0/projdir running in /tmp/pytest-of-debci/pytest-0/popen-gw6/test_allenvs_print_extras_to_f0/projdir
 56s py36: OK ✔ in 0.01 seconds
 56s py37: OK ✔ in 0.02 seconds
 56s py38: OK ✔ in 0.01 seconds
 56s py310: OK ✔ in 0.01 seconds
 56s py39: OK ✔ in 0.01 seconds
 56s   py36: OK (0.01 seconds)
 56s   py37: OK (0.02 seconds)
 56s   py38: OK (0.00 seconds)
 56s   py39: OK (0.00 seconds)
 56s   py310: OK (0.00 seconds)
 56s   py311: OK (0.01 seconds)
 56s   congratulations :) (0.13 seconds)
 56s 
 56s ----------------------------- Captured stderr call -----------------------------
 56s 
 56s =========================== short test summary info ============================
 56s FAILED tests/test_integration_tox4.py::test_allenvs_print_extras_to_file[--print-extras-to-file]

The more test results you can refer to tracker
any help will be appreciated.

[hroncok]

Seems the tox output is sorted nonstandardly. Do you have steps to reproduce this?

[hroncok]

Moreover, does this happen all the time?

[yuzibo]

Not sure you have debian sid, but I set up an armel qemu-user chroot as below:

 sudo sbuild-createchroot --debootstrap=mmdebstrap --arch=armel        \
  --include=debian-ports-archive-keyring,ca-certificates       \
     --make-sbuild-tarball=/srv/sid-armel-sbuild.tgz         \
     sid /tmp/chroots/sid-armel-sbuild/      \
        https://mirror.iscas.ac.cn/debian

The last ${URL} you can replace you want.

Once done, then to edit key-value from profile=sbuild to profile=default of /etc/schroot/chroot.d/sid-armel-sbuild-xx.

At last, sudo schroot -c sid-armel-sbuild to chroot armel env.

But, as you said, it was to reproduce the bug, becasue I built it again but it works! What's more, it passed autopkgtest two times.

Unfortunately, it seems this is not false postive although it's hard to reproduce.

[hroncok]

Whoever knows why this happens can adjust the test I guess. I won't be able to dedicate my time for this myself.

[yuzibo]

Hi,

This issue was reproduced on s390x also, but maybe a little different. See:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1055279#12

I am thinking of skipping temporary the test to unblock the other package.