ON-15724: improve unit test consistency #20
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Looks like zfstackdump is perhaps a github setup issue, so I'll look into what causes that. |
tcrawley-xilinx
approved these changes
May 31, 2024
In zfbonding_lacp, the interfaces known to zf emu are sometimes reset by zeroing a large block of memory. This would also zero the cplane mibs dimensions, including hwport_max. As a result, subsequent tests would fail as no hardware ports existed.
Adding the testjenkins target meant that some tests were not given the correct environments, and would therefore always fail. In this change, zfstackdump is moved to its original test group as it was not observed to fail. The remaining test, zftimestamping, came from the testloop target, so the testjenkins target now takes the environment from testloop to run this test.
Previously, all test scripts returned 0 regardless of the results as many executables were run per script. This change exits with the last failure code (or 0), and also makes sure the test target is provided as an environment variable to be logged.
The pretest.sh script was supposed to be run before each test, but in reality would only run once before kicking off a bunch of tests. For example, `make test` would only run this script at the very start and only a single time. By merging pretest.sh into netns.sh, we make sure that the commands in the pretest.sh script are run before every test target, and before every single test.
The test scripts invoke many applications that may each create a new zf_emu state. Many of the back-to-back tests have had issues with this state persisting and interfering with the tests, causing time outs or other uncertain behaviour. This change deletes the shared memory before and after every test to make sure no errors occur as a result of persistent memory from previous tests.
Previously, we would redirect fd 1000 into egrep to get the output of our stackdump, however, this didn't work on newer Ubuntu's and perhaps others. Instead, we now rely on sysfs to correctly link to our opened fds. As supplementary changes, this now uses cat so the output is kept for the sake of test logs, and grep -E to replace egrep as egrep is becoming deprecated on some newer OSes.
|
Latest push rebases to also include Andy's fix for timestamping and also fixes the issue with |
tcrawley-xilinx
approved these changes
May 31, 2024
We've seen things like this in the past where ubu defaulted to dash not bash. I haven't looked at the context, but that sort of this might be the case here. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These changes improve the consistency of the unit tests, mainly those under the
test_jenkinstarget.The only test that still fails after this patchset is zftimestamping, but this appears to be fixed by #15. For now, I will keep these tests as part of the
test_jenkinstarget until I'm more satisfied that these fixes work after getting more test exposure over time.Testing Done
Repeated lots of iterations of
make testandmake test_jenkinsto observe behaviour over time. Previously, the failing testscripts would fail every 10 or so runs ofmake test_jenkinsbut I have not observed a single failure in quite a lot of runs. This is a very hand-wavy testing done section, which is why I would like to observe these changes over time before committing to saying that these are fine.