Set up end to end Send/Receive testing with LexBox #342
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Will be used in end-to-end testing scenarios
This will enable setting projects to specific commits before the test.
This test can now exercise the "reset project to known revision" functionality in LexBox, proving that it works.
Give it a project code, get an FwProject back, it does the rest. (It's the caller's job to dispose of the FwProject when you're done, though).
Also add test demonstrating that the helper works, and we can modify an entry's citation form, then push the modified entry to LexBox.
Now test code doesn't have to deal with undo/redo accessors.
So far the test is failing, so it's not working yet.
Now, if you tag a test with `[Property("projectCode", "sena-3")]` it
will automatically restore the sena-3 project to its original state at
the end of the test. This will ensure idempotence even if tests fail.
It's causing overload ambiguity when you create an SRTestEnvironment with no parameters. We'll just pass the port number in as a string.
Now we have one folder per test, and it's deleted afterwards if the test passed. If the test failed, the folder is preserved, but it will be deleted the next time the test runs, so investigate before re-running tests if you need the files in the folder.
Now that temp folders are set up properly, we can run tests that clone, commit, and push repeatedly: the LexBox project is automatically returned to its original state at the end of the test, and the temp folders are cleaned up prior to the next test being run.
Turns out the .fwdata file isn't updated until you call .Commit() on the action handler.
Also renamed CommitChanges to CommitAndPush.
Also make it static so that tests can supply their own dependencies without needing to create an instance of ConvertMongoToLcmComments or ConvertLcmToMongoComments in order to use the method.
This causes a .ChorusNotes file to show up in the sena-3 project in LexBox, thereby proving that it can be done. Later we'll turn this into a real round-trip test of some kind.
LcmTestHelper was the wrong place to put CommitAndPush since that needs to deal with URLs, and URLs are already being dealt with in SRTestEnv.
This will enable other code to just ask the test environment what the lexbox username and password are, and not have to look it up from the environment variables all the time.
If you run multiple tests that want to reset the same project, the second one was erroring out because the zip file already existed. We now use random names for the files and dirs in the reset-project method so that this cannot occur.
Plus a basic test demonstrating creating a new project, which will eventually move to the base class or the SRTestEnvironment class.
Run `dotnet test --filter E2E_` to see `hg clone` return 404 on a newly-created project. Won't happen every single time, but will happen often enough that it's likely to cause problems. Solution is going to be to make CloneRepoFromLexbox more resilient against temporary 404s.
|
Okay, commit 779358e demonstrates the race condition with hgweb's dir cache in local LexBox, and commit 80d45bb adjusts these tests to expect newly-created projects to sometimes not be available right away, avoiding the race condition. To test a potential fix in LexBox, use commit 779358e and run |
This was referenced Aug 19, 2024
hahn-kev
requested changes
Aug 20, 2024
There was a problem hiding this comment.
This looks like a great start. I think there's still a bit of work to do before we can merge it in. Inparticualr I'd like to make a bunch of the static code to be non static instead. Also it's not clear to me why we have both SRTest base and e2eTestBase and why one inherits from the other. I'd like to either merge those into one class, or just make one encapsulate the other.
As requested in code review
This ensures that the override will only affect the test it's set in, rather than potentially affecting other tests if we forget to clear the environment variable in a `finally` block.
Rather than assume that the test will be running against the sena-3 project, let's make the test grab an arbitrary entry (the first one that the LexEntryRepository knows about) and use it. Since the entry whose GUID we hardcoded was actually the first LexEntry sorted by GUID, this currently has no effect as we're actually testing the same entry we were before — but it will allow us to change the test later.
Since we're not going to be creating any other test fixtures derived from SRTestBase, there's no point in having two base classes; it has become over-engineering now that we've decided on just one E2E test.
Instead of the tests failing if LexBox isn't available, this will just mark them as ignored.
hahn-kev
requested changes
Aug 21, 2024
Newer library allows passing in an HttpClient, so we don't have to copy cookies around.
We keep the temp directory around if the test failed, but we dispose everything else.
hahn-kev
requested changes
Aug 26, 2024
The library doesn't do this for us. This would be unsafe if we weren't awaiting the result of TusPatchAsync, but by the time TusPatchAsync returns, the FileStream is no longer needed.
hahn-kev
approved these changes
Aug 26, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #339.
This PR uses the LexBox API to set up end-to-end testing of Send/Receive scenarios.
Work completed:
Future plans:
In a new PR, I will port most of the SynchronizeActionTests tests and convert them to real end-to-end tests. Then I'll add some more tests testing things like round-tripping of comments, various merge conflict scenarios, and so on.