Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run all examples in the CI #839

Open
cvybhu opened this issue Oct 16, 2023 · 3 comments
Open

Run all examples in the CI #839

cvybhu opened this issue Oct 16, 2023 · 3 comments
Assignees
Labels
enhancement New feature or request symptom/ci stability Related to continuous integration

Comments

@cvybhu
Copy link
Contributor

cvybhu commented Oct 16, 2023

The CI checks that all of the code examples compile, but it doesn't check whether they actually work.
The code is constantly evolving, and things that used to work in the past might stop working at some point, without anyone noticing.

In #835 a user reported that one of the examples doesn't work. It fails because the specified quorum is invalid for this type of request. This should've been caught in the PR that introduced this example, but wasn't.

It would be good to run all of the examples in the CI. This will help us catch all breakages.

@cvybhu cvybhu changed the title Run examples in the CI Run all examples in the CI Oct 16, 2023
@cvybhu cvybhu added enhancement New feature or request symptom/ci stability Related to continuous integration labels Oct 16, 2023
@cvybhu
Copy link
Contributor Author

cvybhu commented Oct 16, 2023

I don't know any simple way to run all of the examples using a single command.
There's an issue about it: rust-lang/cargo#8356

We can just list every example as a separate step in the workflow, there aren't that many of them.

@piodul
Copy link
Collaborator

piodul commented Oct 17, 2023

Some of the examples cannot be just run like you would run a test - some of them require some CLI options (allocations.rs), user input (cqlsh-rs.rs) or specific environment (cloud.rs).

IMO those tests that don't need any user input should just be integration tests or maybe examples in docstrings (and cargo test runs those). Most of the existing examples could be treated in that way. Not sure about the others, but we could consider what to do with them on a per-case basis.

@nsipplswezey
Copy link
Contributor

Since I'm the user who reported an error in one of the examples, I figured I'd open a PR #845 taking a look at running examples in CI. After running the examples in CI, I agree with @piodul that the examples aren't all consistent enough to run like you would run a test. Along with the examples @piodul highlighted, I just opened this issue #844 that I think is caused by two examples that use the same table naming conventions that cause the other to error when run against the same scylladb cluster. Also the tls example is already in CI. And the cloud example might need a review of its config credentials.

I think a per-case basis certainly makes sense. Hope this little bit of work is helpful.

@Lorak-mmk Lorak-mmk self-assigned this Nov 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request symptom/ci stability Related to continuous integration
Projects
None yet
Development

No branches or pull requests

4 participants