-
Notifications
You must be signed in to change notification settings - Fork 468
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conformance Reporting to reflect v1.1.0 release #3088
Conformance Reporting to reflect v1.1.0 release #3088
Conversation
Hi @xtineskim. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
I agree that the dropdown for previous versions isn't ideal, but it's at least clear. Also, rotating this page as part of the release will encourage implementations to get their new conformance test results in quickly, since they won't show up until they do. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this PR. @xtineskim! I think we may need to add instructions on the RELEASE.md page to rotate this table whenever a new release is going to be created.
/ok-to-test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the quick update @xtineskim! This looks great!
RELEASE.md
Outdated
@@ -70,6 +70,7 @@ For a **MAJOR** or **MINOR** release: | |||
- Run the `make build-install-yaml` command which will generate install files in the `release/` directory. | |||
Attach these files to the GitHub release. | |||
- Update the `README.md` and `site-src/guides/index.md` files to point links and examples to the new release. | |||
- Update the `site-src/implementation-table.md` to `site-src/implementation-table-<oldVersion>.md`. Add the path to the `past_versions` variable in `hack/mkdocs-generate-conformance.py`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we probably want to add some kind of fixed and predictable timeline for this. We can't expect implementations to support the latest release on day 1, but maybe we switch over the default link 1 month after the release or after we've received 3 reports, whichever comes first?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the suggestion that we either do 1month max, or at least 3 reports, that seems solid. We can adjust from there if anyone has complaints.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, +1 on 1 month or 3 reports, whichever happens first.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good to me as well 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, @xtineskim!
/lgtm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @xtineskim! Some small nits about navigation structure, but otherwise LGTM.
hack/mkdocs-generate-conformance.py
Outdated
# gateway_tls_table = generate_profiles_report(reprots,'TLS') | ||
The latest release of Gateway API is shown here. Past Gateway API implementations and feature support can be found here: | ||
|
||
- [v1.0.0](./implementation-table-v1.0.0.md) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thinking about this more, maybe we should aim for the following in terms of file structure:
/site-src/implementations/{VERSION}.md
- Don't track patch versions, all reports for a minor version get lumped together
- In published site, we end up with https://gateway-api.sigs.k8s.io/implementations/v1.1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This approach would mean that we wouldn't publish any reports for a release until we met the min criteria (one month or three implementations), but I think it might be simplest to keep track of. We could also choose to limit this to something like the 3 most recent Gateway API versions so the list doesn't become too long.
mkdocs.yml
Outdated
- Comparison: implementation-table-v1.1.0.md | ||
- Past Version Comparisons: | ||
- v1.0.0: implementation-table-v1.0.0.md |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe something like this to line up with my comment from above
- Comparison: implementation-table-v1.1.0.md | |
- Past Version Comparisons: | |
- v1.0.0: implementation-table-v1.0.0.md | |
- Comparisons: | |
- v1.1: implementations/v1.1.md | |
- v1.0: implementations/v1.0.md |
Thanks @xtineskim! I think once #3137 merges, we'll have 4 reports for v1.1, so if you rebase this PR and regenerate the v1.1 table, we should have enough to merge this. |
3273b0e
to
171606a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should wait for #3168 to be merged, as the cilium report is flawed.
@mlavacca just saw the errors now, it seems like it's tripping since only Istio provides the list of extended Supported Features, whereas all the other implementations don't support them 😅 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @xtineskim!
/approve
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: robscott, xtineskim The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Update so GRPCRoute is listed before TLS Route Co-authored-by: Rob Scott <rob.scott87@gmail.com>
Thanks @xtineskim! /lgtm |
* Reflect v1.1.0, new naming format * Add line to RELEASE.md * Update release, generated implementation table path file names * Chanch file structure per feedback * Check if extended features are present * Run against implementations * Update hack/mkdocs-generate-conformance.py Update so GRPCRoute is listed before TLS Route Co-authored-by: Rob Scott <rob.scott87@gmail.com> --------- Co-authored-by: Rob Scott <rob.scott87@gmail.com>
* Reflect v1.1.0, new naming format * Add line to RELEASE.md * Update release, generated implementation table path file names * Chanch file structure per feedback * Check if extended features are present * Run against implementations * Update hack/mkdocs-generate-conformance.py Update so GRPCRoute is listed before TLS Route Co-authored-by: Rob Scott <rob.scott87@gmail.com> --------- Co-authored-by: Rob Scott <rob.scott87@gmail.com>
* Reflect v1.1.0, new naming format * Add line to RELEASE.md * Update release, generated implementation table path file names * Chanch file structure per feedback * Check if extended features are present * Run against implementations * Update hack/mkdocs-generate-conformance.py Update so GRPCRoute is listed before TLS Route Co-authored-by: Rob Scott <rob.scott87@gmail.com> --------- Co-authored-by: Rob Scott <rob.scott87@gmail.com>
What type of PR is this?
/kind documentation
/area conformance
What this PR does / why we need it:
Added a function to get the latest version of Gateway API, reflects the new naming format (
<GATEWAY|MESH>-<HTTP|TLS|GRPC>
), and a start on how to "archive" the past versions of Gateway APICurrently, for "archiving", each version will have it's own implementation-table-.md. When a new released is cut (tbd frequency by the maintainers), the Comparison link in the nav will point to the latest version. Past releases are then moved to an info drop down - also open to other suggestions if this isn't ideal.
Which issue(s) this PR fixes:
Some of follow up tasks related to #3050
Fixes #
Does this PR introduce a user-facing change?: