Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stabilized the artifact ordering during the compilation #1097

Conversation

Ark-kun
Copy link
Contributor

@Ark-kun Ark-kun commented Apr 6, 2019

This change makes the order of output artifacts stable in Python 3.5.
The sample test seems to be relying on the artifact ordering (it tries to read the first artifact without looking at the name).
With this change the order is always the preserved and stable.

This change is Reviewable

@Ark-kun
Copy link
Contributor Author

Ark-kun commented Apr 6, 2019

/test kubeflow-pipeline-sample-test

@gaoning777
Copy link
Contributor

Could you explain a bit why we need this change?

@Ark-kun
Copy link
Contributor Author

Ark-kun commented Apr 8, 2019

Could you explain a bit why we need this change?

As the title indicates, this makes the order of output artifacts stable in Python 3.5.
The sample test seems to be relying on the artifact ordering (it tries to read the first artifact without looking at the name).
With this change the order is always the preserved and stable.

@gaoning777
Copy link
Contributor

The sample tests do not rely on the strict ordering of the artifacts but instead searches for the artifact.

for artifact in node['outputs']['artifacts']:

@Ark-kun
Copy link
Contributor Author

Ark-kun commented Apr 8, 2019

The sample tests do not rely on the strict ordering of the artifacts but instead searches for the artifact.

They did until I fixed them on Friday night:

s3_data = workflow_json['status']['nodes'][key]['outputs']['artifacts'][0]['s3']

Also the compiler tests do rely on the ordering.

Generally it's a good idea to have stable execution where multiple program runs with the same arguments produce same results.

@gaoning777
Copy link
Contributor

I see. Thanks for the explanation.

@gaoning777
Copy link
Contributor

/lgtm

@Ark-kun
Copy link
Contributor Author

Ark-kun commented Apr 9, 2019

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ark-kun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ark-kun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 0a08617 into kubeflow:master Apr 9, 2019
@Ark-kun Ark-kun deleted the Stabilized-the-artifact-ordering-during-compilation branch April 9, 2019 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants