Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when creating the experiment with artifact_location - bucket #25

Open
Barteus opened this issue Mar 1, 2022 · 4 comments
Open
Assignees
Labels
23.10 Should be fixed by 23.10 bug Something isn't working

Comments

@Barteus
Copy link
Contributor

Barteus commented Mar 1, 2022

When creating the experiment using Python mlflow library and selecting artifact_location to the bucket (not a folder in the bucket) there is an error response of too many 500s.
Code:

experiment_id = mlflow.create_experiment(name="Wine Experiments", artifact_location="s3://my-bucket")

Response:

MlflowException: API request to http://mlflow-server.kubeflow.svc.cluster.local:5000/api/2.0/mlflow/experiments/create failed with exception HTTPConnectionPool(host='mlflow-server.kubeflow.svc.cluster.local', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/create (Caused by ResponseError('too many 500 error responses'))

I could not find the root cause of it. This can be the upstream issue or some missing configuration or missing privileges.

@Barteus
Copy link
Contributor Author

Barteus commented Mar 1, 2022

It works perfectly fine without artifact_location only artifact_location is corrupted.

@DomFleischmann DomFleischmann added this to Needs Triage in MLOps Solution Issues Mar 7, 2022
@DomFleischmann DomFleischmann moved this from Needs Triage to Labeled in MLOps Solution Issues Mar 7, 2022
@ca-scribner ca-scribner moved this from Labeled to Needs Triage in MLOps Solution Issues Oct 13, 2022
@ca-scribner
Copy link
Contributor

Let's try to reproduce this with our current MLFlow charm

@ca-scribner
Copy link
Contributor

This might be a good thing to add to our integration tests, regardless of whether it is currently working

@NohaIhab NohaIhab added the bug Something isn't working label Oct 18, 2022
@NohaIhab NohaIhab moved this from Needs Triage to Labeled in MLOps Solution Issues Oct 20, 2022
@technologic27
Copy link

Hello, I am having a similar issue. Has it been resolved?

@DnPlas DnPlas added the 23.10 Should be fixed by 23.10 label May 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
23.10 Should be fixed by 23.10 bug Something isn't working
Projects
Development

No branches or pull requests

6 participants