Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Downloading crates from crates.io fails with HTTP 503 Application Error from herokucdn #6385

Closed
Jolg42 opened this issue Apr 25, 2023 · 5 comments
Closed

Downloading crates from crates.io fails with HTTP 503 Application Error from herokucdn #6385

Jolg42 opened this issue Apr 25, 2023 · 5 comments
Labels
A-infrastructure 📡

Comments

@Jolg42
Copy link

Jolg42 commented Apr 25, 2023 •
edited
Loading

Problem

Here is an example:

https://github.com/prisma/prisma-engines/actions/runs/4797771434/jobs/8535173263?pr=3917#step:4:1

Downloading crates ...
warning: spurious network error (2 tries remaining): failed to get successful HTTP response from `[https://crates.io/api/v1/crates/ahash/0.7.6/download`](https://crates.io/api/v1/crates/ahash/0.7.6/download%60), got 503
body:
<!DOCTYPE html>
	<html>
	  <head>
		<meta name="viewport" content="width=device-width, initial-scale=1">
		<meta charset="utf-8">
		<title>Application Error</title>
		<style media="screen">
		  html,body,iframe {
			margin: 0;
			padding: 0;
		  }
		  html,body {
			height: 100%;
			overflow: hidden;
		  }
		  iframe {
			width: 100%;
			height: 100%;
			border: 0;
		  }
		</style>
	  </head>
	  <body>
		<iframe src="//www.herokucdn.com/error-pages/application-error.html"></iframe>
	  </body>
	</html>

Is there anything we can do on our side? (besides retrying)

https://status.crates.io/ does not mention any outage.

Steps

  • cargo clean && cargo build

Possible Solution(s)

No response

Notes

No response

Version

No response
@Eh2406
Copy link
Contributor

Eh2406 commented Apr 25, 2023

@rust-lang/crates-io, 503 is a very odd error for this endpoint.

@Jolg42
Copy link
Author

Jolg42 commented Apr 25, 2023

We got many today 😅 in different pipelines (GitHub Actions and on our own infra running with Buildkite)
A retry seems to work for now.

Probably, someone might need to check the endpoint to know what is really happening?

@Jolg42 Jolg42 changed the title Downloading crates from crates.io fails with HTTP 503 from herokucdn Downloading crates from crates.io fails with HTTP 503 Application Error from herokucdn Apr 25, 2023
@weihanglo
Copy link
Member

Going to transfer this to rust-lang/crates-io now. If anyone thinks this is wrong we can transfer it back.

@weihanglo weihanglo transferred this issue from rust-lang/cargo Apr 25, 2023
@Turbo87
Copy link
Member

Turbo87 commented Apr 25, 2023

A retry seems to work for now.

my best guess is that this happened while we were deploying and Heroku was restarting the dynos. this is obviously not a good excuse though. the long-term plan is to move away from Heroku and use proper zero downtime deployments, but unfortunately we haven't had time for that yet.

@weihanglo weihanglo mentioned this issue Apr 26, 2023
Merged
@Turbo87
Copy link
Member

Turbo87 commented Feb 12, 2024

since this issue was opened we've removed nginx from our server, which (in combination with the Heroku restart behavior) was the root cause for the majority of these issues. I'm going to close this issue since the primary issue should be resolved now :)

@Turbo87 Turbo87 closed this as completed Feb 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-infrastructure 📡
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Turbo87 @Jolg42 @Eh2406 @weihanglo