Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify dependencies to Linera binaries #1107

Merged
merged 7 commits into from
Oct 10, 2023
Merged

Conversation

ma2bd
Copy link
Contributor

@ma2bd ma2bd commented Oct 10, 2023

Motivation

We have three ways to obtain Linera binaries at the moment: release mode (look up in CARGO_HOME), debug mode (cargo build), test-runner (look up in CARGO_HOME + cargo install if needed).

This is rather complicated and create some subtle limitations:

  • We cannot run tests in release mode (it will fetch the binaries with cargo install)
  • We cannot rename the test runner because cargo install from the network would break CI.

Proposal

In the long term, cargo should eventually provide a way to depend on the binaries of another crate (rust-lang/cargo#9096). In the meantime, we only have partial solutions. At least we're going for a simple one here (removing lots of code such as the feature detection).

  1. Use a single resolve_binary function what all the cases.
  • In production, it will just look at a directory where the current binary is.
  • In the case of integration tests run from target/{debug,release}/deps, it will look at the directory above.
  • We suggest how to provide the missing binaries (with cargo build or cargo install) but do not do it ourselves any more.

There will be two cargo install documented for users (test runner and linera-service).

For tests, cargo test always build the binaries of the current crate(s). So, the only potential remaining issue is when a test depends on the binary of another crate. This is rare as most e2e tests are inside linera-service. In CI, we run cargo build anyway first.

  1. Add a version check based on the crate version (Linera callers and callees must have the same version).
  • CAVEAT: All Linera binaries must use StructOpt (or at least recognize --version in the same way)
  • Make sure to support --version in schema_export
  1. Details:
  • The first commit in the PR also fixes CI for linera-service-graphql-client.
  • This PR caught a few places where we still used version 0.4.0 instead of 0.4.1.

Test Plan

CI

Release Plan

  • Need to bump the major version number in the next release of the crates.
  • Need to update the developer manual. ==> need to cargo install linera-sdk explicitly

Links

stacked on top of #1106

@ma2bd ma2bd changed the title Cargo binaries Simplify dependencies to Linera binaries Oct 10, 2023
@ma2bd ma2bd marked this pull request as ready for review October 10, 2023 07:16
.await
.unwrap();
assert!(
util::resolve_binary("linera-spaceship", env!("CARGO_PKG_NAME"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤩 🚀

assert!(
util::resolve_binary("linera-spaceship", env!("CARGO_PKG_NAME"))
.await
.is_err()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😞

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol

linera-service/src/util.rs Outdated Show resolved Hide resolved
linera-service/src/util.rs Outdated Show resolved Hide resolved
{
features.push("scylladb");
let components = current_binary_path
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a little harder to read, consider: https://doc.rust-lang.org/std/path/struct.Path.html#method.ends_with

 if path.ends_with(Path::new("target/debug/deps")) || path.ends_with(Path::new("target/release/deps")) {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually Path::new is not even needed

.join(name)
.canonicalize()
.unwrap()
.map_err(|e| {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can just use with_context here (since formatting is expensive and you want to do it lazily). You don't need the map_err as the error will be shown anyway:

Error: Failed to execute and retrieve version from the binary {name} in directory {}

Caused by:
    {e}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with_context seems to only work if you have an anyhow::Result already.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation says the error type just has to be StdError + Send + Sync + 'static.

.trim()
.split(' ')
.last()
.ok_or_else(|| {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here I think?

Copy link
Contributor

@christos-h christos-h left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking, just a few suggestions. LGTM :)

@ma2bd ma2bd merged commit be19dbc into linera-io:main Oct 10, 2023
6 checks passed
@ma2bd ma2bd deleted the cargo_binaries branch October 10, 2023 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants