Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cross-compilation for shipping service #715

Merged

Conversation

svrnm
Copy link
Member

@svrnm svrnm commented Jan 24, 2023

Changes

Related to #709, this enables cross-compilation (arm64 target on amd64 build) for the github build workflow. It brings down the build time for the shipping service on both platforms to ~6min (from 1h14min). There are 2 disadvantages two that:

  • The Dockerfile becomes complicated (there is probably a way to optimize that, but look at that mess...)
  • The final image is not based on alpine anymore but debian slim, because cross-compiling with musl (and without glibc) didn't work for a variety of reasons, so the final image is bigger (97.8MB)

I raise this as a draft PR because this only becomes valuable when other long running builds are reduced accordingly, so merging it is not yet reasonable. The purpose of the PR right now is to discuss if the disadvantages (especially the first one) are worth the effort (assuming something similar can be done for the Quote, Payment & Currency-Services)

Signed-off-by: svrnm <neumanns@cisco.com>
@cartersocha cartersocha added the v1.4 required for 1.4 release label Jan 30, 2023
@svrnm svrnm marked this pull request as ready for review February 6, 2023 18:24
@svrnm svrnm requested a review from a team February 6, 2023 18:24
@svrnm
Copy link
Member Author

svrnm commented Feb 6, 2023

taking this out from Draft state for now. As a note: there is an alternative ways on how to structure this: Conditional stages using build arguments.

By introducing a build argument (e.g. BUILDER_TYPE) one can have 2 different stages for native & cross compilation, looks like this:

ARG BUILDER_TYPE=native
FROM rust:1.61 as builder-native
# native compilation
...
FROM --platform=${BUILDPLATFORM} rust:1.61 as builder-aarch64_on_x86
# cross compile
...
FROM builder-${BUILDER_TYPE} as builder
...
FROM debian:bullseye-slim as release
COPY --from=builder /app/target/release/shippingservice /app/shippingservice
...

The advantage is a clearer set of docker instructions per stage
The build argument needs to be set in the github workflow

Which one do people prefer?

@cartersocha
Copy link
Contributor

131s vs. 105s for me.

When you do a full demo build the docker log goes a bit nuts but I think that's intended with the cross compilation

@austinlparker
Copy link
Member

taking this out from Draft state for now. As a note: there is an alternative ways on how to structure this: Conditional stages using build arguments.

By introducing a build argument (e.g. BUILDER_TYPE) one can have 2 different stages for native & cross compilation, looks like this:

ARG BUILDER_TYPE=native
FROM rust:1.61 as builder-native
# native compilation
...
FROM --platform=${BUILDPLATFORM} rust:1.61 as builder-aarch64_on_x86
# cross compile
...
FROM builder-${BUILDER_TYPE} as builder
...
FROM debian:bullseye-slim as release
COPY --from=builder /app/target/release/shippingservice /app/shippingservice
...

The advantage is a clearer set of docker instructions per stage The build argument needs to be set in the github workflow

Which one do people prefer?

Honestly, I don't feel like the Dockerfile is that confusing as-is; I'd lean towards accepting it as stands in order to avoid adding a special case for the build scripts.

Copy link
Member

@julianocosta89 julianocosta89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my case it went from 990s to 255s (WSL - Ubuntu 20.04)
Great improvement!

@austinlparker austinlparker merged commit 55912a2 into open-telemetry:main Feb 9, 2023
mat-rumian added a commit to SumoLogic/opentelemetry-demo that referenced this pull request Feb 20, 2023
* Fixes open-telemetry#696, Downgrades OTP (open-telemetry#698)

* Use different base images for frauddetectionservice (open-telemetry#691)

* Use different docker base images for frauddetection service

Signed-off-by: svrnm <neumanns@cisco.com>

* Add entry in CHANGELOG

Signed-off-by: svrnm <neumanns@cisco.com>

* update changelog

Signed-off-by: svrnm <neumanns@cisco.com>

* Remove extra blank line

Signed-off-by: svrnm <neumanns@cisco.com>
Co-authored-by: Juliano Costa <julianocosta89@outlook.com>
Co-authored-by: Carter Socha <43380952+cartersocha@users.noreply.github.com>

* fix(paymentservice): respect temporality env var (open-telemetry#693)

* fix(paymentservice): update dependencies so that temporality env var is respected

* fix(changelog): add changelog entry

* Add healthcheck to featureflagservice (open-telemetry#661)

* Add healthcheck to featureflagservice

In order to assess the health of this central service, compose should perform a check on the service.

* Add changelog entry

Co-authored-by: Juliano Costa <julianocosta89@outlook.com>
Co-authored-by: Carter Socha <43380952+cartersocha@users.noreply.github.com>

* bot is broken (open-telemetry#699)

Co-authored-by: Juliano Costa <julianocosta89@outlook.com>

* Fix link to the docs (open-telemetry#701)

* reduce quoteservice spans (open-telemetry#702)

* fixing container name nit - quote service (open-telemetry#706)

* [recommendationservice] - base image, licensing and diagram update (open-telemetry#700)

* update service metric requirement table (open-telemetry#707)

* update service metric requirement table

* add feature table

* Update emailservice Dockerfile to use alpine and multistage build (open-telemetry#703)

* Update emailservice Dockerfile

Signed-off-by: svrnm <neumanns@cisco.com>

* Use alpine

Signed-off-by: svrnm <neumanns@cisco.com>

* Add changelog

Signed-off-by: svrnm <neumanns@cisco.com>

* newline

Signed-off-by: svrnm <neumanns@cisco.com>

* Update CHANGELOG.md

* Update .dockerignore

Signed-off-by: svrnm <neumanns@cisco.com>
Co-authored-by: Juliano Costa <julianocosta89@outlook.com>

* Update dockerfile for adservice to use different base images (open-telemetry#705)

* Enable exemplar export/display support (open-telemetry#704)

* Enable exemplar export/display support

Ensure the collector's prometheus endpoint produces exemplars
Ensure that prometheus collects and stores exemplar data
Ensure that grafana will display links to exemplars from dashboards/the explore tab

* Enable exemplars for latency histogram in demo dashboard

* update contributing guide (open-telemetry#711)

* update contributing guide

* fix lint

Co-authored-by: Carter Socha <43380952+cartersocha@users.noreply.github.com>

* Add random errors in AdService (open-telemetry#694)

* feature(frontend): adding instrumentation metrics (open-telemetry#632)

* feature(frontend): adding instrumentation metrics

* feature(frontend): adding instrumentation metrics

* updating package-lock.json

* Update docker-compose.yml

Co-authored-by: Marc Pichler <marcpi@edu.aau.at>

---------

Co-authored-by: Marc Pichler <marcpi@edu.aau.at>

* Update CONTRIBUTING.md (open-telemetry#720)

* fix metrics import (open-telemetry#721)

Signed-off-by: Pierre Tessier <pierre@pierretessier.com>

* Update .env (open-telemetry#722)

* Updated demo link for Sumo Logic (open-telemetry#718)

* Add build for arm64 arch (open-telemetry#657)

* Add build for arm64 arch

* Add change to changelog

* enable build on PR

* remove build on pr

---------

Co-authored-by: Juliano Costa <julianocosta89@outlook.com>
Co-authored-by: Carter Socha <43380952+cartersocha@users.noreply.github.com>
Co-authored-by: Austin Parker <austin@ap2.io>

* [chore] - fix image version (open-telemetry#726)

Remove the `v` prefix from the image version. We now publish images without this prefix.

* Adds caveats to current exemplar support in the demo architecture (open-telemetry#725)

Also, updates collector version: exemplars for the latency histogram produced by the span  metrics processor are now getting exported for all services and endpoints, not just the first one in the metric family.

Co-authored-by: Pierre Tessier <pierre@pierretessier.com>

* Add Sentry (sentry.io) demo to README (open-telemetry#724)

* Add Sentry (sentry.io) demo to README

* use alpha order

---------

Co-authored-by: Pierre Tessier <pierre@pierretessier.com>

* Drop docs folder and adjust links (open-telemetry#730)

* Drop docs folder and adjust links

* Update PULL_REQUEST_TEMPLATE.md

* Add entry to CHANGELOG.md

* README: drop unused logo reference

* add reflection for grpcurl (open-telemetry#735)

* Add cross-compilation for shipping service (open-telemetry#715)

* Add cross-compilation for shipping service

Signed-off-by: svrnm <neumanns@cisco.com>

* Update CHANGELOG.md

* Update Dockerfile

---------

Signed-off-by: svrnm <neumanns@cisco.com>

* Ensure all http.server.duration metrics have the same description (open-telemetry#738)

* Add OTel logo to README (open-telemetry#744)

* rename proto package from hipstershop to oteldemo (open-telemetry#740)

* prometheus: honor_labels to have the right job and instance (open-telemetry#743)

Co-authored-by: Carter Socha <43380952+cartersocha@users.noreply.github.com>
Co-authored-by: Juliano Costa <julianocosta89@outlook.com>

* Nit fix contributing guide (open-telemetry#747)

---------

Signed-off-by: svrnm <neumanns@cisco.com>
Signed-off-by: Pierre Tessier <pierre@pierretessier.com>
Co-authored-by: Josh Lee <joshleecreates@users.noreply.github.com>
Co-authored-by: Severin Neumann <neumanns@cisco.com>
Co-authored-by: Juliano Costa <julianocosta89@outlook.com>
Co-authored-by: Carter Socha <43380952+cartersocha@users.noreply.github.com>
Co-authored-by: Marc Pichler <marc.pichler@dynatrace.com>
Co-authored-by: Cedric Ziel <cedric.ziel@grafana.com>
Co-authored-by: Brett McBride <brett@deakin.edu.au>
Co-authored-by: James Lawrie <james.lawrie@nytimes.com>
Co-authored-by: Austin Parker <austin@ap2.io>
Co-authored-by: jack-berg <34418638+jack-berg@users.noreply.github.com>
Co-authored-by: Oscar Reyes <oscar-rreyes1@hotmail.com>
Co-authored-by: Marc Pichler <marcpi@edu.aau.at>
Co-authored-by: Pierre Tessier <pierre@pierretessier.com>
Co-authored-by: Pawel Brzoska <53935084+pbrzoska-sumo@users.noreply.github.com>
Co-authored-by: Abhijeet Prasad <aprasad@sentry.io>
Co-authored-by: Patrice Chalin <chalin@users.noreply.github.com>
Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: Alex Boten <alex@boten.ca>
Co-authored-by: Goutham Veeramachaneni <gouthamve+github@gmail.com>
Co-authored-by: Cijo Thomas <cithomas@microsoft.com>
styblope pushed a commit to instana/opentelemetry-demo that referenced this pull request Mar 12, 2023
* Add cross-compilation for shipping service

Signed-off-by: svrnm <neumanns@cisco.com>

* Update CHANGELOG.md

* Update Dockerfile

---------

Signed-off-by: svrnm <neumanns@cisco.com>
juliangiuca pushed a commit to juliangiuca/opentelemetry-demo that referenced this pull request Apr 12, 2023
* Add cross-compilation for shipping service

Signed-off-by: svrnm <neumanns@cisco.com>

* Update CHANGELOG.md

* Update Dockerfile

---------

Signed-off-by: svrnm <neumanns@cisco.com>
jmichalak9 pushed a commit to jmichalak9/opentelemetry-demo that referenced this pull request Mar 22, 2024
* Add cross-compilation for shipping service

Signed-off-by: svrnm <neumanns@cisco.com>

* Update CHANGELOG.md

* Update Dockerfile

---------

Signed-off-by: svrnm <neumanns@cisco.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
v1.4 required for 1.4 release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants