FR: Run Bazel actions inside a container

[ding-ma]

Similar feature to container_run_and_commit in rules_docker

See

[thesayyn]

fundamentally this is a bad idea. see why: #35 (comment)

[alexeagle]

I think it's possible that there's a better way to containerize actions, we should give it some thought. But as we wrote in the main README, the goal here is to prevent unmaintainable design mistakes that we learned from rules_docker.

Also Bazel has its own way of providing isolation for build actions, so it's not clear to me why we would implement another one in rules_oci. I'd like to see some experiments with --experimental_enable_docker_sandbox which allows you to just use strategy=docker. (and --experimental_docker_image allows you to pick the image for the containerized action)

[thesayyn]

I dug a bit into what can be done; this looks possible with the docker strategy. with the https://github.com/GoogleContainerTools/kaniko combination, it should work

[ding-ma]

I'll experiment with --experimental_docker_image and report back findings.

[alexeagle]

I think we should slip this from 1.0 already, since we'd like to get an RC out this week, and it's still a science project.

[aw185176]

Also Bazel has its own way of providing isolation for build actions, so it's not clear to me why we would implement another one in rules_oci

One use case is pre-generating a .bazelrc for remote caching that has a cache key prefix based on the system dependencies that aren't tracked by Bazel. Such a task needs to be ran in the container to be accurate. I do agree that generally speaking such a thing should not really be necessary.

[alexeagle]

@aw185176 I don't think that sounds like a containerization task. If you want Bazel flags to be dynamic based on the system, then a /tools/bazel wrapper is the way to set those flags.

[aw185176]

@aw185176 I don't think that sounds like a containerization task. If you want Bazel flags to be dynamic based on the system, then a /tools/bazel wrapper is the way to set those flags.

With that approach you then need to sync said tools/bazel wrapper to each Bazel repo using the same caching infrastructure. The approach I outlined is based on what the K8s project was doing at one point. My understanding is that it has become somewhat common / recommended outside of that project (eg https://forum.buildkite.community/t/any-experience-setting-up-a-shared-remote-build-cache-using-bazel/1119/4). I wouldn't necessarily recommend it and we are likely to move away from the approach internally when possible, I only wanted to highlight that it is a real thing that real teams do today.

[thesayyn]

Closing as completed #570