hacking: Consider highlighting a toolbx container approach

[garrett]

It might be a good idea to have an "official" toolbx container for development. It would make it much easier to develop for Cockpit and ensure everyone has a working, tested system to start from.

As the toolbox command is available on multiple distributions, toolbx containers can be used by distrobox, and we're already using toolbx, it makes sense to standardize on toolbx.

Therefore, I think we might want to consider promoting @martinpitt's or @allisonkarlitskaya's to be official for the team for usage, or at least make something inspired by those. We could start with Fedora as the dev environment, but it might be good to also consider different environments too, based on Ubuntu, Debian, Arch, openSUSE, etc. But still, even having "just" a Fedora container to start from on Ubuntu would already solve so many of the problems people are having to get a development environment up and working.

• Martin's build-cockpit-toolbox and build-debian-toolbox
• Lis' lisbox

I have my own too, but I have a lot of additional tools I install. Mine's a hard fork from an earlier version of Martin's; we tossed a few ideas to each other. (In my case, it might be interesting to have a Containerfile with a FROM at the top to customize an official container and migrate to that.)

We've been using containers for the website for a few years now, and at least some of the team has been using containers for development. It would be nice to make it official — not just for us, but also for outside developers.

[garrett]

related: #18459

[garrett]

To summarize, I'm asking about two things:

1. Official dev container for Cockpit
2. Improve docs to prefer official container approach first (and fallback later on non-container development)

[martinpitt]

Official dev container for Cockpit

We already have that: https://quay.io/repository/cockpit/tasks This is specifically set up to work with toolbox (or distrobox). Anything that my build-cockpit-toolbox installs in addition is some personal preferences and completely unrelated stuff (like calibre or scanning). In general, toolbox create --image quay.io/cockpit/tasks is the reference container, and I agree it'd be good to document that.

[allisonkarlitskaya]

I agree with what Martin said.

It might make sense to rename it...

[garrett]

Technically, development is a "task" too, right? As long as we document "run this command", it's probably OK? And we can have a suggested name, so the container isn't ambiguously called "tasks". For example:

toolbox create --image quay.io/cockpit/tasks cockpit

(It's probably OK to call a Cockpit development container just "cockpit". I did think about "cockpit-devel", but realistically, virtually everyone will just have one development container for Cockpit, so we should keep it simple for everyone in the docs.)

We could also have a blurb about it being the exact same environment as our bots use. But that's optional. It's cool though, and reassuring as to why a container-based approach like this is ideal, however.

1. We could create a superset container with a few extra tools that inherits from tasks too. But then that's another moving part.
   • Any tool that most people use (such as something like gh, the GitHub CLI tool) might be considered for tasks anyway. It's likely "bloat" for the bots, but useful for development.
2. Probably better yet: We could have a simple example script to actually build a superset container that inherits from tasks too.
   • We do exactly this for the website, where we inherit from GitHub Pages official container and toss a little configuration on top as an inline Containerfile. The question is if we'd want to keep it fully shell or mix in Dockerisms. (It probably doesn't matter really. It'd be toolbox run foo versus RUN foo effectively.)
   • Anyone who would want a superset container we create (option # 1) would actually probably want to customize it themselves, so this is really a better approach anyway.

Considering the above, I think we shouldn't create a superset (option # 1). We should just use the tasks as-is or possibly throw in a useful tool or two. I do think the script (option # 2) would be a good idea, to let people make their environment more comfortable without having to set it up with special stuff manually each time. (They could of course, but it's better to guide them toward the concept of reproducibility.)

Summary

1. Keep it tasks. Lower effort.
2. Document it, but make the resulting container in the example not called tasks, but (probably) cockpit.
3. Share a simple (and easily customizable) all-in-one example script (somewhere) to push people in the direction of reproducibility of their dev environment. It would be based on the tasks container and build a local superset container.

What do you think?

[martinpitt]

@garrett : I agree to your summary. Like you, I am against creating an actual "superset" container, as (1) we wouldn't test it, and it's easy to get out of sync (browser versions!), and (2) this will very quickly lead into the "vim vs. emacs" kind of rabbit hole. The thing is, you don't really need an editor etc. inside the toolbox, you can just continue to edit it on your normal machine (even with VSCode or what not). You'd just build code, VMs, and run tests inside of it. Anyway, I'd leave that kind of flexibility to the developer -- they can use cockpit/tasks as is, or do whichever dnf install or container derivation that they like.

I would reorganize HACKING as follows:

• Move the tools/webpack-* bits to the top. This is what most devs want to start with (hacking on JS), and they should not start with all the "install foo-dev and run autogen.sh etc." struggle.
• Then for C development or integration test, show how to use toolbox
• Point to tools/debian/control and tools/cockpit.spec for devs who want to install dev dependencies locally. This is for people who know what they are doing (which is totally fine!), but it shouldn't be the recommended "no trouble" approach for beginners.

WDYT?

[garrett]

The thing is, you don't really need an editor etc. inside the toolbox, you can just continue to edit it on your normal machine (even with VSCode or what not).

If your editor has a terminal inside (as VS Code does), then you'd want to have the terminal open to the toolbox. Additionally, if your editor requires commands to run inside the container (as VS Code, vim, emacs, etc. all do — having eslint integration, for example, without installing all the commands as overlays on your host), then you'd want it to have access to the commands inside the container, meaning that the best method is to run your editor from within the container.

I meant the script as a way to show how to easily make a custom reproducible build. I actually have a script here that is really simple (easy to follow) mostly works (but different from all of ours) and as a bonus, it supports both toolbox and distrobox (and I've tried distrobox, and it works well for this also; I was able to compile Cockpit from scratch with it just like in toolbox). It's not quite in a sharing state yet, but almost. But I have other things to focus on at the moment (as usual).

So I'm suggesting, for now: We could rework the docs to focus on a toolbox container way first, with the barebones container. Then the script as an example is just "icing on top" (a bonus add-on) later. That's what I was thinking anyway.

[garrett]

Move the tools/webpack-* bits to the top. This is what most devs want to start with (hacking on JS), and they should not start with all the "install foo-dev and run autogen.sh etc." struggle.

I think what we might want to do is split it up into two documents. Or at least have a Table of Contents (which the website has support for) or at least jump links to jump to the section, if in the same document.

Getting started really needs to be simple and in chronological order, however. You can't deal with webpack if you don't have webpack. To get webpack, you need to have a dev environment. The whole point of this issue is to streamline getting to having a working dev environment in a quick, easy manner, so we're not spending half the document talking about getting a dev environment working and focusing mainly on Fedora (omitting Debian, Ubuntu, Arch, openSUSE, etc.) and dependencies.

Therefore, the suggestion was to focus on getting to a state where you can use a container to do stuff (including webpack). We could move manual non-container dev environment to another doc entirely and make it legacy and/or advanced. On a separate and manual installation page, we could then expand on what deps one needs for whatever distros at length... and not have it get in the way of all non-setup developer docs.

[martinpitt]

I sent PR #18503 for that.