[inner-loop] run Isolated docker in debug workspace #16363

iQQBot · 2023-02-13T07:53:23Z

Description

[inner-loop] run Isolated docker in debug workspace

Related Issue(s)

Fixes #

How to test

start a workspace preview env
try gp rebuild
in debug workspace, try use docker, it should Isolated with regular workspace

Release Notes

[inner-loop] run Isolated docker in debug workspace

Documentation

Build Options:

/werft with-github-actions
Experimental feature to run the build with GitHub Actions (and not in Werft).
leeway-no-cache
leeway-target=components:all
/werft no-test
Run Leeway with --dont-test

Publish Options

/werft publish-to-npm
/werft publish-to-jb-marketplace

Installer Options

with-ee-license
with-slow-database
with-dedicated-emulation
with-ws-manager-mk2
workspace-feature-flags
Add desired feature flags to the end of the line above, space separated

Preview Environment Options:

/werft with-local-preview
If enabled this will build install/preview
/werft with-preview
/werft with-large-vm
/werft with-gce-vm
If enabled this will create the environment on GCE infra
/werft with-integration-tests=all
Valid options are all, workspace, webapp, ide, jetbrains, vscode, ssh

werft-gitpod-dev-com · 2023-02-13T07:53:31Z

started the job as gitpod-build-pd-dind.1 because the annotations in the pull request description changed
(with .werft/ from main)

akosyakov · 2023-02-13T09:16:54Z

@iQQBot it is awesome I've noticed following output on shutdown:

[2023-02-13 09:15:43]  INFO Gracefully stopping the debug workspace...
[2023-02-13 09:15:44] ERROR cannot provide Docker activation socket error=wait: no child processes

it could be addressed separately, but we should try no clean up unnecessary or misleading errors. It will make our life better as well.

akosyakov

thank you, it turned out to be not so complicated :)

akosyakov · 2023-02-13T09:43:41Z

@iQQBot if we merge it without docker-up deployed first it is going to break docker in debug workspace all together?

akosyakov · 2023-02-13T09:44:38Z

/hold

we probably should extract a PR for the workspace team first cc @kylos101 would it be possible to patch this thing somehow?

iQQBot · 2023-02-13T09:46:18Z

@iQQBot if we merge it without docker-up deployed first it is going to break docker in debug workspace all together?

Yes....let's extract a PR for workspace team

components/docker-up/docker-up/main.go

akosyakov · 2023-02-14T09:23:25Z

components/gitpod-cli/cmd/rebuild.go

@@ -241,6 +241,11 @@ func runRebuild(ctx context.Context, supervisorClient *supervisor.SupervisorClie
 		return err
 	}

+	err = os.MkdirAll("/workspace/.docker-root-debug", 0710)


@iQQBot I think we should think about long term approach, we will need to move there VS Code user data and JB config/cache folders.

I thought that in regular workspace we should do something like:

/workspace/.gitpod/root/.docker /workspace/.gitpod/root/.vscode-remote /workspace/.gitpod/root/.cache /workspace/.gitpod/root/.config

And then for the debug workspace use something like /workspace/.gitpod/debug-root?

We don't need to change for the regular workspace for now. But for the debug workspace we could do it clean? i.e. start now /workspace/.gitpod/debug-root/.docker.

I wonder whether it also applies to /workspace/.gitpod folder generally? i.e why would we shate gitpod tasks between them?

Maybe instead it should move folders under /workspace/.gitpod and in the debug mount /workspace/.gitpod-debug:/workspace/.gitpod

WDYT? Should we talk sync?

@iQQBot we maybe should consider to keep debug workspace data somewhere under /var/tmp? Because of recent perf concerns

@iQQBot we maybe should consider to keep debug workspace data somewhere under /var/tmp? Because of recent perf concerns

Pudong said it is impossible, because linux allows only 2 layers, and we already have it for root fs

For the structure we agreed on:

/workspace/.gitpod-debug/.docker-root -> /workspace/.docker-root /workspace/.gitpod-debug/.gitpod -> /workspace/.gitpod /workspace/.gitpod-debug/.vscode-remote -> /workspace/.vscode-remote /workspace/.gitpod-debug/.cache -> /workspace/.cache /workspace/.gitpod-debug/.config -> /workspace/.config

akosyakov · 2023-02-14T09:26:56Z

components/supervisor/pkg/supervisor/supervisor.go

@@ -398,9 +398,9 @@ func Run(options ...RunOption) {
 	tasksSuccessChan := make(chan taskSuccess, 1)
 	go taskManager.Run(ctx, &wg, tasksSuccessChan)

-	if !opts.RunGP && !cfg.isDebugWorkspace() {
+	if !opts.RunGP {


in this file we only need to remove a guard

will change after #16368 merged

kylos101 · 2023-02-14T16:46:12Z

@akosyakov we can patch docker-up, we cannot patch ws-daemon.

edit: Assuming this change is dependent on the nsinsider change, you'll need to wait for a new cluster till next week.

kylos101 · 2023-02-17T23:01:03Z

@akosyakov I rerequested your approval because this comment. Also, /var/tmp uses ephemeral storage, which has a smaller limit than /workspace class. For reference /workspace sizes are here.

I'm not sure how much content you're thinking about putting there, but, depending on size, it could violate ephemeral storage limits and cause workspace to stop. Standard ephemeral storage limit is 10GB, Large is 20GB.

edit: also there's a conflict with supervisor.go

iQQBot · 2023-02-21T15:36:59Z

/werft run

👍 started the job as gitpod-build-pd-dind.12
(with .werft/ from main)

iQQBot · 2023-02-21T15:53:35Z

/hold

this PR need hold until new workspace cluster deployed

iQQBot · 2023-02-21T15:56:00Z

already tested

akosyakov

lgtm

/hold

let's wait for workspace deployment

kylos101 · 2023-02-24T21:49:58Z

Workspace deployment is done (both regions are at 100%) @akosyakov @iQQBot

iQQBot · 2023-02-25T03:31:52Z

/unhold

iQQBot requested review from a team February 13, 2023 07:53

roboquat added the release-note label Feb 13, 2023

roboquat added the size/S label Feb 13, 2023

github-actions bot added team: workspace Issue belongs to the Workspace team team: IDE labels Feb 13, 2023

iQQBot marked this pull request as draft February 13, 2023 07:53

roboquat added the do-not-merge/work-in-progress label Feb 13, 2023

iQQBot force-pushed the pd/dind branch 3 times, most recently from 4baee6f to 4d4b239 Compare February 13, 2023 08:45

iQQBot marked this pull request as ready for review February 13, 2023 08:57

roboquat removed the do-not-merge/work-in-progress label Feb 13, 2023

akosyakov approved these changes Feb 13, 2023

View reviewed changes

roboquat added the do-not-merge/hold label Feb 13, 2023

iQQBot force-pushed the pd/dind branch from 4d4b239 to f67aa1e Compare February 13, 2023 09:50

iQQBot mentioned this pull request Feb 13, 2023

[workspace] change iface name to eth0 #16368

Merged

15 tasks

akosyakov reviewed Feb 13, 2023

View reviewed changes

components/docker-up/docker-up/main.go Outdated Show resolved Hide resolved

akosyakov reviewed Feb 13, 2023

View reviewed changes

components/docker-up/docker-up/main.go Outdated Show resolved Hide resolved

iQQBot force-pushed the pd/dind branch from f67aa1e to 17261bd Compare February 13, 2023 15:58

akosyakov reviewed Feb 14, 2023

View reviewed changes

kylos101 requested a review from akosyakov February 17, 2023 22:55

iQQBot marked this pull request as draft February 20, 2023 15:27

roboquat added the do-not-merge/work-in-progress label Feb 20, 2023

iQQBot force-pushed the pd/dind branch from 17261bd to 56db55f Compare February 21, 2023 13:37

roboquat added size/M and removed size/S labels Feb 21, 2023

iQQBot force-pushed the pd/dind branch 3 times, most recently from 3327780 to aa33c8c Compare February 21, 2023 13:59

[inner-loop] run Isolated docker in debug workspace

5749c18

iQQBot force-pushed the pd/dind branch from aa33c8c to 5749c18 Compare February 21, 2023 14:46

iQQBot marked this pull request as ready for review February 21, 2023 15:53

roboquat removed the do-not-merge/work-in-progress label Feb 21, 2023

iQQBot removed the request for review from a team February 21, 2023 15:56

akosyakov approved these changes Feb 22, 2023

View reviewed changes

roboquat removed the do-not-merge/hold label Feb 25, 2023

roboquat merged commit 884a177 into main Feb 25, 2023

roboquat deleted the pd/dind branch February 25, 2023 03:32

roboquat added deployed: IDE IDE change is running in production deployed: workspace Workspace team change is running in production deployed Change is completely running in production labels Feb 28, 2023

axonasif mentioned this pull request Feb 28, 2023

docker-compose is partially not working (panic) when used in a .gitpod.yml task #16614

Closed

loujaybee mentioned this pull request Mar 3, 2023

Epic: Remove need to commit to git (Gitpod "Inner Loop") #7671

Closed

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[inner-loop] run Isolated docker in debug workspace #16363

[inner-loop] run Isolated docker in debug workspace #16363

iQQBot commented Feb 13, 2023 •

edited

Loading

werft-gitpod-dev-com bot commented Feb 13, 2023

akosyakov commented Feb 13, 2023

akosyakov left a comment

akosyakov commented Feb 13, 2023

akosyakov commented Feb 13, 2023

iQQBot commented Feb 13, 2023

akosyakov Feb 14, 2023

akosyakov Feb 17, 2023

akosyakov Feb 20, 2023

akosyakov Feb 14, 2023

iQQBot Feb 17, 2023

kylos101 commented Feb 14, 2023 •

edited

Loading

kylos101 commented Feb 17, 2023 •

edited

Loading

iQQBot commented Feb 21, 2023 •

edited by werft-gitpod-dev-com bot

Loading

iQQBot commented Feb 21, 2023

iQQBot commented Feb 21, 2023

akosyakov left a comment

kylos101 commented Feb 24, 2023

iQQBot commented Feb 25, 2023

[inner-loop] run Isolated docker in debug workspace #16363

[inner-loop] run Isolated docker in debug workspace #16363

Conversation

iQQBot commented Feb 13, 2023 • edited Loading

Description

Related Issue(s)

How to test

Release Notes

Documentation

Build Options:

Preview Environment Options:

werft-gitpod-dev-com bot commented Feb 13, 2023

akosyakov commented Feb 13, 2023

akosyakov left a comment

Choose a reason for hiding this comment

akosyakov commented Feb 13, 2023

akosyakov commented Feb 13, 2023

iQQBot commented Feb 13, 2023

akosyakov Feb 14, 2023

Choose a reason for hiding this comment

akosyakov Feb 17, 2023

Choose a reason for hiding this comment

akosyakov Feb 20, 2023

Choose a reason for hiding this comment

akosyakov Feb 14, 2023

Choose a reason for hiding this comment

iQQBot Feb 17, 2023

Choose a reason for hiding this comment

kylos101 commented Feb 14, 2023 • edited Loading

kylos101 commented Feb 17, 2023 • edited Loading

iQQBot commented Feb 21, 2023 • edited by werft-gitpod-dev-com bot Loading

iQQBot commented Feb 21, 2023

iQQBot commented Feb 21, 2023

akosyakov left a comment

Choose a reason for hiding this comment

kylos101 commented Feb 24, 2023

iQQBot commented Feb 25, 2023

iQQBot commented Feb 13, 2023 •

edited

Loading

kylos101 commented Feb 14, 2023 •

edited

Loading

kylos101 commented Feb 17, 2023 •

edited

Loading

iQQBot commented Feb 21, 2023 •

edited by werft-gitpod-dev-com bot

Loading