Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker-slim died, with the following msg: level=fatal msg="docker-slim: failure" error="unexpected event type" #66

Closed
dforouhari opened this issue Feb 4, 2019 · 20 comments

Comments

@dforouhari
Copy link

Running docker-slim failed with a 'fatal' error msg. Plz see below for the details:

Build env: CentOS 7
Docker version: 18.03.0-ce, build 0520e24
Docker-slim version: 1.23
Docker img info:
Ubuntu: 16.04
Code in Bash + Python
Size = 1.6GB

Output of docker-slim:
$ ./docker-slim build --http-probe img_demo
docker-slim[build]: state=started
docker-slim[build]: info=params target=img_demo continue.mode=enter
docker-slim[build]: state=inspecting.image
docker-slim[build]: info=image id=sha256:bbb4e2cdbb403e8a4508a7921647b1df580ea0a00c0b7f3fa59ae540e39f6129 siz e.bytes=1622646238 size.human=1.6 GB
docker-slim[build]: state=inspecting.container
time="2019-02-04T13:27:42-08:00" level=fatal msg="docker-slim: failure" error="unexpected event type" stack=" goroutine 1 [running]:
runtime/debug.Stack(0x3, 0xc0000c0b40, 0xc00000ef00)
/usr/local/Cellar/go/1.11.1/libexec/src/runtime/debug/stack.go:24 +0xa7
github.com/docker-slim/docker-slim/pkg/utils/errutils.FailOn(0x8fb8a0, 0xc0000472b0)
/Users/q/docker-slim/docker-slim/_gopath/src/github.com/docker-slim/docker-slim/pkg/utils/errutils/er rutils.go:14 +0x51
github.com/docker-slim/docker-slim/internal/app/master/commands.OnBuild(0x0, 0x0, 0x88a600, 0x0, 0x0, 0xc0001 3c0f0, 0x7ffdd0899774, 0x8, 0x0, 0x0, ...)
/Users/q/docker-slim/docker-slim/_gopath/src/github.com/docker-slim/docker-slim/internal/app/master/c ommands/build.go:115 +0xc47
github.com/docker-slim/docker-slim/internal/app/master.init.0.func5(0xc000146140, 0x0, 0xc000146140)
/Users/q/docker-slim/docker-slim/_gopath/src/github.com/docker-slim/docker-slim/internal/app/master/c li.go:480 +0xd76
github.com/docker-slim/docker-slim/vendor/github.com/codegangsta/cli.HandleAction(0x7f26c0, 0x8a0b00, 0xc0001 46140, 0xc000140000, 0x0)
/Users/q/docker-slim/docker-slim/_gopath/src/github.com/docker-slim/docker-slim/vendor/github.com/cod egangsta/cli/app.go:485 +0xc8
github.com/docker-slim/docker-slim/vendor/github.com/codegangsta/cli.Command.Run(0x884f35, 0x5, 0x0, 0x0, 0xc 000047530, 0x1, 0x1, 0x89a23e, 0x3e, 0x0, ...)
/Users/q/docker-slim/docker-slim/_gopath/src/github.com/docker-slim/docker-slim/vendor/github.com/cod egangsta/cli/command.go:207 +0x991
github.com/docker-slim/docker-slim/vendor/github.com/codegangsta/cli.(*App).Run(0xc000010ea0, 0xc00000e080, 0 x4, 0x4, 0x0, 0x0)
/Users/q/docker-slim/docker-slim/_gopath/src/github.com/docker-slim/docker-slim/vendor/github.com/cod egangsta/cli/app.go:250 +0x6ea
github.com/docker-slim/docker-slim/internal/app/master.runCli()
/Users/q/docker-slim/docker-slim/_gopath/src/github.com/docker-slim/docker-slim/internal/app/master/c li.go:730 +0x55
github.com/docker-slim/docker-slim/internal/app/master.Run()
/Users/q/docker-slim/docker-slim/_gopath/src/github.com/docker-slim/docker-slim/internal/app/master/a pp.go:6 +0x25
main.main()
/Users/q/docker-slim/docker-slim/_gopath/src/github.com/docker-slim/docker-slim/cmd/docker-slim/main. go:8 +0x20
" version="linux|Tetra|1.23|8488277befdf57a173acc64973f984caab551ed2|2019-02-03_04:05:17PM (go1.11.1)"

@kcq
Copy link
Member

kcq commented Feb 10, 2019

Thank you for reporting this! I really appreciate! Can you enable debugging and try it again. It's a good idea to enable container logging too. With both enabled your command should look like this: ./docker-slim --debug build --show-clogs --http-probe img_demo

@dforouhari
Copy link
Author

dforouhari commented Feb 11, 2019 via email

@dforouhari
Copy link
Author

dforouhari commented Feb 20, 2019 via email

@kcq
Copy link
Member

kcq commented Feb 20, 2019

This appears to be IPC related. The master app most likely timed out waiting for the start event. It's not clear why without seeing the container logs too. Can you see if you have any container logs for the container with the name that starts with dockerslimk_. Can you check if it's still running or if it exited with an error code?

@dforouhari
Copy link
Author

dforouhari commented Feb 20, 2019 via email

@kcq
Copy link
Member

kcq commented Feb 23, 2019

Thank you! That's exactly what I was looking for :)

The master timed out waiting for the sensor because the sensor exited when it encountered the 'operation not permitted' error from our container log trace. It's strange that it happened with Ubuntu 16.04. There's enough info for a repro. I'll post an update soon.

Thanks again for helping me track this down! I really appreciate it.

@kcq
Copy link
Member

kcq commented Feb 23, 2019

One more question... How is your Dockerfile structured? What user do you use in the image? Can you share the user related Dockerfile lines?

@dforouhari
Copy link
Author

dforouhari commented Feb 23, 2019 via email

@dforouhari
Copy link
Author

dforouhari commented Feb 23, 2019 via email

@dforouhari
Copy link
Author

dforouhari commented Feb 24, 2019 via email

@kcq
Copy link
Member

kcq commented Feb 24, 2019

Thanks for confirming that you are using a custom user in your image. This is the reason for failure because that user isn't privileged enough to start fsnotify. Working on an enhancement to support images with non-default users. Thanks for your help identifying the problem!

@kcq kcq added the WIP label Feb 24, 2019
@dforouhari
Copy link
Author

dforouhari commented Feb 24, 2019 via email

@dforouhari
Copy link
Author

dforouhari commented Mar 1, 2019 via email

@kcq
Copy link
Member

kcq commented Mar 12, 2019

The new 1.24 release ( https://github.com/docker-slim/docker-slim/releases/tag/1.24 ) should address this problem.

@dforouhari
Copy link
Author

dforouhari commented Mar 12, 2019 via email

@dforouhari
Copy link
Author

dforouhari commented Mar 20, 2019 via email

@kcq
Copy link
Member

kcq commented Mar 20, 2019

@dforouhari This is interesting and definitely different :-) docker-slim copied the artifacts from the target image including the file permissions and one of those files isn't accessible by a regular user on your host machine. Need to think about how to solve this... One possible solution is for you to run docker-slim from a root shell.

@kcq
Copy link
Member

kcq commented Mar 23, 2019

@dforouhari i'll create a separate issue for the permission problem

@dforouhari
Copy link
Author

dforouhari commented Mar 23, 2019 via email

@kcq
Copy link
Member

kcq commented Mar 23, 2019

Here's the new issue to track it: #73

@kcq kcq closed this as completed Mar 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants