Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Usage? #44

Closed
gcstang opened this issue Feb 2, 2018 · 14 comments
Closed

Usage? #44

gcstang opened this issue Feb 2, 2018 · 14 comments
Labels

Comments

@gcstang
Copy link

gcstang commented Feb 2, 2018

Maybe I'm confused on how this works but I'm trying to run this on an alpine image with a simple command 'echo "hello world"'. Should this work and make this smaller or is there more required to use your tool?

Simple image from a Dockerfile
`FROM alpine

ENTRYPOINT ["echo","hello world"]`

docker build -t mine .

./docker-slim build mine

@kcq
Copy link
Member

kcq commented Feb 3, 2018

Thank you for the report and thank you for the sample Dockerfile!

Where exactly did you get stuck? Did the 'docker-slim build' command fail? What do you see when you run 'docker images'?

I tried to repro your use case and here's what I got (using the Dockerfile you provided):

$ docker build -t q/ds-issue-44 .

<note: this command created the initial Docker image>

$ docker images

REPOSITORY TAG IMAGE ID CREATED SIZE
q/ds-issue-44 latest b33a7a958577 2 minutes ago 257MB

Now it's time to slim the image (note: skipped most of the command output)...

$ docker-slim build --show-clogs=true q/ds-issue-44
...
docker-slim: press when you are done using the container...
...
Successfully tagged q/ds-issue-44.slim:latest
INFO[0007] docker-slim: created new image: q/ds-issue-44.slim ( has data artifacts: true )

$ docker images

REPOSITORY TAG IMAGE ID CREATED SIZE
q/ds-issue-44.slim latest 30219672f4b9 2 minutes ago 1.39MB
q/ds-issue-44 latest b33a7a958577 6 minutes ago 257MB

The output shows that docker-slim created a minified version of the original alpine image (that was 257MB) and this minified version is 1.39MB.

Now let's run the minified image:

$ docker run --rm -it q/ds-issue-44.slim
hello world

P.S.
Don't forget to use the latest release :)

@gcstang
Copy link
Author

gcstang commented Feb 3, 2018

I'm not sure how your image was 257MB(maybe you're building something else) mine is only 4.15MB after creation but then once running docker slim it becomes 0MB.

`
(Same Dockerfile as above)
$ docker build -t test1 .
test1 latest 132df7fc1b65 4 seconds ago 4.15MB
$ ./docker-slim build --show-clogs=true test1

docker-slim: [build] image=test1 http-probe=false remove-file-artifacts=false image-overrides=map[] entrypoint=[] (false) cmd=[] (false) workdir='' env=[] expose=map[]
INFO[0000] docker-slim: inspecting 'fat' image metadata...
INFO[0000] docker-slim: [sha256:132df7fc1b65dcd6adc2389ba8a90ca7da481c2adcdbf77e0c28da672493c9f4] 'fat' image size => 4147781 (4.1 MB)
INFO[0000] docker-slim: processing 'fat' image info...
INFO[0000] docker-slim: starting instrumented 'fat' container...
INFO[0000] docker-slim: created container => b0a78b1da9e127206f49a2fde26a170a3ab0a995e29cf0b3248cbb2c943411a5
INFO[0000] docker-slim: watching container monitor...
docker-slim: press when you are done using the container...

<!!! I PUSHED ENTER HERE !!!>

INFO[0003] docker-slim: waiting for the container to finish its work...
INFO[0003] docker-slim: shutting down 'fat' container...
docker-slim: container stdout:
hello world
docker-slim: container stderr:
time="2018-02-03T21:12:56Z" level=info msg="sensor: args => []string{"/opt/dockerslim/bin/sensor"}"
time="2018-02-03T21:12:56Z" level=info msg="sensor: creating event publisher..."
time="2018-02-03T21:12:56Z" level=info msg="sensor: creating cmd server..."
time="2018-02-03T21:12:56Z" level=info msg="sensor: waiting for commands..."
time="2018-02-03T21:12:56Z" level=info msg="sensor: monitor starting..."
time="2018-02-03T21:12:56Z" level=info msg="fanmon: starting..."
time="2018-02-03T21:12:56Z" level=info msg="ptmon: starting..."
time="2018-02-03T21:12:56Z" level=info msg="ptmon: monitor is exiting... status= 0"
time="2018-02-03T21:12:56Z" level=info msg="ptmon: done =>0"
time="2018-02-03T21:12:59Z" level=info msg="sensor: waiting for monitor to finish..."
time="2018-02-03T21:12:59Z" level=info msg="fanmon: stopping..."
time="2018-02-03T21:12:59Z" level=info msg="sensor: published 'monitor.finish.completed'"
time="2018-02-03T21:12:59Z" level=info msg="sensor: done!"
docker-slim: end of container logs =============
INFO[0004] docker-slim: processing instrumented 'fat' container info...
INFO[0004] docker-slim: generating AppArmor profile...
INFO[0004] docker-slim: building 'slim' image...
INFO[0004] docker-slim: WARNING - no data artifacts
Step 1/3 : FROM scratch
--->
Step 2/3 : ENV PATH "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
---> Running in b7b2e8d18d4d
Removing intermediate container b7b2e8d18d4d
---> 685b97ad18ff
Step 3/3 : ENTRYPOINT ["echo","hello world"]
---> Running in 21631469e872
Removing intermediate container 21631469e872
---> 7c5ba6c117dc
Successfully built 7c5ba6c117dc
Successfully tagged test1.slim:latest
INFO[0004] docker-slim: created new image: test1.slim ( has data artifacts: false )

$ docker images
test1.slim latest 7c5ba6c117dc About a minute ago 0B

$ docker run --rm -it test1.slim
docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: "echo": executable file not found in $PATH": unknown.
`

@kcq
Copy link
Member

kcq commented Feb 4, 2018

The alpine image was that big because i used the Go alpine image instead of the vanilla alpine image :-) I'll try it with a vanilla alpine too.

The "WARNING - no data artifacts" message means that something happened with the target container during the inspection phase it didn't produce any artifacts.

Are you running it on Linux or Mac OS X?

@kcq
Copy link
Member

kcq commented Feb 4, 2018

Tried it with the latest release (1.19) and a vanilla alpine image:

$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
q/ds-issue-44.slim latest 6c9d520739cb 12 seconds ago 1.39MB
q/ds-issue-44 latest a20b34c608dc 48 seconds ago 3.99MB

$ docker run --rm -it q/ds-issue-44.slim
hello world

@gcstang
Copy link
Author

gcstang commented Feb 4, 2018 via email

@kcq
Copy link
Member

kcq commented Feb 7, 2018

Very interesting :) Sorry for the headaches! What's the Docker version you are using?

Can you also try running docker-slim with more debugging information like this:
./docker-slim --debug build --show-clogs=true test1

@gcstang
Copy link
Author

gcstang commented Feb 7, 2018

No problem.

Mac Edge version 18.02.0-ce-rc2-mac51 (22446)

`./docker-slim --debug build --show-clogs=true test1
docker-slim: [build] image=test1 http-probe=false remove-file-artifacts=false image-overrides=map[] entrypoint=[] (false) cmd=[] (false) workdir='' env=[] expose=map[]
DEBU[0000] docker-slim: new Docker client (default) [6]
DEBU[0000] docker-slim: configured DOCKER_HOST env var
INFO[0000] docker-slim: inspecting 'fat' image metadata...
INFO[0000] docker-slim: [sha256:132df7fc1b65dcd6adc2389ba8a90ca7da481c2adcdbf77e0c28da672493c9f4] 'fat' image size => 4147781 (4.1 MB)
INFO[0000] docker-slim: processing 'fat' image info...
DEBU[0000]

IMAGE HISTORY =>
[]docker.ImageHistory{docker.ImageHistory{ID:"sha256:132df7fc1b65dcd6adc2389ba8a90ca7da481c2adcdbf77e0c28da672493c9f4", Tags:[]string{"test1:latest"}, Created:1517692328, CreatedBy:"/bin/sh -c #(nop) ENTRYPOINT ["echo" "hello world"]", Size:0, Comment:""}, docker.ImageHistory{ID:"sha256:3fd9065eaf02feaf94d68376da52541925650b81698c53c6824d92ff63f98353", Tags:[]string{"alpine:latest"}, Created:1515532258, CreatedBy:"/bin/sh -c #(nop) CMD ["/bin/sh"]", Size:0, Comment:""}, docker.ImageHistory{ID:"", Tags:[]string(nil), Created:1515532258, CreatedBy:"/bin/sh -c #(nop) ADD file:093f0723fa46f6cdbd6f7bd146448bb70ecce54254c35701feeceb956414622f in / ", Size:4147781, Comment:""}}

DEBU[0000] IMAGE INSTRUCTIONS:
DEBU[0000] # new image
DEBU[0000] ADD file:093f0723fa46f6cdbd6f7bd146448bb70ecce54254c35701feeceb956414622f in /
DEBU[0000] CMD ["/bin/sh"]
DEBU[0000] # end of image: alpine (id: sha256:3fd9065eaf02feaf94d68376da52541925650b81698c53c6824d92ff63f98353 tags: latest)
DEBU[0000]
DEBU[0000] # new image
DEBU[0000] ENTRYPOINT ["echo" "hello world"]
DEBU[0000] # end of image: test1 (id: sha256:132df7fc1b65dcd6adc2389ba8a90ca7da481c2adcdbf77e0c28da672493c9f4 tags: latest)
DEBU[0000]
INFO[0000] docker-slim: starting instrumented 'fat' container...
INFO[0000] docker-slim: created container => ce7c212fbd798465c998b2da3285b2f666a25bdd4366dd77298a1ddba11ea97b
DEBU[0000] container NetworkSettings.Ports => map[docker.Port][]docker.PortBinding{"65502/tcp":[]docker.PortBinding{docker.PortBinding{HostIP:"0.0.0.0", HostPort:"32768"}}, "65501/tcp":[]docker.PortBinding{docker.PortBinding{HostIP:"0.0.0.0", HostPort:"32769"}}}
DEBU[0000] cmdChannelAddr=tcp://127.0.0.1:32769 evtChannelAddr=tcp://127.0.0.1:32768
DEBU[0000] sendCmd(&{echo [hello world] [] []})
INFO[0000] docker-slim: watching container monitor...
docker-slim: press when you are done using the container...

DEBU[0014] sendCmd(&{})
DEBU[0014] 'stop' response => 'ok'
INFO[0014] docker-slim: waiting for the container to finish its work...
DEBU[0014] getEvt()
DEBU[0014] docker-slim: sensor event => 'monitor.finish.completed'
INFO[0014] docker-slim: shutting down 'fat' container...
DEBU[0014] docker-slim: getting container logs => ce7c212fbd798465c998b2da3285b2f666a25bdd4366dd77298a1ddba11ea97b
docker-slim: container stdout:
hello world
docker-slim: container stderr:
time="2018-02-07T18:09:22Z" level=info msg="sensor: args => []string{"/opt/dockerslim/bin/sensor", "-d"}"
time="2018-02-07T18:09:22Z" level=debug msg="sensor: cwd => "/""
time="2018-02-07T18:09:22Z" level=debug msg="sensor: setting up channels..."
time="2018-02-07T18:09:22Z" level=info msg="sensor: creating event publisher..."
time="2018-02-07T18:09:22Z" level=info msg="sensor: creating cmd server..."
time="2018-02-07T18:09:22Z" level=info msg="sensor: waiting for commands..."
time="2018-02-07T18:09:22Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:22Z" level=debug msg="sensor: cmd server - got a command => {"name":"cmd.monitor.start","data":{"app_name":"echo","app_args":["hello world"]}}"
time="2018-02-07T18:09:22Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:22Z" level=debug msg="
sensor: command => &{echo [hello world] [] []}"
time="2018-02-07T18:09:22Z" level=debug msg="sensor: 'start' command (&messages.StartMonitor{AppName:"echo", AppArgs:[]string{"hello world"}, Excludes:[]string(nil), Includes:[]string(nil)}) - starting monitor..."
time="2018-02-07T18:09:22Z" level=info msg="sensor: monitor starting..."
time="2018-02-07T18:09:22Z" level=info msg="fanmon: starting..."
time="2018-02-07T18:09:22Z" level=info msg="ptmon: starting..."
time="2018-02-07T18:09:22Z" level=debug msg="sensor: target app started => echo []string{"hello world"}"
time="2018-02-07T18:09:22Z" level=debug msg="sensor: monitor - waiting to stop monitoring..."
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: fanRunMonitor worker starting"
time="2018-02-07T18:09:22Z" level=debug msg="sensor.startTargetApp(echo,[hello world],/)"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: fanRunMonitor worker (monitor) starting"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: data.Mask =>20"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file open"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file path =>/bin/busybox"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: data.Mask =>1"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: event 1"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file read"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file path =>/bin/busybox"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: event 2"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: data.Mask =>1"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file read"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file path =>/bin/busybox"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: data.Mask =>20"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file open"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file path =>/lib/ld-musl-x86_64.so.1"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: event 3"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: event 4"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: data.Mask =>1"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file read"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: file path =>/lib/ld-musl-x86_64.so.1"
time="2018-02-07T18:09:22Z" level=debug msg="fanmon: event 5"
time="2018-02-07T18:09:22Z" level=debug msg="sensor.startTargetApp: started target app --> PID=15"
time="2018-02-07T18:09:22Z" level=debug msg="ptmon: target PID ==> 15"
time="2018-02-07T18:09:22Z" level=debug msg="ptmon: initial process status => 1407"
time="2018-02-07T18:09:22Z" level=info msg="ptmon: monitor is exiting... status= 0"
time="2018-02-07T18:09:22Z" level=info msg="ptmon: done =>0"
time="2018-02-07T18:09:22Z" level=debug msg="ptmon: executed syscall count = 7"
time="2018-02-07T18:09:22Z" level=debug msg="ptmon: number of syscalls: 6"
time="2018-02-07T18:09:22Z" level=debug msg="[102] getuid = 1"
time="2018-02-07T18:09:22Z" level=debug msg="[1] write = 1"
time="2018-02-07T18:09:22Z" level=debug msg="[59] execve = 1"
time="2018-02-07T18:09:22Z" level=debug msg="[158] arch_prctl = 1"
time="2018-02-07T18:09:22Z" level=debug msg="[218] set_tid_address = 1"
time="2018-02-07T18:09:22Z" level=debug msg="[10] mprotect = 2"
time="2018-02-07T18:09:25Z" level=debug msg="sensor: cmd server - timeout... ok"
time="2018-02-07T18:09:25Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:28Z" level=debug msg="sensor: cmd server - timeout... ok"
time="2018-02-07T18:09:28Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:30Z" level=debug msg=.
time="2018-02-07T18:09:31Z" level=debug msg="sensor: cmd server - timeout... ok"
time="2018-02-07T18:09:31Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:34Z" level=debug msg="sensor: cmd server - timeout... ok"
time="2018-02-07T18:09:34Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:35Z" level=debug msg=.
time="2018-02-07T18:09:36Z" level=debug msg="sensor: cmd server - got a command => {"name":"cmd.monitor.stop"}"
time="2018-02-07T18:09:36Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:36Z" level=debug msg="
sensor: command => &{}"
time="2018-02-07T18:09:36Z" level=debug msg="sensor: 'stop' command - stopping monitor..."
time="2018-02-07T18:09:36Z" level=info msg="sensor: waiting for monitor to finish..."
time="2018-02-07T18:09:36Z" level=debug msg="sensor: monitor - stop message..."
time="2018-02-07T18:09:36Z" level=debug msg="sensor: monitor - processing data..."
time="2018-02-07T18:09:36Z" level=info msg="fanmon: stopping..."
time="2018-02-07T18:09:36Z" level=debug msg="fanmon: sending report (processed 5 events)..."
time="2018-02-07T18:09:36Z" level=debug msg="findSymlinks - deviceId=103"
time="2018-02-07T18:09:36Z" level=debug msg="includePaths: map[]"
time="2018-02-07T18:09:36Z" level=debug msg="excludePaths: map[]"
time="2018-02-07T18:09:36Z" level=debug msg="sensor: monitor - saving report to /opt/dockerslim/artifacts/creport.json"
time="2018-02-07T18:09:36Z" level=debug msg="sensor: trying to publish 'monitor.finish.completed' event (attempt 1)"
time="2018-02-07T18:09:36Z" level=info msg="sensor: published 'monitor.finish.completed'"
time="2018-02-07T18:09:36Z" level=info msg="sensor: done!"
time="2018-02-07T18:09:36Z" level=debug msg="defered cleanup on shutdown..."
time="2018-02-07T18:09:36Z" level=debug msg="cleanupOnShutdown()"
time="2018-02-07T18:09:36Z" level=debug msg="sensor: cmd server - error => connection closed"
time="2018-02-07T18:09:36Z" level=debug msg="sensor: cmd server - waiting for a command..."
time="2018-02-07T18:09:36Z" level=debug msg="sensor: cmd server - done..."
docker-slim: end of container logs =============
INFO[0015] docker-slim: processing instrumented 'fat' container info...
INFO[0015] docker-slim: generating AppArmor profile...
DEBU[0015] docker-slim: saving seccomp profile to /Users/gcollins/Desktop/dockerslim/.images/132df7fc1b65dcd6adc2389ba8a90ca7da481c2adcdbf77e0c28da672493c9f4/artifacts/test1-seccomp.json
INFO[0015] docker-slim: building 'slim' image...
INFO[0015] docker-slim: WARNING - no data artifacts
Step 1/3 : FROM scratch
--->
Step 2/3 : ENV PATH "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
---> Using cache
---> 685b97ad18ff
Step 3/3 : ENTRYPOINT ["echo","hello world"]
---> Using cache
---> 7c5ba6c117dc
Successfully built 7c5ba6c117dc
Successfully tagged test1.slim:latest
INFO[0015] docker-slim: created new image: test1.slim ( has data artifacts: false )`

@gcstang
Copy link
Author

gcstang commented Feb 20, 2018

@kcq any updates?

@kcq
Copy link
Member

kcq commented Feb 24, 2018

Thank you for the debug output! It's definitely helpful! Can you list the files and directories here: /Users/gcollins/Desktop/dockerslim/.images/132df7fc1b65dcd6adc2389ba8a90ca7da481c2adcdbf77e0c28da672493c9f4/artifacts . Is there anything there? Is there a directory called 'files'?

There should be a message saying that the master app created that 'artifacts' directory, but I don't see it in your debug messages. I also don't see any debug messages from the sensor app showing that it tried to prepare and copy any artifacts, which indicates that the sensor didn't even try to copy the artifacts.

This is a good lead. Thank you! We'll probably need an update release with more debugging information around directory and file I/O operations to surface permission related problems.

@gcstang
Copy link
Author

gcstang commented Feb 26, 2018

Np, yw

These are the files dockerslim/.images/132df7fc1b65dcd6adc2389ba8a90ca7da481c2adcdbf77e0c28da672493c9f4/artifacts/
(There is directory called files)
Dockerfile Dockerfile.fat creport.json test1-apparmor-profile test1-seccomp.json

Latest Docker Version
docker-ce Version 18.02.0-ce-mac53 (22617)

@kcq
Copy link
Member

kcq commented Feb 28, 2018

Thanks for confirming that there's no 'files' directory there! There'll be a new version to try tomorrow. I greatly appreciate your help tracking this down!

@kcq
Copy link
Member

kcq commented Mar 15, 2019

@gcstang Would you be able to try the latest release ( https://github.com/docker-slim/docker-slim/releases/tag/1.24 ) to see if you still see the problem you experienced or if it's gone now? Thanks again for your report!

@gcstang
Copy link
Author

gcstang commented Mar 15, 2019

This worked!
Thank you

@kcq
Copy link
Member

kcq commented Mar 16, 2019

Great to hear!

@kcq kcq closed this as completed Mar 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants