tools/docker: add missing node in syz-env

[jiangenj]

When act -j build in local host, it reports:
exec failed: unable to start container process: exec: "node": executable file
not found in $PATH: unknown

---

Before sending a pull request, please review Contribution Guidelines:
https://github.com/google/syzkaller/blob/master/docs/contributing.md

---

[tarasmadan]

Hi @jiangenj .
Could you please share the steps to reproduce the original problem?

[jiangenj]

Hi @tarasmadan
I'm using https://github.com/nektos/act to run the workflow.

$ act -j build
[ci/build] 🚀  Start image=gcr.io/syzkaller/env:latest
[ci/build]   🐳  docker pull image=gcr.io/syzkaller/env:latest platform= username= forcePull=true
[ci/build]   🐳  docker create image=gcr.io/syzkaller/env:latest platform= entrypoint=["tail" "-f" "/dev/null"] cmd=[] network="host"
[ci/build]   🐳  docker run image=gcr.io/syzkaller/env:latest platform= entrypoint=["tail" "-f" "/dev/null"] cmd=[] network="host"
[ci/build]   ☁  git clone 'https://github.com/actions/cache' # ref=88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8
[ci/build]   ☁  git clone 'https://github.com/codecov/codecov-action' # ref=5ecb98a3c6b747ed38dc09f787459979aebb39be
[ci/build] Non-terminating error while running 'git clone': some refs were not updated
[ci/build] ⭐ Run Main checkout
[ci/build]   🐳  docker cp src=/local/mnt/workspace/jiangenj/gopath/src/github.com/google/syzkaller/. dst=/local/mnt/workspace/jiangenj/gopath/src/github.com/google/syzkaller/gopath/src/github.com/google/syzkaller
[ci/build]   ✅  Success - Main checkout
[ci/build] ⭐ Run Main echo "date=$(/bin/date -u "+%Y%m%d+%T")" >> $GITHUB_OUTPUT
[ci/build]   🐳  docker exec cmd=[bash --noprofile --norc -e -o pipefail /var/run/act/workflow/get-date.sh] user= workdir=
[ci/build]   ✅  Success - Main echo "date=$(/bin/date -u "+%Y%m%d+%T")" >> $GITHUB_OUTPUT
[ci/build]   ⚙  ::set-output:: date=20240715+01:16:57
[ci/build] ⭐ Run Main test build cache
[ci/build]   🐳  docker cp src=/usr2/jiangenj/.cache/act/actions-cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8/ dst=/var/run/act/actions/actions-cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8/
[ci/build]   🐳  docker exec cmd=[node /var/run/act/actions/actions-cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8/dist/restore/index.js] user= workdir=
| OCI runtime exec failed: exec failed: unable to start container process: exec: "node": executable file not found in $PATH: unknown
[ci/build]   ❌  Failure - Main test build cache
[ci/build] exitcode '126': failure
[ci/build] 🏁  Job failed
Error: Job 'build' failed

The error is | OCI runtime exec failed: exec failed: unable to start container process: exec: "node": executable file not found in $PATH: unknown.

[tarasmadan]

Got it. The goal is to test the changes locally. Thank you for this option.
Have you tried ./tools/syz-env make presubmit?
What went wrong if "yes"?

[jiangenj]

Yes syz-env make presubmit is what I normally use, but sometimes even if it succeed, and all checks succeed in PR but it might failed during merge queue build. So I'd like to similar everything steps locally first.
For example #5002 (comment).

It's fine to use only make presubmit if you don't want to add npm into the docker.
But you can see below that github actions have more jobs than presubmit.

[tarasmadan]

It is Ok to add Node.
I'm hesitating to add the alternative presubmit testing.
If you need it - it means existing approaches don't work for you. And I'm curious why.

[tarasmadan]

Could you please also adjust the documentation?
This section may be improved.

[jiangenj]

updated.

[tarasmadan]

I also tried to run docker build -t gcr.io/syzkaller/env tools/docker/env with your changes and it doesn't work:

E: Failed to fetch http://deb.debian.org/debian/pool/main/f/freetype/libfreetype6_2.12.1%2bdfsg-5_amd64.deb  404  Not Found [IP: ---edited---]
E: Failed to fetch http://deb.debian.org/debian/pool/main/g/gdk-pixbuf/libgdk-pixbuf-2.0-0_2.42.10%2bdfsg-1%2bb1_amd64.deb  404  Not Found [IP: ---edited---]
E: Failed to fetch http://deb.debian.org/debian/pool/main/g/gdk-pixbuf/libgdk-pixbuf2.0-bin_2.42.10%2bdfsg-1%2bb1_amd64.deb  404  Not Found [IP: ---edited---]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

@jiangenj, does it work for you?

[jiangenj]

I also tried to run docker build -t gcr.io/syzkaller/env tools/docker/env with your changes and it doesn't work:

E: Failed to fetch http://deb.debian.org/debian/pool/main/f/freetype/libfreetype6_2.12.1%2bdfsg-5_amd64.deb  404  Not Found [IP: ---edited---]
E: Failed to fetch http://deb.debian.org/debian/pool/main/g/gdk-pixbuf/libgdk-pixbuf-2.0-0_2.42.10%2bdfsg-1%2bb1_amd64.deb  404  Not Found [IP: ---edited---]
E: Failed to fetch http://deb.debian.org/debian/pool/main/g/gdk-pixbuf/libgdk-pixbuf2.0-bin_2.42.10%2bdfsg-1%2bb1_amd64.deb  404  Not Found [IP: ---edited---]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

@jiangenj, does it work for you?

$ docker build -t gcr.io/syzkaller/env tools/docker/env
[+] Building 2736.5s (33/33) FINISHED                                                                                    docker:default
 => [internal] load build definition from Dockerfile                                                                               0.0s
 => => transferring dockerfile: 5.46kB                                                                                             0.0s
 => [internal] load metadata for docker.io/library/debian:bookworm                                                                 3.3s
 => [internal] load .dockerignore                                                                                                  0.1s
 => => transferring context: 2B                                                                                                    0.0s
 => [ 1/29] FROM docker.io/library/debian:bookworm@sha256:1dc55ed6871771d4df68d393ed08d1ed9361c577cfeb903cd684a182e8a3e3ae         8.3s
 => => resolve docker.io/library/debian:bookworm@sha256:1dc55ed6871771d4df68d393ed08d1ed9361c577cfeb903cd684a182e8a3e3ae           0.0s
 => => sha256:e9aef93137af6e967e7242f3b3c8ecd8e6f571d1e6fdd9e72db0befeeae3cf13 49.55MB / 49.55MB                                   6.4s
 => => sha256:1dc55ed6871771d4df68d393ed08d1ed9361c577cfeb903cd684a182e8a3e3ae 1.85kB / 1.85kB                                     0.0s
 => => sha256:820a611dc036cb57cee73613ce9da12530435b03007578153c04622e7f03f19b 529B / 529B                                         0.0s
 => => sha256:7b34f2fc561c06e26d69d7a5a58446e914c599479868abd6f5c5ed95770544a0 1.46kB / 1.46kB                                     0.0s
 => => extracting sha256:e9aef93137af6e967e7242f3b3c8ecd8e6f571d1e6fdd9e72db0befeeae3cf13                                          1.7s
 => [ 2/29] RUN apt-get update --allow-releaseinfo-change                                                                          4.6s
 => [ 3/29] RUN DEBIAN_FRONTEND=noninteractive apt-get install -y -q --no-install-recommends  sudo make nano unzip curl ca-cer  2187.4s
 => [ 4/29] RUN curl https://dl.google.com/go/go1.21.4.linux-amd64.tar.gz | tar -C /usr/local -xz                                  3.1s
 => [ 5/29] RUN mkdir -p /syzkaller/gopath/src/github.com/google/syzkaller &&  mkdir -p /syzkaller/.cache &&  chmod -R 0777 /syzk  0.3s
 => [ 6/29] RUN apt-get install -y -q gnupg software-properties-common apt-transport-https                                       135.8s
 => [ 7/29] RUN curl https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add -                                                    1.7s
 => [ 8/29] RUN add-apt-repository "deb http://apt.llvm.org/bookworm/ llvm-toolchain-bookworm-15 main"                             4.8s
 => [ 9/29] RUN apt-get update --allow-releaseinfo-change                                                                          1.9s
 => [10/29] RUN apt-get install -y -q --no-install-recommends clang-15 clang-format-15 clang-tidy-15                             133.9s
 => [11/29] RUN apt-get install -y -q --no-install-recommends flatbuffers-compiler                                                 3.4s
 => [12/29] RUN sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-15 100                                      0.4s
 => [13/29] RUN sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-15 100                                0.5s
 => [14/29] RUN sudo update-alternatives --install /usr/bin/clang-tidy clang-tidy /usr/bin/clang-tidy-15 100                       0.5s
 => [15/29] RUN sudo update-alternatives --install /usr/bin/clang-format clang-format /usr/bin/clang-format-15 100                 0.4s
 => [16/29] RUN apt autoremove -y -q                                                                                               1.1s
 => [17/29] RUN dpkg --add-architecture i386 &&  apt-get update --allow-releaseinfo-change &&  DEBIAN_FRONTEND=noninteractive ap  36.9s
 => [18/29] RUN curl https://storage.googleapis.com/syzkaller/fuchsia-toolchain.tar.gz | tar -C /syzkaller -xz                    24.1s
 => [19/29] RUN curl https://storage.googleapis.com/syzkaller/netbsd-toolchain.tar.gz | tar -C /syzkaller -xz                     20.1s
 => [20/29] RUN apt-get install -y -q libsqlite3-dev                                                                               6.3s
 => [21/29] RUN wget -O /tmp/Python-2.7.18.tgz 'https://www.python.org/ftp/python/2.7.18/Python-2.7.18.tgz'                        3.6ss
 => [22/29] RUN cd /tmp/ && tar -zxf Python-2.7.18.tgz                                                                             1.0s
 => [23/29] RUN cd /tmp/Python-2.7.18 && ./configure                                                                              14.0ss
 => [24/29] RUN cd /tmp/Python-2.7.18 && make -j2 && make altinstall                                                              73.5s
 => [25/29] RUN ln -s /usr/local/bin/python2.7 /usr/bin/python2                                                                    0.3s
 => [26/29] RUN curl https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-400.0.0-linux-x86_64.tar.gz | ta  4.7s
 => [27/29] RUN gcloud components install --quiet app-engine-python app-engine-go app-engine-python-extras cloud-datastore-emula  35.7s
 => [28/29] RUN chmod 0777 /usr/local/google-cloud-sdk                                                                             0.3s
 => [29/29] RUN echo "export PS1='syz-env🈴 '" > /syzkaller/.bashrc                                                                 0.4s
 => exporting to image                                                                                                            23.9s
 => => exporting layers                                                                                                           23.9s
 => => writing image sha256:93b8f0f30657d6c6b1856e4f421864ad58fa162282fe399cb2c797c04f84b607                                       0.0s
 => => naming to gcr.io/syzkaller/env                                                                                              0.0s
$ echo $?
0

I can run successfully, the error in your output generally means you are behind a proxy or not able to access deb.debian.org in docker.

[tarasmadan]

Thank you for the docker build attempt.
It downloads hundreds of packages and fails to download only 3.

[jiangenj]

install nodejs instead of npm to have less packages installed.

[a-nogikh]

To be honest, I'm not sure we really want to add a nodejs dependency to our Docker containers.

but it might failed during merge queue build.

If you are talking about the recent weeks, act is unfortunately not going to help prevent merge queue failures -- if the PR passed presubmit tests, but failed in the merge queue, it means that the tests themselves are bad/flaky. And we do have a few of such tests now :( We're trying to pinpoint the underlying problems now.

By the way, I'm curious why the CI tests are not automatically run on your PRs - I thought it's enough to have submitted one commit so that we don't have to manually approve CI runs anymore. And you have submitted quite a lot of commits already..

@tarasmadan do you know what else needs to be done?

[jiangenj]

https://docs.github.com/en/actions/managing-workflow-runs/reviewing-deployments

[jiangenj]

To be honest, I'm not sure we really want to add a nodejs dependency to our Docker containers.

but it might failed during merge queue build.

If you are talking about the recent weeks, act is unfortunately not going to help prevent merge queue failures -- if the PR passed presubmit tests, but failed in the merge queue, it means that the tests themselves are bad/flaky. And we do have a few of such tests now :( We're trying to pinpoint the underlying problems now.

By the way, I'm curious why the CI tests are not automatically run on your PRs - I thought it's enough to have submitted one commit so that we don't have to manually approve CI runs anymore. And you have submitted quite a lot of commits already..

@tarasmadan do you know what else needs to be done?

I think there is no harm to support act run, right?
It extends our ability to run workflow locally.

I think there is some approve setting in the project setting to allow ci only after approval? As stated in the link https://docs.github.com/en/actions/managing-workflow-runs/reviewing-deployments.

[tarasmadan]

@tarasmadan do you know what else needs to be done?
No idea.

P.S.
It was configured to require approval for all PRs, not only the first.

[a-nogikh]

Maybe install the tool in the Dockerfile (and in install_prerequisites) and in make act just run it instead of installing it?

I think it would be more in line with the other tests in the Makefile.

[jiangenj]

act will start docker from these ci.yml, so it needs to be installed on host.

[a-nogikh]

I see, thanks for the clarification