Containers don't consume Acorn image, but plain container image

[josedev-union]

This issue appeared in the following cases.

• When running the Acorn app from its local Directory, the initial replicas all consume the correct Acorn image. Once it is scaled out, new containers consume the plain container image.
• When running the Acorn app from an image, only the first container consumes the Acorn image and others don't even at the initial deployment.

[cjellick]

Can you provide some yaml output of the k8s resources in question to illustrate what you mean?

[cjellick]

Hi again. Is there a difference between this issue and the other one you opened: #704?

[josedev-union]

Hi again. Is there a difference between this issue and the other one you opened: #704?

Those are same, sorry, i forgot i did open already.

[josedev-union]

Can you provide some yaml output of the k8s resources in question to illustrate what you mean?

This is the pod spec of the replica created at the initial app deployment.

    spec:
      containers:
      - args:
        - /acorn/scripts/start.sh
        image: 127.0.0.1:30484/acorn/acorn@sha256:ae591017c90b7afd90b8f4261250f3778ce91f60c4349f59480fe666695a8e19
        imagePullPolicy: IfNotPresent
        name: mongodb-3
      imagePullSecrets:
      - name: mongodb-0-pull-d27dc58d-fd5

This is the pod spec of the replica scaled out.

    spec:
      containers:
      - args:
        - /acorn/scripts/start.sh
        image: mongo:5.0.12
        imagePullPolicy: IfNotPresent
        name: mongodb-3
      imagePullSecrets:
      - name: mongodb-3-pull-d27dc58d-fd5

You can find the Acornfile for this app here https://github.com/josedev-union/acorn-library/blob/add-mongodb/mongodb/Acornfile

[cjellick]

Oh that is strange. It should be consistent. This is particularly problematic because the second one is potentially going to reach out to docker hub, I think.

Will get this relatively high on the priority list

[cjellick]

@iwilltry42 this one could be complicated, I don't know. Can you take a stab at it? If you can't figure it out, we can punt to @ibuildthecloud

[iwilltry42]

Simplified example:

// simplified/Acornfile
args: {
  replicas: 1
}

containers: [string]: {
  image: "nginx:latest"
}

if args.replicas == 1 {
  containers: {
    "nginx-0": {}
  }
}
      
      
if args.replicas > 1 {
  for i in std.range(args.replicas) {
    containers: {
      "nginx-\(i)": {}
    }
  }
}

Run

$ acorn run simplified                            
[+] Building 0.7s (5/5) FINISHED                                                                                               
 => [internal] load build definition from acorn-dockerfile-4289540147                                                     0.0s
 => => transferring dockerfile: 72B                                                                                       0.0s
 => [internal] load .dockerignore                                                                                         0.0s
 => => transferring context: 2B                                                                                           0.0s
 => [internal] load metadata for docker.io/library/nginx:latest                                                           0.5s
 => CACHED [1/1] FROM docker.io/library/nginx:latest@sha256:0b970013351304af46f322da1263516b188318682b2ab109186249759118  0.0s
 => => resolve docker.io/library/nginx:latest@sha256:0b970013351304af46f322da1263516b188318682b2ab1091862497591189ff1     0.0s
 => exporting to image                                                                                                    0.1s
 => => exporting layers                                                                                                   0.0s
 => => exporting manifest sha256:e95cea7d0214dfe8fea503b0e6169df0be846e350e9d764c7b446a8eb3a82239                         0.0s
 => => exporting config sha256:64411774c0dc82246506062a741c22705ed436ab455174c4c2aa035eefa9c56b                           0.0s
 => => pushing layers                                                                                                     0.0s
 => => pushing manifest for 127.0.0.1:5000/acorn/acorn:latest@sha256:e95cea7d0214dfe8fea503b0e6169df0be846e350e9d764c7b4  0.0s
[+] Building 0.4s (5/5) FINISHED                                                                                               
 => [internal] load .dockerignore                                                                                         0.0s
 => => transferring context: 64B                                                                                          0.0s
 => [internal] load build definition from Dockerfile                                                                      0.1s
 => => transferring dockerfile: 58B                                                                                       0.0s
 => [internal] load build context                                                                                         0.0s
 => => transferring context: 456B                                                                                         0.0s
 => [1/1] COPY . /                                                                                                        0.0s
 => exporting to image                                                                                                    0.3s
 => => exporting layers                                                                                                   0.1s
 => => exporting manifest sha256:5621b3ac08dd47e72549379b3a59ab70305ef63cfa61c598c91f3f88e9232e22                         0.0s
 => => exporting config sha256:b44065ba80704d5d88d0497b8cf6e1fb8f87056506e5ecb8d59184fad8b77c45                           0.0s
 => => pushing layers                                                                                                     0.1s
 => => pushing manifest for 127.0.0.1:5000/acorn/acorn:latest@sha256:5621b3ac08dd47e72549379b3a59ab70305ef63cfa61c598c91  0.0s
old-shape
STATUS: ENDPOINTS[] HEALTHY[] UPTODATE[] 
STATUS: ENDPOINTS[] HEALTHY[0] UPTODATE[0] pending
STATUS: ENDPOINTS[] HEALTHY[0/1] UPTODATE[1] [containers: nginx-0 ContainerCreating; old-shape is not ready]

$ kgp -n old-shape-1bdbd190-260 nginx-0-5f5d76cf7-nrhpw -o go-template='{{(index .spec.containers 0).image}}'

127.0.0.1:31035/acorn/acorn@sha256:a221f578f392d625faefbdf62b76596552656be4884b958590cba460afe78bc9%                           

$ acorn update old-shape simplified --replicas=2  
old-shape

$ kgp -n old-shape-1bdbd190-260                                                                              

NAME                       READY   STATUS    RESTARTS   AGE
nginx-0-5f5d76cf7-nrhpw    1/1     Running   0          2m46s
nginx-1-744b9bfc54-65vd9   1/1     Running   0          7s

$ kgp -n old-shape-1bdbd190-260 nginx-1-744b9bfc54-65vd9  -o go-template='{{(index .spec.containers 0).image}}'

nginx:latest

Investigation

When doing acorn update, the command does not have a build reference as acorn run has, resulting in the following app definition:

# ...
  App Spec:
    Containers:
      nginx-0:
        Build:
          Base Image:  nginx:latest
          Context:     .
          Dockerfile:  Dockerfile
        Image:         sha256:a221f578f392d625faefbdf62b76596552656be4884b958590cba460afe78bc9
        Permissions:
        Probes:  <nil>
      nginx-1:
        Image:  nginx:latest
        Permissions:
        Probes:  <nil>
# ...

[josedev-union]

https://github.com/acorn-io/acorn/blob/1339ce43d627941c845c49d0fead32aca610f856/pkg/controller/appdefinition/deploy.go#L402
https://github.com/acorn-io/acorn/blob/1339ce43d627941c845c49d0fead32aca610f856/pkg/controller/appdefinition/deploy.go#L301-L306

Is there any usecase to use container.Image(internal.acorn.io/v1) as the image of corev1.Container directly?

[cjellick]

@iwilltry42 what's the next steps? Is this fixable? do you need more input on correct behavior from @ibuildthecloud ?

[iwilltry42]

Hey, sorry for the late reply.
@cjellick it's definitely fixable. We'll need a 3-way mapping of the involved images

flowchart LR
  A[Image Name in Acornfile, e.g. nginx:latest] <--> B[Image SHA at build-time, e.g. sha256:2f7...];
  B <--> C[Acorn Image SHA, e.g. sha256:5a52...]

Loading

Questionable could be if we want to re-pull nginx:latest on update, effectively invalidating the whole mapping and requiring a restart of the running container?

[iwilltry42]

Update: The fix for this is still WIP as it introduces a breaking change to the way we store image information.

[cjellick]

I'm pushing this out to v0.5.0. If you happen to get it done in time for v0.4.1, great