upgrade from 2.8.4 to 2.9.0 or 2.9.1 - containers in restarting state

[inzagod]

Hi,
we wanted to upgrade from 2.8.4 to 2.9 and some containers still in restarting state :
-core
-jobservice
-db
-registryctl
-redis
no logs in journalctl or in /var/log/harbor !!!
the last log is the shutdown info
i never found any logs for them, so strange

our docker version is :
[root@ocphrb01 ~]# docker version
Client: Docker Engine - Community
Version: 19.03.3
API version: 1.40
Go version: go1.12.10
Git commit: a872fc2f86
Built: Tue Oct 8 00:58:10 2019
OS/Arch: linux/amd64
Experimental: false

Server: Docker Engine - Community
Engine:
Version: 19.03.3
API version: 1.40 (minimum version 1.12)
Go version: go1.12.10
Git commit: a872fc2f86
Built: Tue Oct 8 00:56:46 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.10
GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339
runc:
Version: 1.0.0-rc8+dev
GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
docker-init:
Version: 0.18.0
GitCommit: fec3683
[root@ocphrb01 ~]#

our os is RHEL 7.9

after investigating and saw a docker version issue (in the 2.9 doc, docker version seems to be ok : https://goharbor.io/docs/2.9.0/install-config/installation-prereqs/)
we upgraded so our docker version in the latest 20.10 --> core and another one (i don't remember) still in restarting state !
the other containers were up so upgraded the docker version has fixed some containers !
we upgraded with the latest 23 version --> same issue
we upgraded with the latest 24 version --> same issue

we saw in postgresql log : FATAL database registry does not exist !

now, we still in 2.8.4 version because 2.9.x doesn't work for us

is anyone can help up ?

thanks a lot

ludovic

[stonezdj]

First please check the database container's state and log. the docker version should be 20.10.10 or higher, see #19141.

[inzagod]

hi,
we will test next week no problem but as i said in my last post, we upgraded docker in latest version 20/23 and 24

thanks a lot

[inzagod]

hi,

we tried to update from 2.8.4 to 2.9.1 this morning
first, we upgraded docker-ce and docker-ce-cli in version 20.10.24 (the latest 20.10)
after, we upgraded harbor stack in 2.9.1 version and it worked fine ....! all the containers are ready (jobservice restart several times but at the end it is ready)

the log in case of it helps you for the future :

[root@ocphrb01 harbor]# tail -f /var/log/harbor/jobservice.log
Dec 12 09:55:41 172.19.0.1 jobservice[4882]: 2023-12-12T08:55:41Z [INFO] [/pkg/reg/adapter/jfrog/adapter.go:42]: the factory of jfrog artifactory adapter was registered
Dec 12 09:55:41 172.19.0.1 jobservice[4882]: 2023-12-12T08:55:41Z [INFO] [/pkg/reg/adapter/quay/adapter.go:53]: the factory of Quay adapter was registered
Dec 12 09:55:41 172.19.0.1 jobservice[4882]: 2023-12-12T08:55:41Z [INFO] [/pkg/reg/adapter/tencentcr/adapter.go:55]: the factory for adapter tencent-tcr registered
Dec 12 09:55:41 172.19.0.1 jobservice[4882]: 2023-12-12T08:55:41Z [INFO] [/pkg/reg/adapter/harbor/adaper.go:31]: the factory for adapter harbor registered
Dec 12 09:55:41 172.19.0.1 jobservice[4882]: 2023-12-12T08:55:41Z [INFO] [/pkg/config/rest/rest.go:47]: get configuration from url: http://core:8080/api/v2.0/internalconfig
Dec 12 09:55:41 172.19.0.1 jobservice[4882]: 2023-12-12T08:55:41Z [ERROR] [/pkg/config/rest/rest.go:50]: Failed on load rest config err:Get "http://core:8080/api/v2.0/internalconfig": dial tcp 172.19.0.9:8080: connect: connection refused, url:http://core:8080/api/v2.0/internalconfig
Dec 12 09:55:41 172.19.0.1 jobservice[4882]: panic: failed to load configuration, error: failed to load rest config
Dec 12 09:55:41 172.19.0.1 jobservice[4882]: goroutine 1 [running]:
Dec 12 09:55:41 172.19.0.1 jobservice[4882]: main.main()
Dec 12 09:55:41 172.19.0.1 jobservice[4882]: #011/harbor/src/jobservice/main.go:44 +0x3ae

so you can close this issue.

i don't know why but upgrading docker first seems to resolve the issue.

thanks a lot for your help

ludo

[github-actions]

This issue is being marked stale due to a period of inactivity. If this issue is still relevant, please comment or remove the stale label. Otherwise, this issue will close in 30 days.