In this post, I try to help you to get along with certain harbor issues.
Harbor is an open source trusted cloud native registry project that stores, signs, and scans content. Harbor extends the open source Docker Distribution by adding the functionalities usually required by users such as security, identity and management. Having a registry closer to the build and run environment can improve the image transfer efficiency. Harbor supports replication of images between registries, and also offers advanced security features such as user management, access control and activity auditing.
Error Picture:
- GC or Jobs are not executed either manually or automatically
- Jobs are staying pending
- You’ll find an error like this in the logs:
ERROR: worker.fetch - LOADING Redis is loading the dataset in memory
Solution:
To resolve this you need to flush the redis
Exec into the redis container:
$ docker ps |grep -i redis
$ docker exec -it <container_id_or_name> bash
Execute the following commands to flush the Redis
redis-cli FLUSHDB
redis-cli -n DB_NUMBER FLUSHDB
redis-cli -n DB_NUMBER FLUSHDB ASYNC
redis-cli FLUSHALL
redis-cli FLUSHALL ASYNC
Error Picture:
- Replication can not be stopped/started and jobs are Pending
- The result is, you cant delete a failing replication job.
- You see “internal server error”
Solution:
Switch to the replication_execution
table
\d replication_execution;
2. Update the status
of every execution job to Succeed
update replication_execution set status = 'Succeed',total = '1', end_time = now()
3. Go to the UI and delete the former pending/failing replication job
Error Picture:
- Your certificate needs to be replaced because it is about to expire.
Solution:
- Replace the certificate + private key in your harbor installation folder
In my case this was/opt/harbor/ssl/xyz.crt.pem
and/opt/harbor/ssl/xyz.key.pem
- Replace both on the running instance of Harbor (when installing it, it copies those files into another directory) so do the following:
# Copy the certificate
$ cp /opt/harbor/ssl/xyz.crt.pem /var/harbor/data/secret/cert/server.crt# Copy the key
$ cp /opt/harbor/ssl/xyz.key.pem /var/harbor/data/secret/cert/server.key
3. Restart both nginx services
$ docker ps |grep goharbor |grep "nginx -g" | awk '{print $1}'
c1498j072f44
27f4lg4db7a6$ docker restart c1498j072f44 27f4lg4db7a6
4. Check that your certificate and key are renewed.
Error Picture: (Update 2021–03–18)
- The logs show the following error
ERROR: dead_pool_reaper.reap - MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk.
Solution
$ redis-cli config
127.0.0.1:6379> set stop-writes-on-bgsave-error no
Error Picture: (Update 2021–11–15)
- The Vulnerability database does not update
harbor.scanner-adapter/vulnerability-database-updated-at:05/7/20, 7:40 AM
Solution
Append the following lines to harbors docker-compose.yml
trivy-adapter:
[...]
volumes:
- type: bind
read_only: true
source: /var/run/docker.sock
target: /var/run/docker.sock
[...]
Then run docker-compuse up -d
and the trivy adapter should be recreated.
Then we need to manually trigger an update of the database using for example the debian:10
image.
$ docker exec -it trivy-adapter trivy image debian:10
2021-11-15T08:08:13.413Z INFO Need to update DB2021-11-15T08:08:13.413Z INFO Downloading DB...24.71 MiB / 24.71 MiB [--------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 11.25 MiB p/s 3s2021-11-15T08:08:18.599Z INFO Detecting Debian vulnerabilities...
Now when checking harbor again you should also see that the database is updated:
harbor.scanner-adapter/vulnerability-database-updated-at:11/15/21, 7:40 AM
Sources
- https://github.com/goharbor/harbor
- https://www.cyberciti.biz/faq/how-to-flush-redis-cache-and-delete-everything-using-the-cli/
- https://chartio.com/resources/tutorials/how-to-list-databases-and-tables-in-postgresql-using-psql/
- https://stackoverflow.com/questions/19581059/misconf-redis-is-configured-to-save-rdb-snapshots