Docker registry
Tomas
Tomas
Container registries are used for hosting containerized application images. It's highly desirable to run container registry locally for speed, cost and privacy reasons. K3s kubernetes cluster setup earlier is used to host docker container registry, with fedora 32 machine used for management.
Setting up manager.k3s.local
manager.k3s.local is setup using fedora 32 base image, /etc/rancher/k3s/k3s.yaml copied from master.k3s.local to manager.k3s.local:~/.kube/config. Server value in config is changed from https://127.0.0.1:6443 to https://master.k3s.local:6443:
$ mkdir ~/.kube
$ scp -oStrictHostKeyChecking=no master:/etc/rancher/k3s/k3s.yaml ~/.kube/config
k3s.yaml 100% 1052 789.2KB/s 00:00
$ sed -i 's/server:.*/server: https:\/\/master.k3s.local:6443/g' ~/.kube/config
$ grep server: ~/.kube/config
server: https://master.k3s.local:6443
After which, k3s is downloaded as /usr/local/bin/kubectl on manager.k3s.local:
$ sudo wget -q https://github.com/rancher/k3s/releases/download/v1.18.3+k3s1/k3s -O /usr/local/bin/kubectl
$ sudo chmod +x /usr/local/bin/kubectl
$ kubectl get node
INFO[0000] Preparing data dir /home/ansible/.rancher/k3s/data/31f16867327c45f309297f2846b7f471edba2b97c394cddb49d4e70ff7d41bcd
NAME STATUS ROLES AGE VERSION
master Ready master 16m v1.18.3+k3s1
Setting up helm
Helm is kubernetes package manager. It allows deploying various applications from charts which describe how given application is to be setup in kubernetes. Since v3 helm installation has been simplified namely be removing dependency on tiller. Helm can be installed using:
$ curl -sfL https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | sh -
Downloading https://get.helm.sh/helm-v3.2.4-linux-amd64.tar.gz
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm
Once helm is installed, add stable chart repository:
$ helm repo add stable https://kubernetes-charts.storage.googleapis.com/
"stable" has been added to your repositories
Installing docker registry
Docker registry installed using helm to kube-system namespace with data persistence class set to local-path:
$ helm install registry stable/docker-registry \
> --namespace kube-system \
> --set=persistence.enabled=true,persistence.storageClass=local-path
NAME: registry
LAST DEPLOYED: Fri Jun 19 02:12:38 2020
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace kube-system -l "app=docker-registry,release=registry" -o jsonpath="{.items[0].metadata.name}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl -n kube-system port-forward $POD_NAME 8080:5000
Setting up ssl certs
To enable TLS on traefik load balancer, create self signed SSL key and cert:
$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
> -keyout apps.local.key \
> -out apps.local.crt \
> -subj "/CN=*.apps.local"
Generating a RSA private key
...................................................................+++++
................+++++
writing new private key to 'apps.local.key'
-----
Add newly created SSL key and cert to kube-system namespace:
$ kubectl create secret tls traefik-tls-cert \
> --namespace=kube-system \
> --key=apps.local.key \
> --cert=apps.local.crt
secret/traefik-tls-cert created
Remove the key from the filesystem and add certificate to system ca-trust bundle:
$ rm apps.local.key
$ sudo mv apps.local.crt /etc/pki/ca-trust/source/anchors/apps.local.crt
$ sudo update-ca-trust
Configuring traefik ingress controller
To access registry-docker-registry kubernetes service, ingress controller is configured to route http/https requests using traefik-tls-cert and matching hostname docker-registry.apps.local port 5000
$ cat /tmp/ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: docker-registry
namespace: kube-system
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: docker-registry.apps.local
http:
paths:
- backend:
serviceName: registry-docker-registry
servicePort: 5000
tls:
- secretName: traefik-tls-cert
$ kubectl apply -f /tmp/ingress.yaml
ingress.extensions/docker-registry created
Test out new service using curl:
$ curl -iL https://docker-registry.apps.local/v2/ -iL
HTTP/2 200
content-type: application/json; charset=utf-8
date: Fri, 19 Jun 2020 02:14:01 GMT
docker-distribution-api-version: registry/2.0
vary: Accept-Encoding
x-content-type-options: nosniff
content-length: 2
{}
Install docker
Docker package can be used for managing docker images locally, use dnf to install docker on fedora:
$ sudo dnf install docker -y
Last metadata expiration check: 0:19:04 ago on Fri 19 Jun 2020 01:55:06 AM UTC.
Dependencies resolved.
===============================================================================
Package Arch Version Repo Size
===============================================================================
Installing:
moby-engine x86_64 19.03.8-2.ce.gitafacb8b.fc32 updates 51 M
Installing dependencies:
container-selinux noarch 2:2.135.0-1.fc32 updates 47 k
containerd x86_64 1.3.3-1.fc32 updates 32 M
libbsd x86_64 0.10.0-2.fc32 fedora 106 k
libnet x86_64 1.1.6-19.fc32 fedora 64 k
protobuf-c x86_64 1.3.2-2.fc32 fedora 35 k
runc x86_64 2:1.0.0-144.dev.gite6555cc.fc32 fedora 2.7 M
Installing weak dependencies:
criu x86_64 3.14-1.fc32 updates 507 k
Transaction Summary
===============================================================================
Install 8 Packages
Total size: 86 M
Installed size: 314 M
Downloading Packages:
[SKIPPED] container-selinux-2.135.0-1.fc32.noarch.rpm: Already downloaded
[SKIPPED] containerd-1.3.3-1.fc32.x86_64.rpm: Already downloaded
[SKIPPED] criu-3.14-1.fc32.x86_64.rpm: Already downloaded
[SKIPPED] moby-engine-19.03.8-2.ce.gitafacb8b.fc32.x86_64.rpm: Already downloaded
[SKIPPED] libbsd-0.10.0-2.fc32.x86_64.rpm: Already downloaded
[SKIPPED] libnet-1.1.6-19.fc32.x86_64.rpm: Already downloaded
[SKIPPED] protobuf-c-1.3.2-2.fc32.x86_64.rpm: Already downloaded
[SKIPPED] runc-1.0.0-144.dev.gite6555cc.fc32.x86_64.rpm: Already downloaded
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Running scriptlet: container-selinux-2:2.135.0-1.fc32.noarch 1/8
Installing : container-selinux-2:2.135.0-1.fc32.noarch 1/8
Running scriptlet: container-selinux-2:2.135.0-1.fc32.noarch 1/8
Installing : protobuf-c-1.3.2-2.fc32.x86_64 2/8
Installing : libnet-1.1.6-19.fc32.x86_64 3/8
Installing : libbsd-0.10.0-2.fc32.x86_64 4/8
Installing : criu-3.14-1.fc32.x86_64 5/8
Installing : runc-2:1.0.0-144.dev.gite6555cc.fc32.x86_64 6/8
Installing : containerd-1.3.3-1.fc32.x86_64 7/8
Running scriptlet: containerd-1.3.3-1.fc32.x86_64 7/8
Running scriptlet: moby-engine-19.03.8-2.ce.gitafacb8b.fc32.x86_64 8/8
Installing : moby-engine-19.03.8-2.ce.gitafacb8b.fc32.x86_64 8/8
Running scriptlet: moby-engine-19.03.8-2.ce.gitafacb8b.fc32.x86_64 8/8
Created symlink /etc/systemd/system/sockets.target.wants/docker.socket → /usr/lib/systemd/system/docker.socket.
Running scriptlet: container-selinux-2:2.135.0-1.fc32.noarch 8/8
Running scriptlet: moby-engine-19.03.8-2.ce.gitafacb8b.fc32.x86_64 8/8
Verifying : container-selinux-2:2.135.0-1.fc32.noarch 1/8
Verifying : containerd-1.3.3-1.fc32.x86_64 2/8
Verifying : criu-3.14-1.fc32.x86_64 3/8
Verifying : moby-engine-19.03.8-2.ce.gitafacb8b.fc32.x86_64 4/8
Verifying : libbsd-0.10.0-2.fc32.x86_64 5/8
Verifying : libnet-1.1.6-19.fc32.x86_64 6/8
Verifying : protobuf-c-1.3.2-2.fc32.x86_64 7/8
Verifying : runc-2:1.0.0-144.dev.gite6555cc.fc32.x86_64 8/8
Installed:
container-selinux-2:2.135.0-1.fc32.noarch
containerd-1.3.3-1.fc32.x86_64
criu-3.14-1.fc32.x86_64
libbsd-0.10.0-2.fc32.x86_64
libnet-1.1.6-19.fc32.x86_64
moby-engine-19.03.8-2.ce.gitafacb8b.fc32.x86_64
protobuf-c-1.3.2-2.fc32.x86_64
runc-2:1.0.0-144.dev.gite6555cc.fc32.x86_64
Complete!
$ sudo systemctl start docker
Using local registry
Docker image can be created locally or downloaded from docker hub:
$ sudo docker pull alpine
Using default tag: latest
latest: Pulling from library/alpine
df20fa9351a1: Pull complete
Digest: sha256:185518070891758909c9f839cf4ca393ee977ac378609f700f60a771a2dfe321
Status: Downloaded newer image for alpine:latest
docker.io/library/alpine:latest
Once image is on the local system, tag it with registry prefix:
$ sudo docker tag alpine docker-registry.apps.local/demo-app
Tagged image can now be pushed up to the registry:
$ sudo docker push docker-registry.apps.local/demo-app
The push refers to repository [docker-registry.apps.local/demo-app]
50644c29ef5a: Pushed
latest: digest: sha256:a15790640a6690aa1730c38cf0a440e2aa44aaca9b0e8931a9f2b0d7cc90fd65 size: 528
To download the image from the new registry, the local images needs to be deleted first:
$ sudo docker rmi alpine docker-registry.apps.local/demo-app
sudo Untagged: alpine:latest
Untagged: alpine@sha256:185518070891758909c9f839cf4ca393ee977ac378609f700f60a771a2dfe321
dUntagged: docker-registry.apps.local/demo-app:latest
Untagged: docker-registry.apps.local/demo-app@sha256:a15790640a6690aa1730c38cf0a440e2aa44aaca9b0e8931a9f2b0d7cc90fd65
Deleted: sha256:a24bb4013296f61e89ba57005a7b3e52274d8edd3ae2077d04395f806b63d83e
Deleted: sha256:50644c29ef5a27c9a40c393a73ece2479de78325cae7d762ef3cdc19bf42dd0a
$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
Docker can now pull image from the docker-registry.apps.local:
$ sudo docker pull docker-registry.apps.local/demo-app
Using default tag: latest
latest: Pulling from demo-app
df20fa9351a1: Pull complete
Digest: sha256:a15790640a6690aa1730c38cf0a440e2aa44aaca9b0e8931a9f2b0d7cc90fd65
Status: Downloaded newer image for docker-registry.apps.local/demo-app:latest
docker-registry.apps.local/demo-app:latest
$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker-registry.apps.local/demo-app latest a24bb4013296 2 weeks ago 5.57MB