Image registries are areas used for storing and distributing container images. When we create a container image, we use private or public image registries to use it in a cluster environment or to distribute the container image we have created. It is very important to pay attention to the images we use in the cluster for cluster security. In addition, we must ensure that the images we use in the cluster come from trusted sources.
We need to dwell on the concepts of public or private image registries a bit. When creating a container image, a base image is usually used in the Dockerfile, if an image is not created from scratch. The base image used is often obtained from public image registry sources such as Redhat Catalog, Dockerhub, gcr.io.
After creating the container image, we upload it to an image registry used within the organization to deploy it to the cluster later. There are several container image registry options (ECR, ACR, Dockerhub private repository, Nexus repository).
In the organizational world, the process generally proceeds in this way. You should keep your container images, whose development is completed, in private registries, and only certain people should have access to these registries.
We need to ask two questions to create a secure environment:
1-) Are the images used as the base image downloaded from reliable sources?
2-) Are the images used in the cluster downloaded from trusted sources?
Downloading base images from trusted sources
In the Dockerfile we use to create container images, we specify where to download the base image we will use. When obtaining base images from a public registry, we need to use well-known official sources because the base image is actually an image prepared by someone else, and we need to make sure that there is no malicious software inside the image. Also, when a security vulnerability of a package used in the base image is released, the registry we obtained the image from should quickly fix the relevant security vulnerability. For these reasons, when using a base in the application, the image should be downloaded from a secure and reliable source that fixes vulnerabilities as soon as possible. You can use public image registries such as Redhat Catalog, gcr.io, Dockerhub. However, you need to be extra careful when using Dockerhub. As you know, Dockerhub is a public image registry, anyone can upload and distribute their own images there. When using Dockerhub, it is necessary to make sure that the repositories we use are managed by official sources.
You can ensure that base images are only downloaded from trusted sources determined by the DevSecOps team with additional configurations. For example, if you are using an internal private registry within the organization, you can define trusted sources as a proxy and ensure that developers only download images from these sources.
Using trusted images in the cluster environment
Using images from trusted sources is a critical issue for cluster security in a cluster environment. Since container images are packaged products, they contain a lot of components.
Just like we do not allow an unknown application or process to run on our server in a traditional structure, we should not allow container images downloaded from unknown sources to run on the cluster. After all, containers are just processes.
When defining manifests such as pod, deployment, daemonset, we specify where the container image is downloaded from.
As you can see in spec.containers.[*].image, the image that the pod will use is downloaded from the dokcer.io container image registry.
So, can the person defining these manifests write any address in this area and enable it to be used in the pod?
The answer is yes if there is no control and the worker node has internet access.
Manifests such as pod, deployment, and daemonset can be executed by software teams. Software teams may not have information about whether the source from which the image will be downloaded is secure or not.
In such cases, some strict rules must be established. As an organizational policy, secure image registries should be identified, and container image usage should only be allowed from specific image registries.
Opa Gatekeeper can be used to ensure that images are only downloaded from trusted sources.
To set limited rules with Opa Gatekeeper, you need to create a policy, but first, you need to install it in the cluster.
Deploy OPA Gatekeeper to the cluster.
kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/release-3.5/deploy/gatekeeper.yaml
Check the status of the OPA Gatekeeper Pods and wait for them all to come up fully.
kubectl get pods -n gatekeeper-system -w
After completing the OPA Gatekeeper installation, you can create a policy by defining constraint templates.
First, we need to create a constraint template as follows:
Let’s create a file named k8s-allowed-repos.yaml.
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
name: k8sallowedrepos
spec:
crd:
spec:
names:
kind: K8sAllowedRepos
validation:
# Schema for the `parameters` field
openAPIV3Schema:
properties:
repos:
type: array
items:
type: string
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package k8sallowedrepos
violation[{"msg": msg}] {
container := input.review.object.spec.containers[_]
satisfied := [good | repo = input.parameters.repos[_] ; good = startswith(container.image,
repo)]
not any(satisfied)
msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v",
[container.name, container.image, input.parameters.repos])
}
violation[{"msg": msg}] {
container := input.review.object.spec.initContainers[_]
satisfied := [good | repo = input.parameters.repos[_] ; good = startswith(container.image,
repo)]
not any(satisfied)
msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v",
[container.name, container.image, input.parameters.repos])
}kubectl create -f k8sallowedrepos.yml
After applying the constraint template to the cluster, as the second step, we need to apply the following resource to specify the list of repositories we will whitelist. In this example, we only whitelist dockerhub under spec.parameters.repos[*], but you can add different registries according to your needs.
Create a file named whitelist-registries.yaml and apply it to the cluster as follows:
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sAllowedRepos
metadata:
name: whitelist-registry
spec:
match:
kinds:
- apiGroups: [""]
kinds: ["Pod"]
parameters:
repos:
- "docker.io"By applying the policy to the cluster, we guarantee that only images are downloaded from the docker.io container registry.
As a test scenario, let’s try to apply a pod that uses two different images to the cluster and see that the pod that does not comply with the policy is not created in the cluster.
apiVersion: v1
kind: Pod
metadata:
name: allowed-busybox
spec:
restartPolicy: Never
containers:
- name: busybox
image: docker.io/library/busybox
command: ['sh', '-c', 'sleep 3600']kubectl create -f allowed-busybox.yml
You can check that the pod is successfully created in the cluster with the following command:
kubectl get pods
In the other scenario, let’s try to apply another pod that does not comply with the policy to the cluster. As you can see below, this pod uses a container registry other than docker.io, and the policy should not allow this pod to be created.
apiVersion: v1
kind: Pod
metadata:
name: gcr-busybox
spec:
restartPolicy: Never
containers:
- name: busybox
image: gcr.io/google-containers/busybox
command: ['sh', '-c', 'sleep 3600']When we try to create the pod with the command kubectl create -f disallowed-busybox.yml, OPA Gatekeeper will not allow the pod to be created according to the policy applied to the cluster.