Views: 4316
Clients concerned with security like financial institutions need to have their containers pulled from an internal Docker registry
Setup disconnected Docker registry for Rhosp
Pre-requisites:
On registry run:
$ yum update -y
3.2 Install the Offline registry and Tag the images
yum install -y docker-distribution docker yum-utils docker git wget git net-tools bind- bridge-utils bash-completion kexec-tools sos screen
For the registry itself – to use after images are pushed to the offline registry
$ systemctl enable docker
$ systemctl start docker
$ systemctl enable docker-distribution
$ systemctl start docker-distribution
Edit last line that has http address add the server domain name before the :5000
$ echo ‘INSECURE_REGISTRY=”–insecure-registry ol-rhel-disconnected-registry:5000″‘ >> /etc/sysconfig/docker
in /etc/containers/registries.conf add IP and port of insecure registries as well as registries. Remember to put apostrophes around inside the square brackets, as below. Replace ‘ol-rhel-disconnected-registry’ with YOURDOMAIN or IP.
# vi /etc/containers/registries.conf
# This is a system-wide configuration file used to
# keep track of registries for various container backends.
# It adheres to TOML format and does not support recursive
# lists of registries.
# The default location for this configuration file is /etc/containers/registries.conf.
# The only valid categories are: ‘registries.search’, ‘registries.insecure’,
# and ‘registries.block’.
[registries.search]
registries = [‘ol-rhel-disconnected-registry:5000′, ‘registry.access.redhat.com’, ‘docker.io’, ‘registry.fedoraproject.org’, ‘quay.io’, ‘registry.centos.org’]
# If you need to access insecure registries, add the registry’s fully-qualified name.
# An insecure registry is one that does not have a valid SSL certificate or only does HTTP.
[registries.insecure]
registries = [‘ol-rhel-disconnected-registry:5000’]
# If you need to block pull access from a registry, uncomment the section below
# and add the registries fully-qualified name.
#
# Docker only
[registries.block]
registries = []
The same file /etc/containers/registries.conf on the servers looks like:
$ cat /etc/containers/registries.conf
# This is a system-wide configuration file used to
# keep track of registries for various container backends.
# It adheres to TOML format and does not support recursive
# lists of registries.
# The default location for this configuration file is /etc/containers/registries.conf.
# The only valid categories are: ‘registries.search’, ‘registries.insecure’,
# and ‘registries.block’.
[registries.search]
registries = [‘ol-rhel-disconnected-registry:5000’]
# If you need to access insecure registries, add the registry’s fully-qualified name.
# An insecure registry is one that does not have a valid SSL certificate or only does HTTP.
[registries.insecure]
registries = [‘ol-rhel-disconnected-registry’]
# If you need to block pull access from a registry, uncomment the section below
# and add the registries fully-qualified name.
#
# Docker only
[registries.block]
registries = [‘registry.access.redhat.com’, ‘docker.io’, ‘registry.fedoraproject.org’, ‘quay.io’, ‘registry.centos.org’]
COPY it to the other servers
ansible others -m copy -a “src=/etc/containers/registries.conf dest=/etc/containers/registries.conf”
Restart Docker
# systemctl restart docker
# systemctl restart docker-distribution
On registry download, tag and push the images. This is simpler than openshift since all containers are standard tags
From
As in https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/architecture.html
For the offline server, which itself is completely online, Edit
/etc/containers/registries.conf to the following did allow docker pull to work
# cat /etc/containers/registries.conf
# This is a system-wide configuration file used to
# keep track of registries for various container backends.
# It adheres to TOML format and does not support recursive
# lists of registries.
# The default location for this configuration file is /etc/containers/registries.conf.
# The only valid categories are: ‘registries.search’, ‘registries.insecure’,
# and ‘registries.block’.
[registries.search]
registries = [‘ol-rhel-disconnected-registry:5000’, ‘registry.access.redhat.com’, ‘docker.io’, ‘registry.fedoraproject.org’, ‘quay.io’, ‘registry.centos.org’]
# If you need to access insecure registries, add the registry’s fully-qualified name.
# An insecure registry is one that does not have a valid SSL certificate or only does HTTP.
[registries.insecure]
registries = [‘ol-rhel-disconnected-registry:5000’]
# If you need to block pull access from a registry, uncomment the section below
# and add the registries fully-qualified name.
#
# Docker only
[registries.block]
registries = []
The same file /etc/containers/registries.conf on the servers/director should have the public servers removed from search and even moved to the block setting, so they wont be tried by default. The file looks like:
$ cat /etc/containers/registries.conf
# This is a system-wide configuration file used to
# keep track of registries for various container backends.
# It adheres to TOML format and does not support recursive
# lists of registries.
# The default location for this configuration file is /etc/containers/registries.conf.
# The only valid categories are: ‘registries.search’, ‘registries.insecure’,
# and ‘registries.block’.
[registries.search]
registries = [‘ol-rhel-disconnected-registry:5000’]
# If you need to access insecure registries, add the registry’s fully-qualified name.
# An insecure registry is one that does not have a valid SSL certificate or only does HTTP.
[registries.insecure]
registries = [‘ol-rhel-disconnected-registry’]
# If you need to block pull access from a registry, uncomment the section below
# and add the registries fully-qualified name.
#
# Docker only
[registries.block]
registries = [‘registry.access.redhat.com’, ‘docker.io’, ‘registry.fedoraproject.org’, ‘quay.io’, ‘registry.centos.org’]
COPY it to the other servers
ansible others -m copy -a “src=/etc/containers/registries.conf dest=/etc/containers/registries.conf”
Restart Docker
# systemctl restart docker
# systemctl restart docker-distribution
Since we did NOT do a complete install of same minor release with the director, I used a slightly different command to create the overcloud-images.yaml file
I ran the pull-tag-push script on the registry, then from the director I ran the openstack command to create overcloud-images.yaml
Here are the scripts that I ran:
From registry:
#!/bin/bash
#from:https://access.redhat.com/articles/3348761
if test -n “$STY”
then
printf “This is a screen session named ‘$STY’.\n”
ASSET_SERVER=172.28.117.56
docker search registry.access.redhat.com/rhosp13| awk ‘{print $2}’ | grep ^registry.access.redhat.com | while read IMAGE_NAME
do
TAG=13.0
echo “Pulling ${IMAGE_NAME}:${TAG} …”
docker pull ${IMAGE_NAME}:${TAG}
IMAGE_PATH=$(echo ${IMAGE_NAME} | cut -f2- -d/)
docker tag ${IMAGE_NAME}:${TAG} ${ASSET_SERVER}:5000/${IMAGE_PATH}:${TAG}
docker tag ${ASSET_SERVER}:5000/${IMAGE_PATH}:${TAG} ${ASSET_SERVER}:5000/${IMAGE_PATH}:latest
docker push ${ASSET_SERVER}:5000/${IMAGE_PATH}
done
else printf “This is NOT a screen session.\n”
exit 1
fi
to create the overcloud-images.yaml file
openstack overcloud container image prepare –namespace=172.28.117.56:5000/rhosp13 –prefix=openstack- –tag 13.0 –output-env-file=/home/stack/test-disconnect-overcloud_images.yaml
On a real director the script and command should be:
#!/bin/bash
#from:https://access.redhat.com/articles/3348761
if test -n “$STY”
then
printf “This is a screen session named ‘$STY’.\n”
ASSET_SERVER=172.28.117.56
docker search registry.access.redhat.com/rhosp13| awk ‘{print $2}’ | grep ^registry.access.redhat.com | while read IMAGE_NAME
do
TAG=$(openstack overcloud container image tag discover –image ${IMAGE_NAME} –tag-from-label {version}-{release})
echo “Pulling ${IMAGE_NAME}:${TAG} …”
docker pull ${IMAGE_NAME}:${TAG}
IMAGE_PATH=$(echo ${IMAGE_NAME} | cut -f2- -d/)
docker tag ${IMAGE_NAME}:${TAG} ${ASSET_SERVER}:5000/${IMAGE_PATH}:${TAG}
docker tag ${ASSET_SERVER}:5000/${IMAGE_PATH}:${TAG} ${ASSET_SERVER}:5000/${IMAGE_PATH}:latest
docker push ${ASSET_SERVER}:5000/${IMAGE_PATH}
done
else printf “This is NOT a screen session.\n”
exit 1
fi
The openstack command to build the overcloud-images should be
openstack overcloud container image prepare –namespace=172.28.117.56:5000/rhosp13 –prefix=openstack- –tag-from-label {version}-{release} –output-env-file=/home/stack/test-disconnect-overcloud_images.yaml
I am unsure whether DockerInsecureRegistryAddress needs to be manually added to overcloud-images.yaml
clinet
DockerInsecureRegistryAddress: [‘myregistry.local:8787’]
Troubleshooting
When building the overcloud-images.yaml file. I got an error
Not found image: docker://172.28.117.56:5000/rhosp13/openstack-gnocchi-statsd:latest
time=”2019-03-12T09:40:48+02:00″ level=fatal msg=”Error reading manifest latest in 172.28.117.56:5000/rhosp13/openstack-gnocchi-statsd: manifest unknown: manifest unknown”
to solve, I manually downloaded and tagged the image.
[root@rhel-disconnected-registry cloud-user]# docker tag registry.redhat.io/rhosp13/openstack-gnocchi-statsd:13.0 172.28.117.56:5000/rhosp13/openstack-gnocchi-statsd:latest
[root@rhel-disconnected-registry cloud-user]# docker tag registry.redhat.io/rhosp13/openstack-gnocchi-statsd:13.0 172.28.117.56:5000/rhosp13/openstack-gnocchi-statsd:13.0
[root@rhel-disconnected-registry cloud-user]# docker push 172.28.117.56:5000/rhosp13/openstack-gnocchi-statsd:latest
[root@rhel-disconnected-registry cloud-user]# docker push 172.28.117.56:5000/rhosp13/openstack-gnocchi-statsd:13.0
When building the overcloud-images.yaml file, If you get the following error, you will need to either label each image as told by error or change the tag variable to 13.0 as in the first script :
Image 172.28.117.56:5000/rhosp13/openstack-aodh-notifier has no tag 13.0-66.
Available tags: 13.0, latest
For monitoring you will need to add to the overcloud-image.yaml creation:
-e /usr/share/openstack-tripleo-heat-templates/environments/services-docker/fluentd-client.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/services-docker/sensu-client.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/services-docker/collectd.yaml \
–output-env-file /home/stack/templates/overcloud_images.yaml