Skip to content

Docker Registry

SCDF support container usage on various SCDF computing nodes. A docker registry was setup for users to store and use docker images . SCDF users should be able to pull/push images there from SCDF networks and BNL campus network.

A pull through cache registry is also available for users to access public dockerhub images from SCDF computer nodes.

Pull through cache registry for docker hub: (public image only)

The pull through cache registry hostname is dockerhub.sdcc.bnl.gov

The URL for pull through cache registry is dockerhub.sdcc.bnl.gov/namespace/image.tag

If people normally use 

docker pull
image:tag
docker pull
namespace:tag
singularity pull
docker://namespace/image:tag

The images then can be accessed via:

docker pull dockerhub.sdcc.bnl.gov/library/image:tag
docker pull dockerhub.sdcc.bnl.gov/namespace/image:tag
singularity pull
docker://dockerhub.sdcc.bnl.gov/library/image:tag

docker default to use "library" as namespace.

examples:

docker pull
dockerhub.sdcc.bnl.gov/library/centos7:latest
docker pull
dockerhub.sdcc.bnl.gov/tensorflow/tensorflow:latest-gpu-jupyter
singularity pull docker://dockerhub.sdcc.bnl.gov/library/ubuntu:lastest
singularity pull docker://dockerhub.sdcc.bnl.gov/tensorflow/tensorflow:latest

Private registry ( SCDF account needed)

You can use your campus workstation , laptop to push docker images in this private registry.

Most of SCDF servers are not docker enable for users due to security reason.

The images stored in the registry can be accessed from computers on BNL campus network and SCDF network.

The repo management for the registry is at https://registry.sdcc.bnl.gov

You can login with SCDF account from BNL campus and SCDF network.

After login you can create teams , add member to teams, create namespace , set access visibility of namespace.

User can create teams , team owner can add team member of different roles. Team owner can add namespace and set access property(visibility) to be private(need authentication) or public.

By default every user has a private namespace same as your userID.

We strongly recommend creating an application token so when you push/pull images, you only use token instead of SCDF password as docker command save password in clear text by default on your workstation. You can create application token through profile by click the icon or your user name on top right corner. Then in middle section you can click "create" button to create up to 5 application tokens. We suggest have your screen capture tool ready as the new created token only appear 5 seconds and there is no way to retrieve later. However you can always create more and delete if you missed.

The URL for the repo image: registry.scc.bnl.gov/namespace/image:tag

To push an image to the repo you need first use "docker login registry.sdcc.bnl.gov". We suggest use application token instead of SCDF password or configure credential helper.

example: 

$ docker login -u zhihua registry.sdcc.bnl.gov
Password:
WARNING! Your password will be stored unencrypted in /home/zdong/.docker/config.json.
Configure a credential helper to remove this warning. See
<https://docs.docker.com/engine/reference/commandline/login/#credentials-store>
Login Succeeded
$ docker tag 42b97d3c2ae9
registry.sdcc.bnl.gov/zhihua/bzbox:latest
$ docker push registry.sdcc.bnl.gov/zhihua/bzbox:latest
The push refers to repository [registry.sdcc.bnl.gov/zhihua/bzbox]
0fd05bf2930d: Pushed
latest: digest: sha256:b862520da7361ea093806d292ce355188ae83f21e8e3b2a3ce4dbdba0a230f83 size: 527

To use your stored image on SCDF or campus computer:

#if the namespace is private need docker login
docker
login registry.sdcc.bnl.gov
docker pull
registry.sdcc.bnl.gov/namespace/image.tag
singularity pull --docker-login docker://registry.sdcc.bnl.gov/namespace/image:tag
#optionally singularity can set Environment Variable to avoid be prompted for password
export SINGULARITY_DOCKER_USERNAME=YourUserID
export SINGULARITY_DOCKER_PASSWORD=YourTokenOrPasswd
singularity run
docker://registry.sdcc.bnl.gov/namespace/image:tag

Example singularity use on Institutional cluster:

[zdong@icsubmit01 ]$ export SINGULARITY_DOCKER_USERNAME=zhihua
[zdong@icsubmit01 ]$ export SINGULARITY_DOCKER_PASSWORD=`cat ~/p/k1.txt`
[zdong@icsubmit01 ]$
srun -N1 -n1 -t 5:00 -p debug --gres=gpu:2 --pty singularity run --nv
docker://registry.sdcc.bnl.gov/zhihua/my-tf1
INFO:
Using cached SIF image
________
_______________
___
__/__________________________________
____/__
/________
__
__
/
_
_ \\_
__ \\_
___/
__ \\_
___/_
/_
__
/_
__ \\_ | /| / /
_
/
/
__/
/ / /(__
)/ /_/ /
/
_
__/
_
/ / /_/ /_ |/ |/ /
/_/
\\___//_/ /_//____/ \\____//_/
/_/
/_/
\\____/____/|__/
You are running this container as user with ID 6996 and group 6996,
which should map to the ID and group for your user on the Docker host. Great!
tf-docker ~>
nvidia-smi
Wed Sep
8 16:19:29 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.32.00
Driver Version: 455.32.00
CUDA Version: 11.1
|
|-------------------------------+----------------------+----------------------+
| GPU
Name
Persistence-M| Bus-Id
Disp.A | Volatile Uncorr. ECC |
| Fan
Temp
Perf
Pwr:Usage/Cap|
Memory-Usage | GPU-Util
Compute M. |
|
|
|
MIG M. |
|===============================+======================+======================|
|
0
Tesla K80
Off
| 00000000:8A:00.0 Off |
0 |
| N/A
24C
P8
27W / 149W |
0MiB / 11441MiB |
0%
Default |
|
|
|
N/A |
+-------------------------------+----------------------+----------------------+
|
1
Tesla K80
Off
| 00000000:8B:00.0 Off |
0 |
| N/A
23C
P8
30W / 149W |
0MiB / 11441MiB |
0%
Default |
|
|
|
N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes:
|
|
GPU
GI
CI
PID
Type
Process name
GPU Memory |
|
ID
ID
Usage
|
|=============================================================================|
|
No running processes found
|
+-----------------------------------------------------------------------------+
tf-docker ~>