Running Docker Containers on Jetstream

Getting Started

Docker is available on the newer Jetstream images (CentOS v7 & 8, Ubuntu v16, 18, & 20). You can one of these images to launch Docker or install Docker yourself. All docker commands on this page are run on an instance based on the Ubuntu 20.04 Devel and Docker image. Once installed, you can test your installation:

1 2 root@jetstream-docker:~# docker -v Docker version 19.03.8, build afacb8b7f0

Alternatively, you can test your installation with:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 root@jetstream-docker:~# docker run hello-world Hello from Docker! This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps: 1. The Docker client contacted the Docker daemon. 2. The Docker daemon pulled the "hello-world" image from the Docker Hub. (amd64) 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading. 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal. ...

Running Docker from prebuilt images

Let's start with the lightweight image - alpine. Alpine is a minimal Docker image based on Alpine Linux.

In the example below, we run a container based on the alpine image and list files. Since the image was not available locally, docker fetches the image from the Docker registry; saves a local copy; runs a container; lists files.

Example 1 - Run an alpine image

1 2 3 4 5 6 7 8 9 10 11 root@jetstream-docker:~# docker run alpine ls -l Unable to find image 'alpine:latest' locally latest: Pulling from library/alpine 4c0d98bf9879: Pull complete Digest: sha256:08d6ca16c60fe7490c03d10dc339d9fd8ea67c6466dea8d558526b1330a85930 Status: Downloaded newer image for alpine:latest total 56 drwxr-xr-x 2 root root 4096 Jan 28 21:51 bin drwxr-xr-x 5 root root 340 Feb 11 21:07 dev drwxr-xr-x 1 root root 4096 Feb 11 21:07 etc ...

Note: If a version (tag) is not provided, the ‘latest’ version will be fetched.

docker pull alpine command can also be used to fetch the latest version of alpine.

docker ps shows all running containers.

docker ps --all shows all containers; previously run and currently running containers. You can also get your container id with this command.

docker rm <container id> deletes containers.

docker run -it alpine sh runs an interactive tty session.

1 2 3 4 5 6 root@jetstream-docker:~# docker run -it alpine sh / # ls bin dev etc home lib media mnt opt proc root run sbin srv sys tmp usr var / # / # exit root@jetstream-docker:~#

You can leave the container with the exit command.

To keep the container active you should use ctrl + p, ctrl +q . This will leave your container without actually quitting.

1 2 3 root@jetstream-docker:~# docker ps --latest CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 2eb2d1a43d5f alpine "sh" 2 minutes ago Up 2 minutes keen_wright

docker attach <container id> will let you back in the container.

Note: You cannot attach a stopped container.

Example 2 - Run a Jupyter image

docker search jupyter searches Docker Hub for images. In this example we use the jupyter/minimal-notebook image one of the official Jupyter Docker images.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 root@jetstream-docker:~# docker search jupyter NAME DESCRIPTION STARS OFFICIAL AUTOMATED jupyter/datascience-notebook Jupyter Notebook Data Science Stack from htt… 790 jupyter/all-spark-notebook Jupyter Notebook Python, Scala, R, Spark, Me… 332 jupyter/scipy-notebook Jupyter Notebook Scientific Python Stack fro… 283 jupyterhub/jupyterhub JupyterHub: multi-user Jupyter notebook serv… 276 [OK] jupyter/tensorflow-notebook Jupyter Notebook Scientific Python Stack w/ … 259 jupyter/pyspark-notebook Jupyter Notebook Python, Spark, Mesos Stack … 185 jupyter/base-notebook Small base image for Jupyter Notebook stacks… 135 jupyter/minimal-notebook Minimal Jupyter Notebook Stack from https://… 123 jupyterhub/singleuser single-user docker images for use with Jupyt… 39 [OK] jupyter/r-notebook Jupyter Notebook R Stack from https://github… 36 ...

Since Jupyter runs on port 8888 we need to expose the port.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 root@jetstream-docker:~# docker run -p 8888:8888 jupyter/minimal-notebook Unable to find image 'jupyter/minimal-notebook:latest' locally latest: Pulling from jupyter/minimal-notebook 83ee3a23efb7: Pull complete db98fc6f11f0: Pull complete ... ... [I 16:18:58.609 NotebookApp] Jupyter Notebook 6.2.0 is running at: [I 16:18:58.609 NotebookApp] http://fd4bcf198bc5:8888/?token=fd9ae5ddc2b1616ae915b50621808e0553b6f15a6cbe9139 [I 16:18:58.609 NotebookApp] or http://127.0.0.1:8888/?token=fd9ae5ddc2b1616ae915b50621808e0553b6f15a6cbe9139 [I 16:18:58.609 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). [C 16:18:58.615 NotebookApp] To access the notebook, open this file in a browser: file:///home/jovyan/.local/share/jupyter/runtime/nbserver-6-open.html Or copy and paste one of these URLs: http://fd4bcf198bc5:8888/?token=fd9ae5ddc2b1616ae915b50621808e0553b6f15a6cbe9139 or http://127.0.0.1:8888/?token=fd9ae5ddc2b1616ae915b50621808e0553b6f15a6cbe9139

You can use one of the links above to access your Jupyter notebook.

Note: You must replace the container id - fd4bcf198bc5 or localhost address 127.0.0.1 with the IP address of your Jetstream VM.

Building a Docker Image

You can build a Docker image by reading instructions from a Dockerfile. A Dockerfile contains a list of commands that the Docker daemon calls while creating an image. A Docker image consists of stacked read-only layers of instructions from the Dockerfile.

Here’s an example of a Dockerfile:

1 2 3 4 FROM alpine:3.12 RUN apk update COPY app.py /app CMD python /app/app.py

Each instruction creates a layer :

FROM creates a layer from the alpine:3.12 Docker image.

RUN installs packages required by the alpine distribution.

COPY copies files to your destination folder.

NOTE: You can include an additional RUN after COPY to build any applications copied over.

CMD specifies the command to run within the container.

 

Example 3 - Deploy your app in a Jupyter image

What we need:

  • Application

Login to your Jetstream VM. Create and cd into a directory named mynotebook.

1 cd ~ && mkdir mynotebook && cd mynotebook

Create your application in the directory mynotebook.

1 2 def introduce(name): return 'Hello ' + name
  • Dockerfile

1 2 3 FROM jupyter/minimal-notebook COPY model.py /home/jovyan/work/ EXPOSE 8888

 

Data Management

Docker offers multiple ways to mount data into a container from the docker host.

Docker Volumes

Docker can create a volume in the container when the container is built.

Example:

  1. Here we create a volume named dataVolume1 and a container named Container1 and exit the container.

1 2 3 4 root@jetstream-docker:~# docker run -it -v /dataVolume1 --name Container1 ubuntu root@6671fd54a47d:/#exit exit root@jetstream-docker:/#

2. If we inspect the container we see dataVolume1 listed in the Volumes section.

1 2 3 4 5 6 7 8 9 10 11 root@jetstream-docker:~# docker inspect 6671fd54a47d ... ... "Image": "ubuntu", "Volumes": { "/dataVolume1": {} }, "WorkingDir": "", "Entrypoint": null, ... ...

3. We can restart and attach the container. We see dataVolume1 when we run the ls command.

1 2 3 4 5 root@jetstream-docker:/# docker restart Container1 Container1 root@jetstream-docker:/# docker attach Container1 root@6671fd54a47d:/# ls bin boot dataVolume1 dev etc home lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys tmp usr var

4. We can cd into the dataVolume1 directory and create or edit files. You cannot access these files if you exit the container.

1 2 3 4 5 6 7 8 9 root@6671fd54a47d:/dataVolume1# ls root@6671fd54a47d:/dataVolume1# cat > Newfile This file is on a volume in the container. ^C root@6671fd54a47d:/dataVolume1# ls Newfile root@6671fd54a47d:/dataVolume1# exit exit root@jetstream-docker:/#

Bind mount

When you run a container, you can bring a directory from the host system into the container, and give it a new name and location using the -v or --volume flag.

Example:

  1. In this example, the instance was launched on Exosphere and we see the mount point for the external volume attached is /media/volume/sdb.

2. Log into the instance and execute your docker commands as root. Here we create a container named Container2 and a docker volume named datavolume is mounted at the mountpoint for your external volme.

1 2 3 root@jetstream-docker:~# docker run -it --name Container2 -v /media/volume/sdb:/datavolume ubuntu root@ab425c22cc2e:/# ls bin boot datavolume dev etc home lib lib32 lib64 libx32 media mnt opt proc root run sbin srv sys tmp usr var

3. You can then create or add data to this volume.

1 2 3 root@ab425c22cc2e:/# cd /datavolume/ root@ab425c22cc2e:/datavolume# ls lost+found root@ab425c22cc2e:/datavolume# touch some.txt

4. When you exit your container, the contents of your docker volume will still be accessible outside your container.

1 2 root@jetstream-docker:/media/volume/sdb# ls lost+found some.txt